Build your own AI based Dynamic Reversing Lab, x64dbg automate
Watch on YouTubeVideo summary
This video tutorial guides viewers through building a dynamic analysis laboratory powered by artificial intelligence, specifically designed to automate tasks within x64dbg on a Windows virtual machine. Building upon a previous static analysis setup involving a REMnux machine and large language models, this session focuses on connecting the Linux-based analysis environment to a Windows 10 VM to enable debugging and unpacking capabilities. The process begins with installing OpenSSH on the Windows machine and configuring network settings, such as using host-only adapters, to ensure secure communication between the two virtual machines without exposing them to the public internet. Viewers are instructed to set up firewall rules, manage RAM allocation for both systems, and configure SSH access so that the AI assistant can remotely execute commands and monitor malware samples on the Windows platform.
Once the network connection is established, the tutorial details how to integrate an MCP (Model Context Protocol) server into the CloadAI framework running on REMnux, allowing the AI to interact with the Windows VM via SSH. The presenter demonstrates troubleshooting steps for common connectivity issues, such as adjusting SSH configuration files to enable password authentication and regenerating keys. After successfully connecting the systems, the video explores how the AI can leverage command-line tools like Sysinternals suites to perform dynamic analysis tasks that would otherwise be difficult to automate. This includes running scripts to deobfuscate code, dumping strings from applications, and utilizing instrumentation techniques to analyze behavior in real-time, effectively turning the Windows VM into a remote execution environment controlled by the AI model.
The core of the demonstration involves installing and configuring an x64dbg automation plugin on both the Windows machine and the REMnux Python environment. The presenter walks through downloading the necessary 64-bit and 32-bit plugins, installing Visual C++ runtime libraries, and setting up a dedicated virtual environment for the AI to use. By connecting this specialized MCP server to the AI model, the system gains the ability to automatically unpack packed malware samples, such as those compressed with UPX or encrypted using specific stubs. The video shows the AI successfully identifying an entry point, bypassing anti-debugging techniques, and extracting the original payload, which is then transferred back to the REMnux machine for further static analysis using tools like PE-bear.
The tutorial concludes by emphasizing that while this AI-driven lab significantly accelerates the reverse engineering process, it does not replace the need for human expertise. The presenter explains that large language models can handle repetitive tasks like unpacking and logging, but a skilled analyst must still verify the results to ensure accuracy and validate the findings. The video also offers insights into how packed samples often do not exhibit high entropy because encryption only affects specific portions of the data, leaving much of the file structure intact. Ultimately, this setup provides a powerful foundation for both professional and hobbyist malware analysts who wish to leverage AI to enhance their dynamic analysis workflows while maintaining rigorous oversight over the automated processes.
Read the full video transcript
Welcome to Malware Analysis for
Hedgehogs. Last week we set up a static
analysis lab with a large language model
and this week we are giving it
capabilities to debug files, to unpack
files with x64dbg
on a Windows VM. And also the basic
capabilities for it to run samples on
the Windows VM. For instance, it can use
PowerShell to start logging tools like
Sysinternals tools from the command line
and have some basic monitoring
capabilities and so on. So you have
basically, yeah, you have the setup for
dynamic analysis. And if you want to
learn how to analyze malware yourself,
check out my courses on
malwareanalysisforhedgehogs.learnworlds.com.
So what do you need to start here?
First off, you need to watch the last
video if you haven't already because you
need the REMnux machine that we set up
there. It contains CloadAI and it has
static analysis tools to analyze
samples. And this VM here will then
connect to the Windows 10 VM
which you also need. And this is a
standard analysis lab. I'm assuming that
most people who watch my videos already
have this because they analyze malware
either professionally or in their free
time. But if you don't have this
already, there is a free video series on
my website
malwareanalysisforhedgehogs.learnworlds.com.
So if you go there to the beginners
course
and the setup for the Windows 10 lab is
for free, so you can watch this fully
and
yeah, use this as a basis and come back
here and continue with the setup so that
we connect the two VMs.
Oh, and don't be confused. You don't
need the Windows XP machine. I just have
it still here because I made a video
about Virut. So now let's install
OpenSSH on Windows. And to do that, we
navigate to
Win32
Open
SSH
GitHub.
That should be correct. So we are on
PowerShell Win32 OpenSSH.
And we just go to the release page.
And we get our
zip archive here.
So here we are.
Now let's open PowerShell as
administrator.
Run as administrator.
We install it.
Start
service.
That works.
And set service, sorry.
Set service.
That's fine. And now let's see if
um
firewall rule was set.
There is none
set.
Okay, so we set the firewall rule
and
I think
it is
that
firewall
rule
for SSHD.
Now we have the rule.
And if we look for that, we also get the
rule. So this
is what it should look like.
So to allow REMnux to use the Windows 10
VM we need to change some of the network
settings. Firstly, this should not have
access to the internet for security
reasons.
Instead, we will just provide access for
REMnux.
And we use host-only adapter. Take note
of the name here so that we choose the
same one for REMnux.
And the rest looks all right. We click
okay. Now we go to REMnux and we do the
same except this one needs internet
access. So we go to adapter two.
Say enable and now we also set host-only
adapter. Now you need to check that this
is the same name as our Windows 10 VM.
And
yeah, we keep this.
And um
now let's continue.
Now let's boot both VMs.
You need to make sure that your host
system has enough RAM for both of them.
If you are tight on RAM, if you have
like 16 GB total
you may want to specify like um
8 GB for the Windows VM and 4 GB for the
REMnux VM which should be the default
for REMnux.
Um
So yeah, let's wait until everything
booted and then continue.
One last thing.
You need to check IP config for the IP
address.
And that's this one. Note it down for
the next step.
On REMnux, we will now add SSH MCP to
Cload
using Cload MCP add.
And now here you need to note down the
IP address that we just added.
And then you add the username and
password of the Windows VM.
So user
in my case is Alicia.
So just set your password here.
And now we should check if that works.
Check that, we will start Cload.
And we will check MCP.
And you can see now it is trying to
connect here.
And it says it's connected, so reconnect
worked for me.
And I'm going to tell it to test the
connection so that it uses that.
And it says now the remote SSH server
may be down. It can't access it. So you
need to recheck firewall and uh maybe
restart the server on the Windows VM
to do that.
Let's go back to the Windows VMs.
But before we do that, let's do some
basic checks first.
Let's just see if we can ping the
Windows
IP
address.
So, this works.
So, we can access it.
Now, let's try to use SSH for it.
And there, it says connection reset.
Let's now try some configuration
changes. We go to
C program data SSH. There is SSHD
config.
So, we have here
port 22.
We uncomment this one.
And then there should be password
authentication. Yes.
Let's do this. Let's save this.
And now you reopen
PowerShell.
And you say
restart service.
Okay, so let's do some more fixes here.
So, you see these files here.
Just some
Uh
let's run them.
We say yes to all.
Why doesn't it say yes to all?
Well.
And uh see
user file permissions. That's done.
And now let's regenerate the keys.
How we
like so.
And for debugging, we can run SSHD EXE
with minus D minus D.
And we get debug output here. We see the
server's listening. Now, I'm going to
switch to Remnux and check if I can
connect to this.
So, now you enter SSH username at key.
And we say yes.
Enter our password. Now, it works. Can
see now that we are on
Windows. So,
Yes, so we can
see that.
Run Claude.
Right.
Claude.
And now
check on the MCP.
Again,
test the SSH
MCP.
Let's see if Claude has access to it
now.
So, this worked.
And Claude can now use it to analyze
files on the Windows VM.
How can it use it? For instance, it can
execute scripts to deobfuscate them. It
can use uh for instance,
instrumentation to run
dump strings from electron applications
on OJS applications, things like that.
And you can dump some tools onto the VM
and then tell it to use these tools.
There are some command line tools for
instance for Sysinternals,
autoruns, and things like that. If you
tell Claude how to use them, where the
tools are,
and
you know, add some skills for these, and
then it's able to use these tools for
dynamic analysis as well.
Now, there's also a
an MCP for X64 DBG,
which we can also install next.
So, let's install X64 DBG automate. I'm
now on the Windows machine.
And I'm going to enable internet for a
short time so that I can
next.
Let's
install the plugin.
And that's on GitHub. Darius.
That's the right one.
We go to releases, and we need
the 64-bit version, I guess.
From this.
So, that's the 64-bit version. And
that's the 32-bit version.
Furthermore, you may
Let me go to the
installation page. You may see that
uh you need the Visual C++ runtime
redistributable. So, if you don't have
it,
also may want to install both of them.
But, I already have this.
So,
to install it, you go to X64 DBG
release, and then the respective plugins
folder.
So, to install this, we go inside of
this and just copy all of this.
And we do the same for 32-bit
plugins.
Okay.
We already installed
the libraries.
And now
if we run X64 DBG,
let me see if this works.
Let's just go with 64-bit.
Go plugins, and we see X64 DBG automate.
And we see here
settings. So, we can
change those.
The example
port
in the documentation is asked.
So, let me just use that one.
Okay.
And maybe the same one here.
I'm not sure if these are different
settings.
There. Let me make
check on them. It's not here. So, I
guess they are saved in here.
So,
let's search for automate. And here it
is.
You can also change the settings here
if you want to do that.
So, now let's install this on Remnux.
And
first,
we create a virtual environment. Let's
do this here in the analysis folder. We
end So, this is the Python environment
we're going to use.
And
the end.
And now we created it here.
To activate it, you make
source bin
activate. Yeah. Now, you see this VM
on the front.
and now we can install
x64 dbg automate
MCP
and also upgraded
And this is something we should probably
tell Claude that the VN for Python is in
there so that it always uses this.
Um
To do that we do Claude
and CP
And now we can see here x64 dbg
MCP is connected.
Now let's do the same thing again.
And
let's not ask again to connect to x64
dbg.
And it says it's connected. So this
should work now
and I'm going to test this with a small
example.
So my test sample is this sample right
here. It's
You can use any sample. Just pick
something with UPX.
In this case I have cryptbot and I'm
going to put it in here.
And uh
yeah, let's
go to the entry point and then
yeah, now we going to ask Claude to
unpack it.
Let's try this
and see what it tells us.
And this works. So now we can watch how
Claude unpacks our sample.
The process is gone.
In it run reset unsupported. Could you
reload the sample?
It is interesting.
So I um told it that it has access to
SSH MCP which we installed first.
And let's see if it
can now reload the sample on its own so
I don't have to do this anymore.
And here with SSH you can actually do
anything
on that machine. So you can
tell it to use Sysinternals tools for
instance and
yeah, monitor the sample
create logs
which is important that you provide it
with access to command line tools and
not just
graphical versions.
Seems it can't find the sample at the
moment. Maybe it's not there anymore.
Let me check.
It's still there. So this shouldn't be
the issue.
So it seems it now finished the
anti-debug to-dos and it's now running
to OEP of the unpacked payload. It's
nice that it tells me where
uh where it currently is.
Just to show you how this looks like on
the Windows 10 VM while
Claude is unpacking the sample.
This is what you see.
Most of the time you don't see anything
and sometimes window changes a little
bit. Usually it's rather boring to watch
but
more interesting to see what it writes
into the
Remnux VM. So here we just saw
a
command on the PowerShell terminal. So
that's why this popped up.
So um it's done.
It attempted to obtain the unpacked
sample for analysis and wanted to put it
onto the Remnux machine.
Um
but yeah
that looks good. Let's take a look at
the Windows VM.
Let's now take a look at the unpacked
sample.
And maybe let's compare this.
Going to run
Protux analyzer to create visualization.
And
here's the first part.
This is
interesting cuz this is this is the
packed sample here. We know it's packed
but it doesn't have high entropy.
At least not that high.
And if we compare both
we can see that this is basically almost
the same. So we have a different
structure here on the right side. The PE
structure, the the location of the
sections is a little bit different.
Um and on the left side everything is in
So this part here
the biggest part here is section
.
brackets one
Let's look a little bit closer here.
quite
revealing I think.
This is the same structure, right?
But what we can see is there are
different colors on the left. And when
you analyze how the unpacking stub works
it only decrypts every third byte.
So only every third byte is encrypted
and will change. The rest is just like
it was
well, like the original and that's what
we see here. So
here it's well encrypted
but most of the data
2/3 of the data is just like it is on
the right side where it's in a plain
text state.
So it does not need high entropy
for a sample to be packed because high
entropy is a
is an attribute of secure encryption.
It's not or of compression but it's not
necessarily an attribute of a packed
sample.
So you have now the basis for a very
capable analysis lab that is based on
large language models. So that's a very
useful tool. You can now do static
analysis and dynamic analysis.
And
as I said, it's a tool. It is not a
replacement of a reverse engineer. So
far
you still need the expertise of a
reverse engineer to verify that the
things the large language model fi-
figures out are correct and that the
approach is valid.
What's your experience with using large
language models in malware analysis?
Please write me a message below the
video. Also, if you have any questions,
please write below. And if you want the
sample, it's on sample.pdf.cc.
I will also put a video
link into the video description below.