DEF CON SG 1 - CSIT Village - Kan Onn Kit - Struct Reconstruction In Aether

Hi, hi. Can you hear me? Loud and clear? Okay. Hi, I'm Ankit. I'm a former cyber specialist who worked with CSIT on Aether and call that development. I'll be sharing on how we designed and architected the struct creation system that can be currently found in Aether. So, for those who missed our talk yesterday, Aether stands for the AI engine to help engineers reverse. It is an AI-powered reverse engineering co-pilot that assists reverse engineers with tedious reverse engineering tasks in their day-to-day at jobs. So, here's some background as to why we made Aether. Cyber incidents often involve malware, and this malware research and analysis team will conduct an in-depth investigation to understand the complexities of advanced malware. These technical insights will then be used for incident response and threat hunting. Now, we do have our in-house automated analysis platform called A-Cube to help triage known malware. A-Cube is based on the Cape V2 sandbox with and has worked well for us in the past. However, it is built and treat to triage and analyze common malware, and not specialized malware that we often see in cyber incidents. Manual reverse engineering is still required to handle these unknown malware. So, that's where Aether comes in. It acts as a force multiplier to assist our reverse engineers to find out what these threats do. To give an illustration of how Aether works, consider this code snippet. We can see that there are some magic string, and an experienced malware analyst will intelligently guess that this is some sort of magic bytes in the network communication. However, we can't really infer what this magic string means from this code snippet. A human analyst will need to go deeper into other functions to perform further analysis, and this is very time-consuming. So, after running Aether, this is how the code snippet will look like. We can see that the function and variables have been meaningfully renamed, and we can see that Aether has also added meaningful comments that that can easily read. Now, if we read the same code snippet again, we can infer much more things, such as the decryption algorithm, decryption key, packet signature, etc. The nested function logic is brought up for the analysis, and this reduces the effort on analyzing the malware. So, Ether now has three main groups of features that work in tandem with each other to help a human analyst generate accurate insights. So, first we have the AI annotations, which we have talked about in the previous talk yesterday. This helps rename functions and variables and set meaningful comments. It is also capable of performing function summarization, which overall helps improve readability of the pseudo code. But, this still has limitations. If you look at the V12 variable there, we can see it looks like some sort of struct-like variable that is hard for a human to read without creating proper structs. So, we went one step further to use AI to perform struct reconstruction. Now, you can see that there are many different fields there, and it has been properly renamed, and you can better understand what this code snippet is doing. So, we also recently started development on the Ether chatbot, which helps perform analysis across different functions and provide answers to user queries. So, this chatbot will tap on our context engine and benefit from our previous annotations made by Ether to give a more accurate and comprehensive analysis. So, for the rest of this presentation, let us go into a deep dive into how this struct reconstruction feature works. So, what's a struct? Structs, which are short for structures, are user-defined types that group variables of different types under one name. Instead of defining multiple variables, a struct allows one to package different data types into one single structure. For example, this first pseudo code has code that has multiple references and variables with unknown pointers that we is hard for analysts to understand and comprehend. So, if we can define an appropriate struct, such as this student struct defined here, the purpose of the pseudo code becomes much clearer. For example, this seems to be some sort of printing of what the student has and the date of birth, etc. So, structs are very useful in understanding what a piece of pseudo code does. And this is especially important and useful when we're analyzing malware. The moment that we can accurately define a struct for the current pseudo code, we should be able to understand the rest of the pseudo code better and how its output is used in other functions. So, before we decided to create our own bespoke struct creation system, we want to have a sensing of where the current open-source solutions are with their struct creation capabilities. We created a toy C++ program, uh student_info.c, that implements a student struct, like you see here, to test the abilities of the state-of-the-art models, as shown on the side here. We note that the courses field is not used in the original code, so we do not expect the LLMs to infer what its use will be in the pseudo code. So, currently, when a malware analyst wants to use AI to assist them in creating a struct, they would use a then state-of-the-art LLM, like GPT-5, along with a model context protocol, MCP, like IDA Pro MCP, which is a very popular uh model context protocol to interact with IDA. This allows the LLM to interact with the decompiler and to create the struct. However, LLMs are quite bad at making these structs. We can classify the so-called badness of these structs in three different ways. First, the accuracy. Even though IDA will graciously handle the addition of padding between fields when we declare them via the interface or using IDA Python, it does not add padding when we set it using IDA Pro MCP. This is a fundamental problem when LLMs use the tools that IDA Pro MCP provides. The misalignment of these fields, as you can see here, will cascade into the pseudo code, causing the generated struct to be inaccurate and the analysis to be wrong. Second, usability. A lot of hand-holding is required to direct these state-of-the-art models to do what we want it to do. GPT-5 seems to be able to formulate a decent strategy to gather the required components of the binary before reconstructing the struct, but we still needed to put in a lot of effort to develop a decent prompt for it to do so. Finally, speed. GPT-5 takes forever to run. It took at least 8 to 19 minutes to generate these results and would often take longer if the structs are more complicated or if major corrections are needed. In reality, struct reconstruction is very taxing. When malware analysts have tried to recreate structs, they would first do have to have a feeling for where the variables are and what variables are indeed structs, something that an LM is currently unable to do. To use an example, if V12 here was not highlighted, would you on initial guess that it is actually a struct? Furthermore, analysts would then have to identify all the areas where this same struct variable is used across all the functions and then make an educated guess of what the fields of the struct are. The biggest challenge that we face when creating structs is to maintain coherence across all the functions that use this struct. Even after making a guess of the struct, they would still need to check if the fields names agree with what the code is doing. This involves the analyst going into many, many functions to ensure that the names make sense, which takes a very long time. We believe that we can use an agentic workflow to automate these two steps. So, we took inspiration from our in-house reverse engineers to come up with a way to guide the agent to accurately recreate the struct. We decompose the problem of struct reconstruction into three main phases. The gathering phase would first find and gather relevant struct structs functions, sorry. They would first find relevant functions to the code that is relevant to the struct creator's context. This context is different from Aether's annotator's context. Here, we're just focusing on the pseudocode that's relevant to reconstructing the current pseudo struct variable. Next, the applying phase will then use appropriate tools exposed by the IDA Pro MCP to define and set the struct within IDA's local types. Finally, the assessing phase will check if the set struct matches the existing context by making the agent reflect on its actions. Alas, although such a high-level architecture should work in theory, problems arose when we tried to implement it as is. When we tested Aether, we realized the ability for LLMs to reason is critical to obtain accurate annotations and struct creations. We knew that hallucinations or inaccuracies in the annotated output would reduce analyst trust in Aether, and so we needed a way for the LLM to assess its own correctness on the fly. So, reasoning LLMs seems to be a great fit for our needs. However, these LLMs are not without issue. Our experiments indicated that many open-source LLMs often produce bubbles and irrelevant reasoning that often repeats what was derived or ends up looping endlessly for thousands of tokens. As you can see here, this could also result in the LLM crashing out. So, another desire was for the LLMs to be imbued with the reverse engineering mindset and knowledge. This was sorely lacking in current SOTA, sorry, state-of-the-art reverse engineering LLMs, and we needed a lot of additional prompting to direct the model to perform its analysis like a reverse engineer. So, to solve both of these issues, we turned to a tried and true method we introduced way before the boom in LLMs, chain-of-thought prompting. First described by Wei et al. in 2022, chain-of-thought prompting is the method of imbuing the model with a series of reasoning steps that we wanted it to consider when solving a problem. Of course, reasoning LLMs typically have reasoning built in, but chain-of-thought prompting allows us to have finer control over the quantity and quality of the model's reasoning. Our desire for a more directed, controlled, and relevant thought process can be achieved by replacing the reasoning phase of reasoning LMs with chain-of-thought prompting. So, as an example of how we did this, let us examine part of the chain-of-thought prompt of the setter. Remember, the setter's job is to take a human-readable guess of the C struct and add appropriate padding for setting within IDA. So, the focus of the setter LM should be only on adding padding, not checking for the correctness of the offsets or the accuracy of the names. This is reflected in this preamble of the chain-of-thought prompt, which asks to only identify the fields and the offsets in the struct. It does not ask whether the names make sense, just that the offsets are correct. Of course, we also provided LMs with an example of a reasoning attempt that we felt was applicable for what it should be doing. With chain-of-thought prompting, we can see an obvious change in the thinking of the LMs. Before chain-of-thought prompting, the default thinking mode of the LMs is quite dense. It generates a lot of reasoning and explanations, but a significant portion of the thinking is too verbose and unhelpful for the current task. Sometimes, the LM would even repeat itself and go into circles. With chain-of-thought prompting, you can see that the thinking portion here is sparser, and we can even see a little bit of its initial analysis beginning at the bottom of this image. This allows us to reduce token usage of by the LM and direct its reasoning and processing to what we want it to do, which ultimately also reduces analysis time. So, even with the use of chain-of-thought prompting, there were some issues with the idealized architecture that we dreamed of having. Our architecture had to evolve to incorporate other desires that were not present in the initial design phase. These considerations meant that our idealized form of the struct creator had to evolve with our needs. After several months of R&D and testing, here's what the struct creator became. From our idealized architecture of five nodes, our struct creation system blew up and became this behemoth that you can see on the right. This increased complexity is also needed to deal with the increase in control that a human analyst has on the system and to deal with the intricacies with IDA's API. Our struct creation system now also uses a non-reasoning LLM, country coder for ATB 825B, which is a result of our switching to chain of thought prompting. So, let us zoom into this applying phase for a second. Why is it so complicated? Well, our initial idea for just a single guess node that creates a human-readable C struct and then getting the setter to add appropriate padding before setting in IDA has some issues. Due to the inherent randomness of LLMs, the struct that was set in IDA could have some issues such as incorrect padding or syntax errors. Now, it is quite wasteful to go through the entire loop just to fix these issues that can be corrected immediately. So, that's why we have an inner loop that you can see here, returning back to the setter if the struct has issues that can be fixed immediately. So, zooming out again, the difference in these two loops, the inner loop within the applying phase and the outer loop between the assessing and gathering phase, is their purpose. The inner applying loop focuses on struct level issues such as the padding and type issues mentioned previously. The outer assessing loop focuses more on the context level issues. This agent is responsible for checking if the coherence is maintained across all the functions when setting the struct. So, does the struct fit the given code context? Does the code access any reserved fields? These issues and more are what's checked by the assessing phase and this outer loop. So, to check whether IDA struct creator beats IDA Pro MCP and other SOTA LLMs, we use the same student info.c sample as before, beginning the analysis from the same function and directing the LLM on which variable to create the struct for. So, here's Aether's output on the tower.c sample when compared to original sample. Based on the generated structs, we can see that the fields offsets in Aether's struct are correct with appropriate padding inserted to maintain alignment. The accuracy of the names and types is also maintained despite using a non-reasoning LLM. This quick test shows that our solution has addressed many pain points with IDA Pro MCP, even though we use country coder, which is a much less more much less powerful model as compared to GPT-5. So, let us compare Aether's performance with IDA Pro MCP when using GPT-5. Aether is generally more accurate with IDA Pro MCP with issues regarding padding and offsets occurring less frequently than with GPT-5 and IDA Pro MCP. The struct creation system is also integrated with Aether's other features, allowing for one-click struct creation without the need of hand-holding or prompting. Aether is also way faster than IDA Pro MCP, taking only a fraction of the time as compared to GPT-5. It should be emphasized, though, that the performance of this struct creator heavily depends on the quality of Aether's output of Aether's annotations before activating the struct creator. So, how well does this struct creation system perform when dealing with real malware? We tested our system on the White Fly APT malware sample found in 2018. In particular, we looked at the second stage of the malware, which was responsible for interacting and communicating with the White Fly C2 server. Our internal malware analyst already generated a report of the sample, so we could reference it and make sure that Aether's output agreed with what's known internally. The White Fly sample first needed some manual patching of the IAT, the in- import address table, within the IDA project. Once that's done, we asked IDA to generate annotations for the functions in the binary to allow our struct creator to have sufficient context for determining the structs used within the binary. Without these annotations, the struct creator would not be able to function well. So, that is important to note here. Within the binary, we noticed several functions that seem to use some structs. For example, within the AIRS.C2 communication function found at 401 to E1, we see that V4 seems to be some sort of hook struct. We ran the struct creator on V4 and saw that the generated struct was decent. When we cross-reference the struct with our internal report, we found that it matched our own analyst's findings. So, looking further into that function, we see that V14 also appears to be some other struct. This time handling some sort of network communication. Even though our generated struct, as you can see here, looks quite barren, it surprisingly matched our own internal report on what this struct is. For the struct to be more complete, however, some manual intervention is needed. For example, this identified V table field ideally would have been populated with pointers to the functions. However, as of now, IDA is unable to populate this V table because it cannot dive into the memory of the binary. So, analysts still have to go into the referenced memory locations to identify and fill in what's in the V table manually. Regardless, we can see that the struct creator got the fields right. So, what about structs with more fields? So, let's look at the pseudo code that has been teased at the start of the presentation. This is the AIR perform injection function at 4029C7. In here, we see that V12 seems to be some sort of payload with decryption and fields that depend on the architecture of the system running the malware. So, the struct creator seemed to have some humor because this is its first attempt. I would do I think you will agree that you want more detail than just generic fields. But it's not technically wrong. The subsequent attempts were better with it inferring fields that mainly match what was found in the binary. One issue though was that this command base address or the command base address above the reserved three field should actually be some sort of encryption key such as encryption key 32-bit. The larger issue with the struct creator is that the type of the variable V12 that the struct creator decided on was a struct pointer instead of a struct type. This affects the quality of the generated pseudo code and the assessor's ability to check the correctness of the generic of the guessed struct. So once we set the correct type on the right, the pseudo code became much cleaner. Albeit with some minor errors like the command base address issue mentioned above and a reserved six access that you can see over there. So looking back at our architecture, we could see that there's some issues with our approach. One big problem is with correctly and accurately identifying the offsets that each field should be at since LLMs are inherently bad with numbers. Our current approach also doesn't work well with nested structs. We really wanted to fix the issue with incorrect offsets so we set out to redesign the struct creation tool by changing the focus of what the LLMs does. While our old approach tried creating the struct as a C-like manner and asking the LLM to add appropriate padding, our our revised approach would ask the LLM to identify appropriate parts of the pseudo code that indicate offsets of the fields. Then either Python can automatically resolve padding within IDA. This shift would involve splitting the existing guesser node into two sub nodes. The identifier sub node would identify possible nested structs within the main struct and the field guesser would identify the offsets and types of the fields within each struct. So, as a comparison, here's the different versions of the struct creator system. The architecture of the first version is much more complex with many sub nodes and two loops to achieve the struct creation task successfully. The new and improved version on the right is much simpler and only requires one loop to determine the fields that are present in the struct and to set it within IDA. When we ran the new struct creation system on the V12 variable again, we see that the variables Sorry, the issues that we found in with the variables previously have been fixed in the new version. The previously named V6 Sorry, reserved 6 access has been changed correctly to check if it's a 64-bit system and the command base address has been correctly renamed to be a decryption key for 32-bit architectures. So, with that, we have come to the end of my talk. Uh if you're interested in seeing uh further developments on the new struct creation system or to try out what IDA can do, we are excited to announce that it's open source. You can find our GitHub page or scan the QR code to access it and we would love to hear your feedback on IDA. You can also try out IDA on the booths at the back, which also is works in tandem with AQR. And with that, I wish you have a great day ahead and enjoy the rest of the conference. Thank you.