Saturday, 6 March 2010

UnPacking Malicious Executables

This document provides instructions on how to unpack NsPack 3.7 using the Ollydbg debugger. It forms a small sample of a paper I am working on for a SANS gold submission for the GREM certification.

The Ollyscripts used in this process will be included in the GREM paper. Where custom plug-ins are required, these have been noted. In Malware analysis (e.g. Worms and RootKits), code is frequently ‘packed’. Packing is a process where the executable is compressed and is hence is difficult to analyse (for instance a strings search will return little information). The packed file is called from a new section which is a unpacking routine.

The process in this post also includes instructions on how to fully restore the import table so the file can be restored to its original state and executed.

Tools required:

  • OllyDebug v1.10
  • OllyDump plug-in
  • Import Reconstructor 1.6
  • OllyScript Plugin

NsPack 3.7 is a simple compressor. It does not support Anti-Debug or Anti-Disassembly features. It used configurable section names (defaulting to .nsp).

Stage 1

The first step is to validate that the correct packer has been used on the samples that we are analysing.

In the case of NsPack, 2 tools are used to ensure that this was correct prior to starting execution. These are:

  • LordPE (
  • RDG Packer Detector v0.6.6 2k8

In each case the samples have been validated as NsPack 3.7 compressed executables.


Figure 1: PEiD determination of NsPack 3.7


Figure 2: RDG determination of NsPack 3.7

In each case, the samples have been validated as correctly being packed using NsPack version 3.7.

Unpacking in Olly

The following section details the process to unpack NsPack 3.7 compressed executables using Olly.

To load and extract the packed executable images:

1. Get the OllyDbg program from

2. Get the OllyDump plugin from (this is provided with source as a deliverable)

3. Extract the file, ollydump.dll file into OllyDbg's plugin directory (e.g. C:\Reversing\Olly\Plugins).

4. Run OllyDbg,

5. Click File->open,

6. Select the executable to unpack.

It is also good practice to ensure the Plugin has been correctly loaded. To do this:

1. Click File->Plugins

2. Check that OllyDump has been loaded (this is displayed in the image below).


Manual Unpacking

With OllyDebug running and the OllyDump plugin loaded (see above), click “F3” to load the packed sample. An alert should appear noting that the sample is packed. Click ok (see the image below).


The packed sample will now be loaded into Olly. Note that a warning that the sample is packed is again displayed. Again, click ‘Yes’ to continue.


At this point, the packed executable has been loaded into Olly. The executable should not be running, but should at this stage be “paused”.

Next, enter “F8” (select the F8 function key) to step through the packed sample.


At this point you should notice that the Registers have changed (see image below).


From the image above, you can see that the values in the registers have changed (the original load is on the left, with the alteration subsequent to hitting F8 on the right).

Right Click the ESP register and select “follow in dump”.


This will provide a dump of executable section that we are going to follow in order to find the “OEP”.

This dump is displayed in the figure below. Note the data contained in the Hex dump field displayed in the window at the lower left of the screen.


The dump below contains the address of the ESP register. You can see that we have highlighted the initial four (4) byte values (as displayed in the figure below with the values highlighted in grey).


Using these values we want to set a hardware breakpoint. We do this using the following setting:

“Breakpoint -> Hardware, on access -> Dword”

You do this by selecting the highlighted values above and right clicking. This process is displayed in the image on the following page.

Setting a hardware breakpoint allows us to follow the execution of the program to this point and then to stop (or interrupt) the execution of the program.


With our breakpoint, we want to hit the “F9” function key to “run” the executable until it hits the breakpoint that we have set. This takes us to a jump command. This is displayed in the figure below:


We can follow this jump by entering “F7” to “step into” the command. This will allow us to run a single machine code command and hence to follow where the jump command takes us.

After the jump, you will notice that some of the code looks strange (see the figure below).


Enter “Ctrl-A” which will analyse the data and treat is as code. It was not treated as code previously, because before this was all data. When the unpacker executed, the code was written to these memory locations.


Once this process has completed, you will note that the code is far easier to understand. This is displayed in the previous figure.

Next we want to dump the process.

To do this, we will use the OllyDump plugin.

To do this, select:

  • Plugins -> OllyDump -> Dump Debugged Process

This process is displayed in the figure below:


When the plug-in displays, unselect the “Rebuild Import” option.


At this point we will not use the Rebuild method from the OllyDump Plugin.


Select “Dump”.

Then select the file to save the dumped executable as.


By loading this into PEiD we can see that the file is no longer packed:


However, we have not fixed the IAT and hence the executable will not run as yet.


As such, we need to fix the IAT.

To do this at this stage, we will run ImpRec.

First, attach ImpRec to the running process (as displayed in the figure below).


Notice that the OEP is not correct. Remember, the OEP was supplied using OllyDump (above):


As such, we need to fix up the OEP in ImpRec:


Then select “IAT AutoSearch” to continue.

When ImpREC finds the value, it will display a message, click on “OK”:


Next, get the imports. This is done by clicking “Get Imports” on the lower left of the screen:


We can see from the image above that all of the imports have been found successfully. This is demonstrated by the “valid: Yes” flag in the “Imported Functions Found” field.

As ImpRec has correctly determined these values, we need to fix the dump. To do this, look at the lower right-hand side of the screen and select “Fix Dump”.

You will be presented with the location of where you want to save the repaired and unpacked executable.


Enter the name of the dumped executable that you are fixing and select open.


As is displayed above, the log should show that the unpacked executable was saved. In this case (and this is not unusual) the unpacked executable is larger than the original file (before it was initially packed).


We see from the figure above, that “cmd.exe”, a file that was initially 312Kb in size was packed to just 148Kb, but when it was unpacked, it has grown to 456Kb.

The unpacked file also runs correctly now that the IAT has been repaired:


Automation with OllyScript

To automate this process, we use OllyScript. We begin by loading the sample into OllyDbg (as occurred in the previous section).

Start by opening OllyDbg. Go to:

  • Plugins -> OllyScript -> Run Script -> Load...

This has been displayed in the image below:


To do this, OllyScript needs to be installed before you open Olly. Again, as in the last section, ensure that you have copied the plug-in to the correct directory.

Load the script first, and then open the sample (as in the first instance.


Again, we are at the start of our packed executable.

This time, instead of manually finding the OEP, we will use the script that we loaded. To do this, go to:

  • Plugins -> OllyScript -> Run Script -> C:\Data\HCL...


Select the correct script that is loaded into the OllyScript plug-in. The script used in this instance has been included in the Appendix (below).

We will first get the statement that this code is packed again. Select “Yes” to continue.


The script should complete and return the following message:


Click “OK” to continue.

At this point, we should find that we are at the OEP.


At this point, we will dump and reconstruct the IAT in the same manner as in the previous section.

Summary of the process

The summary of the method to uncompress NsPack in OllyDbg is:

1. At entry point, add a breakpoint in the PUSHA instruction and run the application.

2. After it breaks, follow the ESP register value in dump, add a hardware breakpoint with 4 bytes length in the first bytes.

3. Run the application again (F9).

4. At the next break (BP), the EIP will be at the transfer command.

5. Simply single step into it (F8) and the value at EIP will be at the original entry point.

Analysing NsPack itself

Looking at the NSPack executable, and using PEiD we see that NsPack is itself packed using ASProtect Version 2.1.x.


It is also possible to quickly get the OEP of NsPack 3.7 using PEiD:

Plugins -> Generic OEP Finder


In this case we have the OEP returned at OEP: 004897F7.


Next, we start Olly. The following plug-in is essential:

IsDebugPresent API

AsProtect has a debugger detection routine. The plugin is needed to ensure that the program does not crash prematurely. This file is available from OpenRCE:

To load and enable this plug, go to:

  1. Plugins, IsDeBugPresent
  2. Select “option”


The auto-hide function should be set to match the load times of the host running the analysis.


Next, select whether to automatically hide the debugger not (Autohide). If checked when you load an exe, debugger is hidden and you can choose how long thread will sleep until patch byte API is done (Sleep Time).

It is also possible to manually hide or restore debugger with menu option. Ensure that the exceptions have been disabled (other than Kernel32 Memory access violations) by entering “Alt-O” in Olly and removing any ticked boxes:


Quit and load NsPack 3.7.

Enter “F9” to ‘run’ the program. This will take us to the first exception:


We will need to count the number of exceptions that are returned. With this information we can restart and step directly to the final exception, BP on the code section (where we should reach the OEP).

Next, dump the program and repair the IAT.

To do this, enter “Shift + F9” for each returned exception. Remember to count the number of exceptions returned.

Skipping past the exceptions, we can interact with NsPack:


From this we have the message that the executable is packed. This was already known, so simply select “yes” to continue.clip_image088

Olly has now loaded the module and is awaiting our input. We should go directly to this point if the plug-in (IsDebugPresent) loaded:


The reason for counting the exceptions was to be able to jump directly to final exception.

Select “M” (See below circled in the upper left). This gives us the Memory map below:


We now set a breakpoint on the section ‘code’ (do this with the mouse – below, or by entering ‘F2’):


Using the standard techniques, we can then rebuild the IAT.

Transfer Command

00000000 61 POP A

00000001 9D POP F

00000002 E9 ?? ?? ?? ?? JMP <value>

Entry Point Signature

00000000 9C PUSH F

00000001 60 PUSH A

00000002 E8 00 00 00 00 CALL 00000003

00000007 5D POP EBP

00000008 83 ED 07 SUB EBP, 7

0000000B 8D ?? ?? ?? ?? ?? LEA ECX, [EBP-value]

00000011 80 39 01 CMP Byte PTR [ECX], 1

00000014 0F 84 ?? ?? ?? ?? JZ value

9C 60 E8 00 00 00 00 5D 83 ED 07 8D ?? ?? ?? ?? ?? 80 39 01 0F ?? ?? ?? 00 00

Hence a simple signature could be defined as:

[NSPack 3.7 -> Liu Xing Ping]

signature = 9C 60 E8 00 00 00 00 5D 83 ED 07 8D 85 ?? ?? FF FF ?? 38 01 0F 84 ?? 02 00 00 ?? 00 01

ep_only = true

Basic Details of NsPack 3.7

In general, NsPack’d files report having three sections (.nsp0, .nsp1, and .nsp2). This is user configurable and these can be set to any value. Consequently, the Entry Point Signature (above) is a better means of detecting NsPack than simply using the section headers alone.

PE Structure information

PE Info returns the following information above a generic NsPack compresses file.

( base data )

entrypointaddress.: 0x7b48e3

( 3 sections )

name viradd virsiz rawdsiz ntrpy md5

.nsp0 0x1000 0x3b0000 0x0 0.00 d41d8cd98f00b204e9800998ecf8427e

.nsp1 0x3b1000 0xab000 0xaa6c3 7.99 bc5e2a11a697427c5ec95bb5cabea1dc

.nsp2 0x45c000 0x128b 0x0 0.00 d41d8cd98f00b204e9800998ecf8427e

Note: The section names are variable and can be set to anything by the user.

( 1 imports )

> KERNEL32.DLL: LoadLibraryA, GetProcAddress, VirtualProtect, VirtualAlloc, VirtualFree, ExitProcess

As noted, the user can change the section names from the default ‘.nspX’ value.

The first section is unpacked.

Calculating the PE File Execution Start Offset in NsPack’d files

In the image below, we see the header information of a typical program that is packed using NSPack. This first example uses the standard options and naming for the section headers.


This file also has the following Optional Header Section:


With the standard NsPack section naming conventions.


The address of entry point that is stored in the optional header is a relative virtual address (RVA), where the loader will begin execution. An RVA is simply the offset of an item, relative to where the file is memory-mapped.

A comparison of the unpacked Notebook.exe and an NsPack version of the same are displayed below loaded into Protection ID to display the section and header values.


And the unpacked version:


The following are the basic stages used to get to the file execution start offset:

1. Determine each section’s virtual memory map (that is the virtual start address and end address. The virtual address and virtual size for each section can be found in the section header from the executables PE Header).

2. Establish in which section’s virtual space the address of entry point is located.

3. Validate the offset of that section as per the section header. In the section header the pointer to raw data field gives us the file-based offset where the section data/bytes begin.

4. Calculate the difference between the address of entry point and the virtual address of the section in which the entry point lies. Add this difference to the pointer to raw data, which is the file-based offset of the section, in order to get the file-based execution start offset for the particular file.

Hence, using this data we can calculate the file execution start offset for this file:

[ (Address of Entry Point) – (Virtual Address) ] + (Pointer to Raw Data)

= (file execution start offset)

The ‘Pointer to Raw Data’ value is also called the ‘Offset’ or ‘Raw Address’. Now, by inserting the values from our tables above, we get (these values come from the .nsp0 section header and the main optional headers):

(0x000380F9 - 0x00001000) + 0x00000400 = 0x00038CF9

This calculated value is not necessarily the offset where file execution actually begins with NsPack compressed files.

If we take another example, in this case packed with several NsPack options applied, we get a different type of calculation.

File Optional Header

Number of sections: 02 Section alignment: 00001000

Address of entry point: 00001010 File alignment: 00000200

Image base: 00400000

Section Headers

Section Virtual Virtual Size of Pointer Characteristics

name size Address raw data raw data

nsp0 00004000 00001000 0000000B 0000001C E0000060

nsp1 0000203D 00005000 00000CFD 00000200 E0000060

Hence we have:

(0x00001020 - 0x00001000) + 0x0000001B = 0x0000003B

In Windows, the loader rounds the pointer to raw data to 0x00000000 as it is lower that the ‘file alignment value’ (in this example = 0x00000200). As a consequence, the loader assumes that the first section, nsp0, starts at file offset 0 and loads the section accordingly in the memory. So if we round the pointer to raw data, as the loader does, the file execution start offset is calculated as follows:

(0x00001020 - 0x00001000) + 0x00000000 = 0x00000040

The offset 0x00000040 is located within the DOS header of the PE file. Hence this means that it can land within the reserved section of the DOS header (this section is normally filled with zeros). From this location, NsPack inserts a five-byte jump instruction. The reason is that this will transfer control to code further ahead in the program.

Note: It is essential that a check is implemented for occasions where the pointer to raw data is not a multiple of the file alignment. In these instances, this value needs to be rounded to the nearest multiple and the remaining extra bytes should be passed over. For files whose file alignment value is not 0x00000200, the loader rounds it to a multiple of 0x00000200.

As noted, the section header names are variable and as can be seen in the section header table displayed below, these can easily be changed (with a flag in the program) to a different set of values.


Also note that section ‘.nsp1’ (or its equivalent if renamed) can extend beyond the raw file offset of section ‘.nsp2’

The decompression algorithm

NsPack uses a single format for compression/decompression. There seems to be little difference between the versions of the program for this function.

The initial section of all NsPack 3.7 compressed executables is .nsp1 (or the renamed functional equivalent) with first bytes, 9C,60,E8,00.

The basic layout the routine is:

PUSHAD /* PUSHAD saves all the register values onto the stack */

/*de-compression routine here (see appendix)*/

POPAD /*POPAD restores the previously saved data */

/* from the stack to the registers */

JMP OEP /* The Real Original EP */

The Jump to the OEP is made after the de-compression has run and the executable code has been decompressed.

The de-compressed code is in effect the original code and it does not have a record that any additional code has been executed prior to arriving at the OEP. The reason for this is that the instructions in the de-compressed code are expecting certain values. These may conflict with any errors that might result from variations in the register values. Consequently, the only instruction that will interact with the values placed on the stack by PUSHAD is the final POPAD instruction.

The full process will be released soon in a paper I am working on at present.