Tuesday, 7 July 2009

A dry run on NGrep

In this post I will go over a dry run on using NGrep. We will start with capturing a file using TCPDump into a PCAP format and then reading this into NGrep to extract the information we seek.

Step 1
The first stage is to capture all of the data we need into a PCAP file. For this we will use TCPDump. To do this, we are going to capture all traffic (with the complete payload, '-s 1500') to a file called capture.pcap.

tcpdump -nn -i eth0 -s 1500 -w capture.pcap
The command above and in the image below is what we have used for this. See the posts on TCPDump for more details on the TCPDump command settings. In the image below, I have used the 'date' command before and after the capture so that you can see this has run for a number of minutes.
Note that we are not translating the IP address and protocols. This means that we have to learn what the protocols usually are and also we are less likely to make assumptions (for instance, TCP 80 is not always web traffic).

Step 2
Now that we have our PCAP capture, we can start an examination. It is always a good idea (in incident response and forensic situations) to make a hash and a copy of the file.
You can see how we have done this and validated the copy as well in the image above.

Step 3
We can see the contents of the entire capture with NGrep by looking at our capture file.
ngrep -xX -I working.pcap
The command above and in the image below as output displays the packet capture in Hex and ASCII. This is of course far too detailed and we need to filter the results to make any detail from this.
In this instance, the packet output displayed shows a HTTP GET request and associated data. The Proxy used is clear inside the packet as is the web site called (http://www.msnbc.com). From the GET request data we can see that the user is making a search request (the page is - /search?).

Step 4
Now, say we are looking for a specific incident that occured using HTTP on TCP 80 (the default) to the server http://www.msnbc.com, we could first restrict the output to HTTP on TCP port 80 only (port 80)
ngrep -W byline -I working.pcap port 80
The output for this command is displayed below:
Notice that the output contains the details of the HTML recieved from the web server. This is from any web server however and we wish to restrict this to a single host. We will also save this output to a file using the followin g command (where we have the server, www.montrealgazette.com):
ngrep -I working.pcap -O /tmp/traffic.pcap "GET" host www.montrealgazette.com and tcp port 80
The command and output is displayed below.
We have now saved the selected traffic we wish to investigate. We can verifiy that we can read this using another PCAP compatible program (in the instance in step 5 TCPDump).

Step 5
Here we are validating that we have saved the data capture we wanted using TCPDump to read the extract capture file (/tmp/traffic.pcap):
The output is displayed below showing us that the packet capture has worked.
Here you can see we have a limited set of traffic to a specific host with a specific request (HTTP GET). NGrep is a powerful tool and well worth taking the time to learn how to use. It can be far easier to have a small extract of a capture to work with or even to determine if a file contains information work investigating before you go into details.

NGrep can do this for us.

Next, we will look at some other tools (including Wireshark and DNSTop).


Shahbaz said...

I like what I see. Learning NGrep now. :D 5

Craig Wright said...

Shahbaz, grep and NGREP are some of the most fundamental of tools. The following videos will also help:



Learning RegEx with grep and NGrep will enable you to create many complex filters and do things you see only in the most expensive of forensic tools.