Tuesday, 7 July 2009


DnsTop is a libpcap application. Like NGrep and tcpdump (covered in previous posts), it is an application that captures and analyses network traffic. DnsTop can be used to display various tables of DNS traffic on your network. As the site states, dnstop displays tables of:

  • Source IP addresses
  • Destination IP addresses
  • Query types
  • Response codes
  • Opcodes
  • Top level domains
  • Second level domains
  • Third level domains
  • etc...
Compiling and installing varies (with some RPMs available). Generally this is a standard "./configure && make && make install" process. This is of course dependant on your system's installed libraries.

Running dnstop is simple. the '-4' option sets IPv4, the '-l 9' option sets the application with up to 9 tables. Remember, the more tables, the more process memory and CPU used. Usually this is not an issue. Finally, the interface or file is set. In this case eth0. A PCAP format file can also be used as the input.
dnstop -4 -l 9 eth0
This means you can capture a PCAP file using TCPDump, Snort or any other sniffer of your choice and feed the data to dnstop for analysis. For instance, we can review the file, "save.pcap" and display the dns query results from this.
dnstop -4 -l 9 save.pcap
If you start with level 9 (-l 9) as an option, you have a far greater range of options.

The program is interactive. Select the level and it will display the various output styles. For instance, in the saved data we have the output for level 1 (TLD):
The domains at level 2:
Servers at lower levels and we get to a level with source, hosts and counts displayed:
Using the type option (enter t) you can drill down into a summary of the DNS query types recieved:
or the summary of the opcodes (enter o in the interactive screen).

Next, BPF (Berkley Packet Filters).

A dry run on NGrep

In this post I will go over a dry run on using NGrep. We will start with capturing a file using TCPDump into a PCAP format and then reading this into NGrep to extract the information we seek.

Step 1
The first stage is to capture all of the data we need into a PCAP file. For this we will use TCPDump. To do this, we are going to capture all traffic (with the complete payload, '-s 1500') to a file called capture.pcap.

tcpdump -nn -i eth0 -s 1500 -w capture.pcap
The command above and in the image below is what we have used for this. See the posts on TCPDump for more details on the TCPDump command settings. In the image below, I have used the 'date' command before and after the capture so that you can see this has run for a number of minutes.
Note that we are not translating the IP address and protocols. This means that we have to learn what the protocols usually are and also we are less likely to make assumptions (for instance, TCP 80 is not always web traffic).

Step 2
Now that we have our PCAP capture, we can start an examination. It is always a good idea (in incident response and forensic situations) to make a hash and a copy of the file.
You can see how we have done this and validated the copy as well in the image above.

Step 3
We can see the contents of the entire capture with NGrep by looking at our capture file.
ngrep -xX -I working.pcap
The command above and in the image below as output displays the packet capture in Hex and ASCII. This is of course far too detailed and we need to filter the results to make any detail from this.
In this instance, the packet output displayed shows a HTTP GET request and associated data. The Proxy used is clear inside the packet as is the web site called (http://www.msnbc.com). From the GET request data we can see that the user is making a search request (the page is - /search?).

Step 4
Now, say we are looking for a specific incident that occured using HTTP on TCP 80 (the default) to the server http://www.msnbc.com, we could first restrict the output to HTTP on TCP port 80 only (port 80)
ngrep -W byline -I working.pcap port 80
The output for this command is displayed below:
Notice that the output contains the details of the HTML recieved from the web server. This is from any web server however and we wish to restrict this to a single host. We will also save this output to a file using the followin g command (where we have the server, www.montrealgazette.com):
ngrep -I working.pcap -O /tmp/traffic.pcap "GET" host www.montrealgazette.com and tcp port 80
The command and output is displayed below.
We have now saved the selected traffic we wish to investigate. We can verifiy that we can read this using another PCAP compatible program (in the instance in step 5 TCPDump).

Step 5
Here we are validating that we have saved the data capture we wanted using TCPDump to read the extract capture file (/tmp/traffic.pcap):
The output is displayed below showing us that the packet capture has worked.
Here you can see we have a limited set of traffic to a specific host with a specific request (HTTP GET). NGrep is a powerful tool and well worth taking the time to learn how to use. It can be far easier to have a small extract of a capture to work with or even to determine if a file contains information work investigating before you go into details.

NGrep can do this for us.

Next, we will look at some other tools (including Wireshark and DNSTop).

Monday, 6 July 2009

Using NGrep

You can monitor and search for about any packet combination you can think of with NGrep. For instance, you can set it up to watch TCP port 25 (SMTP) and monitor and record email traffic.
Using RegEx filter expressions, you could even filter this for selected source or destination hosts, email addresses or content. An example of this is the use of such a filter to protect intellectual property. A small string could be embedded into documents (e.g. Word files in the templates) and this could then be used as a search term.
As from the image above (FTP) and that below (Telnet - in this case for Cisco and similar devices) we could look for insecure authentication. This example looks for username and password combinations that are sent over clear-text pathways.
In addition, the "-W byline" option will display data is a manner that is better presented for us humans. The example below shows us collecting HTTP (TCP 80) traffic. This example allows us to see the actual HTTP calls to the server and the ASCII data contained within this process.
We could even restrict this to selected web pages that we want to monitor.

NGrep supports IP (TCP & UDP) and ICMP. It also understands Ethernet, PPP, SLIP, FDDI, Token Ring and null interfaces.

If you want to learn more, download Ngrep and have a read of the usage page at sourceforge.

Sunday, 5 July 2009

More NGrep

NGrep will either treat the search input as case sensitive (the default) or with the use of the '-i' option will treat it as being case insensitive. This is, you can search with or without capitalisation in a standard string. Of course, this does not matter if you are using a well formed RegEx.

TCPDump is simpler and as a result makes a better capture engine then NGrep (not that you can not do this with it). As such, it is better to use TCPDump to capture to a PCAP format file, and then to analyse the saved data with NGREP. The '-I' flag is used to select the PCAP format file that you want to read.

By default, NGrep will display the ASCII of a file and insert a '.' in place of unprintable characters. The '-x' parameter is used to print both Hex and ASCII in the output (recommended).

More on NGrep again tomorrow. Tomorrow we will go through a dry run of using NGrep.