Friday, 3 July 2009


Ngrep is a network (pcap format) based search tool. Just like its file based compatriot, grep, it is used when searching for strings in the payload (or the header for that matter). Ngrep supports:

  • Basic ASCII strings (e.g. "ASCII"),
  • Regex (this is Regular Expressions such as GET.*php"), and
  • Hex strings (e.g. 0029A)
The optimum results can be achieved using RegEX. These are a little arcane, so I will recommend a few tools to help use and learn these (there are also many good books and web sites for this). Hex also proves good if you know exactly what you are looking for, this can include using sections for captured data.

Just like any good command line program, we have options. Some of these are listed below:
-i Ignore case for regular expression
-d Read data from a specific live interface
-I Read data from the specified pcap-file
-O Save matching packets to a pcap format file
-x Dump packet contents as hexadecimal and ASCII
-X Dump packet contents as hexadecimal and ASCII and then treat the matched expression as a hexadecimal string
As a result, we can use NGrep as a filter and poor-man's scanner to find traffic on a network.

The RexEx coach is a means of training yourself into using Regular Expressions. A screenshot is provided below:
RegBuddy is one of those tools that incorporate a number of predefined RegEx Searches.
NGrep will run on both Windows and Unix/Linux. I will post on the UNIX/LINUX versions.

Linux binary RPMs are available. So installation is simple. Optionally, you can just unpack the static NGrep binary and copy it to a location of your choosing. Adding to the path will also make little a little easier. This static binary includes all of the library routines that are needed.

To unpack the Linux binary, use the bzip2 command. This should be done with the following format:
bzip2 -dc ngrep-1.44-linux-elf-static.bz2 >/usr/local/bin/ngrep
Now that we have installed NGrep, we will move onto more on how to use NGrep tomorrow...

Identifying network based attacks using TCPDump

I will not address issues and differences between TCPdump or Windump in this post. I prefer UNIX, so this is where I will focus on. That stated, there are only minor differences between TCPdump and Windump and most of what you can do in one is the same on the other (some flags do vary).

TCPdump uses the libpcap library. This can capture traffic from a file or an interface. This means that you can save a capture and analyse it later. This is a great aid in incident response and network forensics.

With a file, capture.pcap, we can read and display the data using the "-r" flag. For instance:

tcpdump -r capture.pcap
Will replay the data saved in the file, capture.pcap. By default, this will display the output to the screen. In reality, the data is sent to SDTOut (Standard Out), but for most purposes the console and SDTOut are one and the same thing.

Using BPF (Berkley Packet Filters), which I shall cover in a subsequent post, you can also restrict the output - both collected and saved. In this way, you can collect all data to and from a host and then strip selected ports (or services) from this saved file.

Some of the options that apply to tcpdump include (quoted with alterations from the Redhat tcpdump MAN file):
-A Print each packet (minus its link level header) in ASCII.
-c Exit after receiving a set number of packets (defined after c).
-C Before writing a raw packet to a savefile, check whether the file is currently larger than a given file_size. Where this is the case, close the current savefile and open a new one.
-d Dump the compiled packet-matching code in a human readable form to standard output and stop.
-dd Dump packet-matching code as a C program fragment.
-ddd Dump packet-matching code as decimal numbers (preceded with a count).
-D Print the list of the network interfaces available on the system and on which tcpdump can capture packets.
This can be useful on systems that do not support the use of the "ifconfig -a" command.
-e Print the link-level header on each dump line.
-F Use file as input for the filter expression. An additional expression given on the command line is ignored.
-i Listen on interface. This specifies the system interface to listen for traffic on.
-L List the known data link types for the interface and exit.
-n Don't convert host addresses to names. This can be used to avoid DNS lookups.
-nn Don't convert protocol and port numbers etc. to names either.
-N Don't print domain name qualification of host names.
-p Don't put the interface into promiscuous mode.
-q Quick (quiet?) output. Print less protocol information so output lines are shorter.
-r Read packets from file that has been created using the "-w" option.
-S Print absolute, rather than relative, TCP sequence numbers.
-s Snarf snaplen bytes of data from each packet rather than the default of 68 bytes.
-T Force packets selected by "expression" to be interpreted the specified type.
-t Don't print a timestamp on each dump line.
-tt Print an unformatted timestamp on each dump line.
-ttt Print a delta (in micro-seconds) between current and previous line on each dump line.
-tttt Print a timestamp in default format proceeded by date on each dump line.
-v When parsing and printing, produce (slightly more) verbose output.
-vv Even more verbose output.
-vvv Even more verbose output.
-w Write the raw packets to file rather than parsing and printing them out.
-x Print each packet (minus its link level header) in hex.
-xx Print each packet, including its link level header, in hex.
-X Print each packet (minus its link level header) in hex and ASCII.
-XX Print each packet, including its link level header, in hex and ASCII.
-y Set the data link type to use while capturing packets to datalinktype.
-Z Drops privileges (if root) and changes user ID to user and the group ID to the primary group of user.
This is by no means the complete list of options for tcpdump and I recommend that the reader looks over the man page to learn more.

When we are capturing data for incident handling and forensic purposes, remember NOT to notify the source hosts of your captures. Use the (-n) flags where possible to ensure that you are not looking up IP addresses from a remote DNS Server. This can flag that you are investigating.

In nearly all cases packets will flow over the screen far quicker than you can hope to read them. This is why we should always try top capture data to file even if we read it immediately (as noted using the '-r' option from above).

To capture data to a file, we should always specify the interface that we intend to capture on (this is using the -i flag). This makes our life a little easier by ensuring that tcpdump does not inadvertently start on an interface other than the one we want it to start on.

In *NIX (Unix/Linux), interface names will take the form of one of the following:
  • eth0, eth1, ... Ethernet
  • ppp4, PPP
  • le0. BSD Ethernet interface
  • l0. The loopback interface.
Loopback is a special "virtualised" interface for the host to send packets to itself. In *NIX, you can listen on the loopback. By issuing the following command, we can obtain a list of the interfaces that tcpdump may use:
tcpdump -D
We also see this in the image embedded below:
In this case, we have the options to use, Ethernet (eth0), Loopback(lo) or to listen on all configured interfaces (any). The following command is a fairly standard way to initiate tcpdump:
tcpdump -nqtp
As noted, the '-n' option is essential in incident response work. Using the '-p' option stops the host entering promiscuous mode. This does not make a great deal of difference on modern switched (other than span ports) networks as the traffic is isolated with each host becoming its own collision domain. It does stop rogue broadcast traffic to some extent from being logged and unless your desire is actually to capture all traffic (such as for an IDS), the '-p' option can be advantageous.

The '-q' and '-t' options are used to limit the volume of information returned (see above for details). Other options include using '-v' to '-vvv' to make a more verbose output. This will depend on your desired result (and disk space).

An interface in promiscuous mode will accept and store/display all packets that are received by the host. This is whether or not they were intended for this current host. That is, packets for any destination, not just the host you are on. When listening or "sniffing" whilst connected to either a hub or a switch SPAN port, you will be able to listen for packets for any machine on a particular Ethernet segment when in promiscuous mode.

To listen for for a selected port or protocol we can simply note what we are filtering for. As an example, if we want to watch TCP 80 (usually HTTP or web traffic - although not always) we can specify this using the terminology, "tcp port 80". We can see this in the image below where the '-X' option has been used to capture the payload as well:
If we now increase the SNAPLEN value (this is the maximum size of the packets we capture using the '-s' option, we can see the full payload. This is detailed in the image below:
Ideally, we could capture all of the traffic to a file such as will result from the following command:
tcpdump -npXA "tcp port 80" -s 1500 -w www.pcap
In this instance, we are saving the complete web traffic for the host to a file called "www.pcap". We can then analyse this file using tcpdump, or we can access it with any other program that supports the "pcap" format (such as Wireshark" or "ngrep").

In my next post I will detail how ngrep is used and offer a few examples on how you can use this simple program to search through pcap capture files for data (including signs of a breach or other useful data).
  • Remember...
It is possible to capture and save all packets and traffic to and from a host to a 'pcap' format file and then to extract selected information at a later time.