network

Network tools

Tcpdump

Tcpdump, as said on its front page, has two products:

• tcpdump: command line tool
• libpcap: C library for network traffic capture.

libpcap

The document is very short. I spend 10 min going through it and can understand how to use it.

Gopacket is golang wrapper on top of it.

tcpdump tricks

There are many articles online explain the meaning of each field in the output of tcpdump. My favorite resource is TCP Wikipedia.

Popular options:

• -X: print out both ASCII and hex text.
• -i: specify the interface
• -e: print out MAC address.

Note, tcpdump 4.99 also prints out interface name. See post.

Examples:

# capture TCP traffic on port 8000 and print out the content in ASCII
tcpdump -i any tcp and port 8000 -A

# Filter by the last four bytes of tcp packaet to be `}}\r\n`
# ip[2:2] gets the length of the ip packet. Here we assume it is ipv4.
# In ipv4's header, the 3rd and 4th bytes denotes the total length of the
# ip packet. TCP body is the last part of an IP packet. So in this way, we
# can filter the last bytes of a TCP packet.
tcpdump -i eth0 'tcp and ip[ip[2:2]-4:4] = 0x7d7d0d0a)'


# Use complicated logical operation.
# Note: in order to not get splitted by shell, the long filter is put inside
# a single quote.
tcpdump -i eth0 'tcp and (src host redis-master-0.redis-headless.production.svc.cluster.local and port 6379 and tcp[32:2] = 0x2a32 and ip[ip[2:2]-4:2] = 0x7d7d) or (dst port 6379 and (tcp[55:4] = 0x5a414444 or tcp[55:4] = 0x5a52454d)) -X'

For TCP, libpcap captures each individual tcp segment. So the question is how we can reassemble relevant segments into a whole request. There are many implementations online. Such as goreplay, Akita Cli.

Redis support

I am surprised to see that tcpdump can understand Redis protocol. Below is a sample messages captured for a redis service

15:54:13.588890 IP ip-172-31-54-65.us-east-2.compute.internal.47494 > 172-31-93-210.redis-master.production.svc.cluster.local.6379: Flags [P.], seq 6889:7083, ack 71, win 1002, options [nop,nop,TS val 4244669374 ecr 1964526694], length 194: RESP "MULTI" "ZADD" "unacked_index" "1707321253.5875661" "74b3b7a6-e63a-4c3a-b2fb-865fb901b25a" "HSET" "unacked" "74b3b7a6-e63a-4c3a-b2fb-865fb901b25a" [|resp]
        0x0000:  4500 00f6 6df1 4000 fe06 21be ac1f 3641  E...m.@...!...6A
        0x0010:  ac1f 5dd2 b986 18eb 1c9c b985 64d0 7f92  ..].........d...
        0x0020:  8018 03ea 1f3e 0000 0101 080a fd00 83be  .....>..........
        0x0030:  7518 4c66 2a31 0d0a 2435 0d0a 4d55 4c54  u.Lf*1..$5..MULT
        0x0040:  490d 0a2a 340d 0a24 340d 0a5a 4144 440d  I..*4..$4..ZADD.
        0x0050:  0a24 3133 0d0a 756e 6163 6b65 645f 696e  .$13..unacked_in
        0x0060:  6465 780d 0a24 3138 0d0a 3137 3037 3332  dex..$18..170732
        0x0070:  3132 3533 2e35 3837 3536 3631 0d0a 2433  1253.5875661..$3
        0x0080:  360d 0a37 3462 3362 3761 362d 6536 3361  6..74b3b7a6-e63a
        0x0090:  2d34 6333 612d 6232 6662 2d38 3635 6662  -4c3a-b2fb-865fb
        0x00a0:  3930 3162 3235 610d 0a2a 340d 0a24 340d  901b25a..*4..$4.
        0x00b0:  0a48 5345 540d 0a24 370d 0a75 6e61 636b  .HSET..$7..unack
        0x00c0:  6564 0d0a 2433 360d 0a37 3462 3362 3761  ed..$36..74b3b7a
        0x00d0:  362d 6536 3361 2d34 6333 612d 6232 6662  6-e63a-4c3a-b2fb
        0x00e0:  2d38 3635 6662 3930 3162 3235 610d 0a24  -865fb901b25a..$
        0x00f0:  3637 3531 0d0a                           6751..

You see that all components in the ASCII output are correct deserialized. Also, notice the word RESP. Initially, I thought it stands for response. But I couldn’t recall any part in tcpdump man page that mentions this prefix. After a while, I realized that it is Redis serialization protocol (RESP)! This is the code where this prefix is added to the output.

Not only Redis, other binary protocol are also supported as well. See code. It seems the protocol is auto detected by port number. If the service uses a different port, then we can use argument -T <type> to force packets to be interpreted by a specific protocol.

TODO: I do not see Kafka protocol in the list. Maybe make a contribution to it?

A small note about RESP: it uses \r\n for separator. So the sample response above has a lot of 0d0a. I may need to remember this code as it is quite common. :)