Analyzing TCP Dumps

Analyzing TCP dumps is very common use case when it comes to production support, since it is very useful and handy way to figure out network related issues. In this article I would like to walk you through that. Before start following this, you need to have thorough understanding of TCP/IP protocol suite[1].  To quickly recap your TCP/IP knowledge you may refer to this[2]. Also you need to have Wireshark packet sniffer tool installed in your system.
Scenario: Let’s invoke a simple currency converter web service with a sample SOAP request. Then we’ll capture the TCP dump and analyse it.
Before we start let me give you a brief introduction about TCP/IP suite. At the network layer, IP provides functions such as addressing, delivery, and datagram packaging, fragmentation and reassembly. At the transport layer, TCP and UDP are concerned with encapsulating user data and managing connections between devices. Other protocols provide routing and management functionality. Higher-layer protocols use these services, allowing them to concentrate on what they are intended to accomplish. Communication using TCP is said to be reliable because TCP keeps track of data that has been sent and received to ensure it all gets to its destination. UDP is an unreliable protocol. Also TCP connection is bidirectional.
Ok, here we go. I am going to use a reverse proxy written using HTTPCoreNIO[3] to send a SOAP request to the backend currency converter service. Clone the project[3] into your local file system and open it using whatever the IDE you are familiar with. Find the config.properties file resides under the src/main/resources directory and change the following properties as given.

Note: Leave keystore, keystorepassword, truststore and truststorepassword entries empty since we are not invoking any HTTPS/secure endpoint for the moment.
Then run the NHttpReverseProxy class using the IDE. The reverse proxy will listen for requests on port 8585. The reverse proxy is something similar to the ESB proxy except the fact that it does not conduct any mediation. It merely forwards the messages it gets to the backend service and sends the response back to the client.
We are going to send the following request to the currency converter web service[4]. The WSDL definition for the service can be found here[5].






Copy the above request to a file and save it in your file system. Start the wireshark packet sniffer tool and give the following filter condition.
ip.addr==173.201.44.188
This is the IP address of the backend currency converter service which we are going to call.


Move into the directory where you have saved the sample SOAP request given above and use the following curl request to send the SOAP message to the reverse proxy.
curl -v -d @currencyReq.xml -H "Content-Type: text/xml; charset=utf-8" -H 'Host: www.webservicex.net'  "SOAPAction:urn:ConversionRate"  http://localhost:8585/currencyconvertor.asmx


Send around four such curl requests consecutively and a sample response is given below.

Then forcefully terminate the application using your IDE and save the TCP dump. A sample TCP dump which I have obtained can be found here[6].
Now it’s time for us to analyse this TCPdump.


Reverse Proxy resides on the machine with the IP address 192.168.1.100 where as the backend service resides on the machine with the IP address 173.201.44.188. I have extracted important parts from the TCPdump and shown below.


Connection Establishment using three-way handshake
SYN-ACK






tcp_connection_establishment.jpg
Initially Reverse Proxy sends a [SYN] message to the backend service. Then the backend service replies with [SYN/ACK] and reverse proxy again [ACK] that. This is called as 3 way handshake TCP connection establishment process. Now a TCP connection is established between the two ends. The next step is data transferring phase.
Data Transfer
Then probe down the TCPdump and identify the data transfer between the peers. Click on that particular row  with HTTP/XML in the protocol column as shown in the image below and analyse it. You may identify the SOAP request sent above. The Reverse Proxy sends the SOAP request to the backend and the backend Acknowledges [ACK] it. Then you may see that the backend sends the response(200 OK) over HTTP/XML protocol and the Reverse proxy Acknowledges [ACK] it. Click on that response row and analyse the response which is similar to the one I have given above. Also note that HTTP/XML implies SOAP protocol. What we are doing here is sending XML data over HTTP application layer protocol.
DataTransfer
tcp_data_transfer.jpg



The value in the sequence number field of a segment defines the number assigned to the first data byte contained in that segment[2].
The value of the acknowledgment field in a segment defines the number of the next byte a party expects to receive.  The acknowledgment number is cumulative[2].
For an example, in my case I am taking first POST request which starts at sequence number 200 and TCP segment length is 311. You can get these information by merely analyzing the Transmission Control Protocol section under the selected row. Now the backend service has received up to 511 bytes and the next expected sequence number is 511, which is sent with the [ACK] segment. If the ACK is not received for this segment till the Retransmission timer expires, TCP merely retransmit the segment again. With TCP you may never loose your data.
Connection Termination using three-way handshake
All the way down you may see the TCP connection termination process. Since we terminated the Reverse proxy application forcefully after some time, the Reverse proxy starts an Active Close event and sends a [FIN] segment to the backend service. Then the backend service responds with [FIN/ACK] segment to the Reverse proxy which is a Passive Close. This [ACK] is for the previous [FIN] segment sent from the Reverse proxy, and a new [FIN] segment is sent out from the backend service to terminate the TCP connection from it’s end. Finally the reverse proxy sends an [ACK] segment and TCP connection is terminated/teared off. This is also a three way handshake connection termination process. Each TCP connection has two directions and each end of the connection is terminated separately.
FIN-ACK
tcp_connection_termination.jpg
TCP Keep Alive: Usually we are not terminating a TCP connection after each request/response cycle is over due to performance reasons. Creating a new TCP connection is costly especially when it comes to secure/HTTPS connections. So what we do is, reuse the same TCP connection, over multiple request/response cycles. This will improve the performance.
Piggybacking: If ACK, SYN, FIN is sent together with data, we call it as piggybacking. The term comes from tieing something on the back of a Pig and dispatching along with the beast. Nice terminology which is interesting after a weird session.
Default TCP Ports: Default TCP port for HTTP is 80, whereas default TCP port for HTTPS is 443.
There are lot of TCP/IP related concepts such as half-closed state, Reliability, Flow control, Congestion control  etc. which I could not cover in this article. I will thoroughly recommend you to refer [1][2] in order to gain more knowledge about TCP/IP protocol suite.




References


Comments

Post a Comment

Popular posts from this blog

Introducing Java Reactive Extentions in to a SpringBoot Micro Service

Image
Introduction Today we are going to take a look at Reactive Programming using Java8 and RxJava, which is a library for composing asynchronous and event-based programs by using observable sequences. RxJava stands for Reactive Extensions for the JVM. It extends the observer pattern [Gamma95] to support sequences of data/events and adds operators that allow you to compose sequences together declaratively while abstracting away concerns about things like low-level threading, synchronization, thread-safety and concurrent data structures [1]. Pre-Requisites : Having a good knowledge about Java8, Functional Programming and spring boot microservices. What is Reactive programming? In essence, Reactive programming is an asynchronous programming paradigm concerned with data streams. The system time, a list of entries from the DB/API, user clicks, everything can be a stream of data. Data from different streams can easily be combined and transformed, and in the end processed/observed by the
Read more

Optimal binary search trees

Image
Suppose that we are designing a program to translate text from English to French. For each occurrence of each English word in the text, we need to look up its French equivalent. We could perform these lookup operations by building a binary search tree with $n$ English words as keys and their French equivalents as satellite data. Because we will search the tree for each individual word in the text, we want the total time spent searching to be as low as possible. We could ensure an $Θ(log_2 \;n)$ search time per occurrence by using a red-black tree or any other balanced binary search tree. Words appear with different frequencies, however, and a frequently used word such as the may appear far from the root while a rarely used word appears near the root. Such an organization would slow down the translation, since the number of nodes visited when searching for a key in a binary search tree equals one plus the depth of the node containing the key. We want words that occur frequently in the t
Read more

Edit distance

Image
Biological applications often need to compare the DNA of two (or more) different organisms. A strand of DNA consists of a string of molecules called bases, where the possible bases are adenine, guanine, cytosine, and thymine. Representing each of these bases by its initial letter, we can express a strand of DNA as a string over the finite set $\{A, C, G, T\}$. For example, the DNA of one organism may be $S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA$, and the DNA of another organism may be $S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA$. One reason to compare two strands of DNA is to determine how “similar” the two strands are, as some measure of how closely related the two organisms are. We can, and do, define similarity in many different ways. For example, we can say that two strands are similar if the number of changes needed to turn one into the other is small. [1] We formalize this notion of similarity in our Edit distance problem. 15-5 Edit distance - Problem Statement In order to transform one sour
Read more