Understanding TCP Flags: Analyzing Network Issues with Packet Capture Techniques

I. Introduction to TCP Flags

TCP, short for Transmission Control Protocol, is a transport layer protocol in the OSI model that provides reliable, connection-oriented transmission services using TCP flags to manage and control communication.

In the TCP header, there is a TCP flags field, which indicates the purpose of the current packet. The TCP connection flags field is 6 bits long, comprising six different flags, any of which may be used in a TCP connection. These six flags are:

1. Urgent (URG): Notifies the receiving host that the TCP packet contains urgent data;

2. Acknowledgement (ACK): Used to acknowledge receipt of a TCP packet from the other host;

3. Push (PSH): Instructs the receiving host to forward the packet to the application layer immediately;

4. Reset (RST): Indicates that the TCP connection has been reinitialized by the other host;

5. Synchronization (SYN): Used to establish a TCP connection with the other host;

6. Finish (FIN): Used to close a TCP connection.

TCP flags in different packets may be the same or different, and by decoding the packets, we can ascertain the current operation and its purpose. For example, in the three-way handshake of TCP, the first step sets the SYN flag; the second step sets both the ACK and SYN flags; the third step sets the ACK flag. By utilizing the characteristics of TCP flags, we can analyze common network application issues.

II. Analyzing Network Issues Using TCP Flags

        When encountering an inaccessible TCP service on a target host, we can capture packets during the access process to identify the cause of the inaccessibility. We’ll use Colasoft Network Analyzer 5.0, with Telnet as an example to demonstrate the analysis method.

        Figure 1 shows the scenario of using the Telnet command on a Windows client (Client hostname: wangym) to access another host. Based on the result from Figure 1, neither host’s Telnet service can be accessed normally, but it’s uncertain whether this is due to network issues or because the host does not provide Telnet service.

               

Note: 

1. The Telnet command used here assumes the target server uses the default port configuration, i.e., the Telnet server port is TCP 23;
2. Some users might consider using the Ping command to test network connectivity, but since the ICMP protocol carrying Ping commands can lead to malicious attacks, posing a security threat, some ISPs or network administrators have disabled ICMP forwarding on their layer-three devices. In such scenarios, the Ping command cannot accurately test host connectivity.

Figure 2 shows the packet information captured by Colasoft Network Analyzer 5.0 when using the Telnet command from the Windows client to access the host at 192.168.2.10.

                                 (Figure 2: Original Packet from Telnet to 192.168.2.10)

From Figure 2, it is evident that three packets were communicated between the client and the host at 192.168.2.10. Upon closely inspecting the packets and their decoding, all three packets originate from the client and are sent to the 192.168.2.10 host. The acknowledgment numbers in all three packets are 0, and the SYN flag in the TCP flags is set, indicating these packets represent the first step of the TCP three-way handshake: the TCP SYN packet. No packets from 192.168.2.10 are received by the client, suggesting the physical link between the client and 192.168.2.10 is interrupted, possibly due to the non-existence of the 192.168.2.10 machine or because the machine is powered off.

Figure 3 shows the packet information captured by Colasoft Network Analyzer 5.0 when using the Telnet command from the Windows client to access the host at 192.168.2.100.

                                 (Figure 3: Original Packet from Telnet to 192.168.2.100) 

        From Figure 3, when using the Telnet command to access the 192.168.2.100 host, six packets were exchanged between the two hosts. Upon analyzing the packets and decoding them, packets 1, 3, and 5 are sent from the client to the 192.168.2.100 host, with acknowledgment number 0 and SYN flag set, indicating these packets are TCP three-way handshake’s first step: the TCP SYN packets. Packets 2, 4, and 6 are sent from the 192.168.2.100 host to the client, with acknowledgment number 1589766797 and both ACK and RST flags set, indicating these packets are acknowledgment packets from 192.168.2.100 to the client, rejecting the client’s connection request and informing the client that the current host (192.168.2.100) has not opened the requested service and is terminating the connection.

Note: We notice in Figures 2 and 3, the client sends three identical TCP SYN requests to the server (192.168.2.10 and 192.168.2.100). Why does this happen? This is due to the TCP protocol specification. After the client initiates the first step of the three-way handshake with a TCP SYN, if no SYN/ACK response is received from the server, it tries again after waiting awhile. If the connection fails three times, it doesn’t repeat this process, which is why we see three identical TCP SYN packets in the figures.

Through packet capture analysis of the two scenarios above, we determine that the reason for inaccessibility to the 192.168.2.10 host is a broken physical link, possibly due to the absence of the 192.168.2.10 machine or its powered-off state. The inaccessibility to the 192.168.2.100 host is due to the 192.168.2.100 machine not providing the requested Telnet service, i.e., the TCP 23 port is not open.

This method applies to all TCP services. When a server (of various TCP applications) cannot be accessed, users can use this method to track and analyze packets to assist in troubleshooting the issue.