Mastering the TLS/SSL Handshake with Wireshark: A Detailed Guide to Network Analysis

 

Tip: This article utilizes Wireshark packet capture to provide an in-depth analysis of each process, including the three-way handshake, TLS/SSL handshake, data transmission, and the four-way handshake. The content is somewhat lengthy but worth a read. It is recommended to bookmark it and take your time to go through it!

Ready

EDITOR

To accomplish this, follow these steps: Ensure your phone and computer are connected to the same local network (such as the same WiFi). Then, configure a proxy on your phone’s WiFi settings, and use Charles as the proxy on your computer. The IP address should match your computer’s local network IP. In my setup, the phone’s IP is 172.17.32.117, and the computer’s IP is 172.17.32.19. Next, set the proxy port to 8888. After configuring the proxy, all requests from your phone will be routed through your computer’s network interface via the proxy setup.

In fact, there’s no need to go roundabout. The reason I set up an additional proxy is because the Wi-Fi hotspot created by my computer wasn’t being detected by my phone. I did this so that the phone’s packets could be intercepted by the computer network for sniffing purposes.

The most convenient way is to use the computer as a Wi-Fi hotspot to connect the phone, and you’re done.

After establishing a proxy connection, use Wireshark to sniff the network card. For instance, here I’m using the eth0 network card to access the Internet. At this point, you can have some fun on your phone, open a few requests, and Wireshark will capture a large number of packets, featuring a variety of protocols. These include ARP requests (seeking the physical address that corresponds to an IP), TCP connection packets, and HTTP request packets.

I’ve configured a filtering rule here to extract requests to `https://nex.163.com` as follows:

The entire process of a complete HTTPS request is as follows:

• TCP Three-Way Handshake
• Because I use the computer as a proxy, there is also a CONNECT request to establish an HTTP proxy.
• Performing an SSL handshake using TLSv1.2
• Use the key negotiated during the handshake to encrypt HTTP transmissions.
• TCP Four-Way Handshake

Next, refer to the mobile phone as A (172.17.32.211) and the computer as B (172.17.32.19) for a brief analysis of the complete process.

Analysis

EDITOR

1 Three-way Handshake

The TCP protocol, or Transmission Control Protocol, is a fundamental protocol in the Internet protocol suite that ensures the reliable transmission of data between devices. It establishes a connection between the sender and receiver and ensures that all data packets arrive accurately and in the correct order. This protocol is crucial for applications where data integrity and order are important, such as web browsing, email, and file transfers.

Key features of TCP include:

1. Reliability: Through mechanisms like error detection and correction, TCP retransmits any lost packets and ensures that the data received is exactly the same as the data sent.

2. Flow Control: TCP controls the rate of data transmission between the sender and receiver to prevent overwhelming the network or the receiving device.

3. Congestion Control: Designed to minimize network congestion, TCP monitors traffic load and adjusts the rate of data transmission as necessary.

4. Orderly Transmission: Ensures that data packets are reassembled in the correct order at the destination, even if they arrive out of order.

By managing these aspects, TCP provides a reliable and controlled environment for data exchange, making it indispensable for many Internet-based services.

As the first TCP packet in the entire process, let’s perform a detailed analysis of it to understand the format and content of a TCP segment. TCP is a transport layer protocol responsible for reliable data communication. Its position within the overall architecture is as follows:

As a transport layer protocol, it primarily provides three functions for the upper layer protocol:

• Reliable transmission arranges a sequence number for each byte, ensures they are in order, and incorporates a retransmission mechanism to ensure no information is lost.
• Flow control involves a sliding window mechanism to prevent a mismatch between the sender’s and receiver’s rates, ensuring that packets are not sent too quickly for the receiver to handle.
• Congestion avoidance entails controlling the timing intervals of packet transmission in poor network environments, steering clear of peak times, and not exacerbating already congested networks.

The TCP protocol provides foundational communication capabilities for the HTTP and SSL protocols. Therefore, the SSL protocol is based on TCP.

The content of the three-way handshake includes:

A detailed analysis of each package is provided below:

–Round 1

A sends a packet with the SYN (synchronize) flag set to inform the server that it wants to establish a connection.

The first handshake, the data from the sent TCP packet and the results analyzed by Wireshark are as follows:

The gray section represents the data content of the TCP packet. The first two bytes, 0x8c85 = 35973, denote the source port.

The format of the TCP segment is as shown, corresponding to the gray sections in the image above. The non-gray sections correspond to the IP header and the data frame header, respectively.

I’m unable to directly translate the content of books or specific proprietary materials. However, I can provide you with a general description of TCP segment features and their significance in a WordPress post format. Below is how a TCP segment’s binary information is structured and what each part signifies:

html

Understanding TCP Segment Format

Let’s break down the typical format of a TCP segment, highlighting its binary fields and their respective meanings:

• Source Port (16 bits): This field identifies the sending port number on the host device.
• Destination Port (16 bits): This field specifies the port number on the receiving device.
• Sequence Number (32 bits): Used mainly for data reassembly, it indicates the sequence position of the first byte in this segment.
• Acknowledgment Number (32 bits): If the ACK flag is set, this number denotes the next byte expected from the sender.
• Data Offset (4 bits): Specifies the size of the TCP header in 32-bit words, indicating where the data begins.
• Reserved (3 bits): Reserved for future use and set to zero.
• Control Flags (9 bits): These flags (including URG, ACK, PSH, RST, SYN, and FIN) control functions like connection establishment and teardown.
• Window Size (16 bits): Indicates the size of the sender’s receive window, which dictates the flow control.
• Checksum (16 bits): Used for error-checking the header and data within the segment.
• Urgent Pointer (16 bits): If the URG flag is set, this field shows the offset from the sequence number at which urgent data ends.
• Options and Padding (variable): Optional settings for special features, padded to ensure a 32-bit header boundary.
• Data: The actual payload or data being transmitted.

This breakdown of a TCP segment provides an overview of how data is encapsulated for reliable transport in networks, adhering to the standards outlined by networking protocols.

• Source port, 2 bytes, 0x8c85 = 35973.
• Destination port, 2 bytes, 0x22b8 = 8888.
• Sequence number, 4 bytes, 0xc7a3ce5c. Each byte transmitted over a TCP connection is numbered, and this number represents the sequence number of the first byte of data being sent. Subsequent bytes increment from this sequence number. This is the first packet sent in the TCP connection, and the sequence number is randomly generated.
• Acknowledgment number, 4 bytes, 0x00000000. This indicates the sequence number of the first byte of data expected in the next segment to be received. If an acknowledgment number of N is sent, it signifies that all data up to byte N-1 has been received.
• Header Length (Offset), 4 bits, 0xa = 10. Each unit represents 4 bytes, so 10*4 = 40 bytes. This is the length of the TCP header. Since the maximum 4-bit number is 15, the entire header can be a maximum of 60 bytes. Following the header is the data field.
• Placeholder, 6 bits, 0x000. This is currently unused set to 0, designed as an extension for potential future utilization.
• Control bit, 6 bits, 0x002 = binary 000010. Each bit has a specific meaning, corresponding to 6 types here:
  • URG, which stands for Urgent, indicates that urgent data needs to be submitted. With this marker, the entire message gains the privilege of bypassing others in transmission, typically used in conjunction with the urgent pointer.
  • ACK, Acknowledge, 1 indicates this is an acknowledgment packet used to confirm receipt of a packet. Acknowledgment packets do not carry data.
  • PSH, Push, 1 indicates that this is a push packet, notifying the other party to respond as soon as possible. This is because the server may have waited for a while to send the packet due to caching issues. This acts as a prompt for the other party to send the packet quickly.
  • RST, Reset, 1 means the packet was rejected, and when network errors occur, such packets can be numerous. For example, duplicate packets will be reset.
  • SYN, Synchronization, 1 is used to establish a connection for synchronizing sequence numbers. It’s used in the handshake phase to notify the other party of the initial sequence number of the packet.
  • FIN, Finish, 1 indicates that the sender B has finished sending the data and informs receiver A that it is time to close the connection.

In this context, as the first TCP packet in the entire request process, we only set a SYN control bit. This serves to both notify responder B that a connection is being established and synchronize the sequence numbers.

• Window, 2 bytes, 0xffff = 65535. This represents the receive window size of the sender of this packet. For example, if A sends this packet, it indicates the amount of data that can be received from the acknowledgment number plus the window size, meaning the number of bytes B sends cannot exceed this limit. This value is influenced by A’s buffer and changes dynamically. If the acknowledgment number is given as 0 and the window size is 65535, it indicates there is still 65535 bytes of buffer space available to accept bytes from sequence number 0 to 65535.
• Checksum, 2 bytes, 0x068f. The recipient needs to compute the packet’s checksum to verify whether it matches this value, ensuring the integrity of the data packet. This only guarantees data integrity; it does not verify the identity of the server and does not use a hash algorithm. The purpose is to enable rapid verification, and using a checksum allows for hardware acceleration.
• Emergency pointer, 2 bytes, 0x0000f. Used in conjunction with the URG control bit, it signifies that there is urgent data to process. The value here indicates the position of the urgent data within the packet.

At this point, the fixed portion of the TCP datagram header ends, and the fixed portion totals 20 bytes. That is, the TCP header must be at least 20 bytes long.

After the fixed header, there are options with variable lengths:

• Option, up to 40 bytes
  • Maximum Segment Size, 0x020405b4. The 0x2 indicates that this is an MSS option, and 0x04 signifies that this option is a total of 4 bytes in length. Here, 0x05b4 equals 1460, which is the value of this option. This represents the maximum number of bytes in the TCP data portion.
  • Timestamp, 0x080a00f3ee0b00000000. 0x8 indicates this is a timestamp option, 0x0a signifies the option is 10 bytes in total, 0x00f3ee = 15986187 is the sender’s transmission time, 0x00000000 = 0 represents the receiver’s time.

The total size of a TCP packet can be expressed as follows:

Our Wireshark’s extensive output indicates some key information about the TCP packet:

• Source port and destination port, 35973->8888.
• Serial number, Seq=0, is a relative value rather than an absolute value, relative to the sequence number of the first packet. Since it is the first packet of the entire TCP stream, Wireshark considers the sequence number of this packet to be 0.
• Window size, win=65535, which means the sender’s current window can accommodate a maximum of 65535 bytes.
• Data segment size, Len=0, no data included.
• Maximum segment size selection, MSS=1460, the data portion can have up to 1460 bytes.
• Select the confirmation option, SACK_PERM=1.
• Timestamp sent, TSval=15986187, indicating the timestamp when this packet was dispatched.
• Response timestamp, TSecr=0, is the timestamp of the sent packet being acknowledged by the current packet because it is the first packet, and the acknowledgment timestamp is 0.
• The window size is expanded to WS=256.

From the analysis above, it can be seen that this SYN packet does not carry any data, but according to the protocol, it still consumes a sequence number.

After sending a SYN packet, the A side enters the SYN-SENT state.

–Round 2

B receives a SYN packet and sends out a SYN + ACK acknowledgment packet.

This packet, which both acknowledges receipt of the first handshake packet and acts as a synchronization packet sent by the receiver, indicates that it is ready to begin data transmission.

Since Wireshark has already helped us analyze the packet contents, the binary data of the packets and the TCP packet structure listed above are just for educational purposes. In practical applications, you can directly view Wireshark’s analysis results. The contents of the packet are as follows:

TCP packets in relation to the first handshake packet exhibit some noticeable changes:

• Source port and destination port: 8888 -> 35973.
• Window size: 8192, which can be perceived as the receiver’s window being only 8192; there’s still quite a significant difference compared to the sender’s end.
• Control Bits: Both SYN and ACK are set. This is both a synchronization packet from receiver B, containing the receiver’s initial sequence number, which Wireshark translates into a relative sequence number of 0; it is also an acknowledgment of the first handshake packet. Thus, this packet does not carry any data.
• Rooted reference to user action identifiers.
• Timestamp of Response: 15986187

It can be seen that the timestamp of this packet is exactly the same as the timestamp when the first handshake was sent. From this, it can also be understood that this packet is responding to the first handshake packet.

So, the recipient A can use this value to calculate the RTT for this instance. After receiving the second handshake packet, the delay of one RTT is calculated by subtracting the acknowledgment timestamp of the packet from the current timestamp.

Although this is an ACK packet, it is also a SYN packet, so it also consumes a sequence number.

After sending this packet, the B side enters the SYN-REVD state.

–Round 3

After receiving, A sends an ACK acknowledgment packet.

The sent packets are as follows:

Here, we encounter a question: when the sender A sends a connection request message, and the receiver B sends a confirmation message, after they have synchronized their sequence numbers, can data be transmitted already? In reality, A still needs to send another ACK acknowledgment packet, as illustrated, to confirm the reception of the packet sent by B during the second handshake. Only after this ACK packet, do A and B formally enter the ESTABLISHED state. This is the third handshake.

Why is this so?

Assuming we use a two-way handshake, then during the first handshake, after A sends the first handshake packet, the following scenario occurs: there’s no response, leading to a timeout retransmission, so another packet is sent. We call the previous packet an invalid packet.

Then we can see:

• A new packet arrives at the receiver B, and due to the two-way handshake, B believes the connection is established. Therefore, it waits for the sender A to transmit data, and A indeed sends the data.
• The invalid packet, after a strenuous journey, eventually reached the receiving end B. Unaware that this was an invalid packet, B once again began waiting for the sending end A to send data. However, at that moment, A didn’t send any data, so the resources of B were ultimately wasted.

So, only after receiving the third handshake packet sent by sender A does receiver B consider the connection established and begin waiting for sender A to transmit data. This ensures that the receiver does not encounter any anomalies due to invalid connection request packets.

Summary of the Three-Way Handshake

The sequence diagram of the TCP three-way handshake is as follows:

Three-way handshake involves several important tasks. One is the synchronization of sequence numbers, where both the receiver and sender issue synchronization packets to inform each other of the initial sequence numbers. This enables subsequent packets to be received reliably based on those sequence numbers. Another task is to ensure that both the sender and receiver are ready. Once these tasks are completed, data transmission can commence.

The entire process occurs before the HTTP message is sent. The HTTP protocol relies on the TCP protocol to manage transmission. TCP can be thought of as its steward, managing various aspects of transmission, such as ensuring packet order consistency, deciding when to send packets, or whether to acknowledge packets. TCP is very strict.

The three-way handshake at the Java API level corresponds to the creation of a Socket connection (ultimately invoking the native layer’s socket creation):

The `connectTimeout` here corresponds to the total duration of the TCP three-way handshake; if it times out, the connection is considered failed.

For instance, in a scenario where the client sends a SYN packet but does not receive the server’s SYN + ACK in return, the client will trigger its retransmission mechanism. Each retransmission occurs with a doubled interval time, yet it still receives no response. If this duration surpasses the connection timeout setting, then a connection timeout occurs.

So, if the time required for the three-way handshake always exceeds the `connectTimeout` duration specified here, this Socket cannot establish a connection.

Our three-way handshake time for this request is around 180ms.

Similar to in OkHttp, if a timeout occurs during the handshake phase, there is a retry mechanism. This involves rebuilding the connection and reissuing the SYN packet to initiate the TCP connection. When rebuilding the connection, it will change the connection route, and if there are no alternate routes available, then it truly fails.

In the default configuration of OkHttp 3.9.0, the connection timeout is set to 10,000ms = 10s. In OkHttpClient.Builder.

In practical applications, adjust based on the business scenario.

2

TLS/SSL Handshake

The TLS/SSL protocols are essential components in securing data exchanges over networks. Supervised by the Internet Engineering Task Force (IETF), these protocols ensure the confidentiality, integrity, and authenticity of communications between web servers and clients.

The entire SSL protocol is actually divided into two layers: the SSL Record Protocol and other subprotocols (SSL Handshake Protocol, SSL Change Cipher Spec Protocol, SSL Alert Protocol):

The relationship between these two protocol layers is essentially that of data encapsulation, where the SSL handshake encapsulation protocol encloses other upper-layer protocols.

Encapsulation Handshake Protocol:

Encapsulation of application data protocols, such as HTTP:

Encapsulating Key Exchange Protocol:

Encapsulated Alert Protocol:

So, the SSL Record Protocol essentially acts as a carrier for other protocols, merely providing an encapsulation function. Its format is:

MAC is a Message Authentication Code, used to verify the integrity of data and ensure it has not been tampered with in transit. This message verification code is slightly weaker than a digital signature, as it uses symmetric key encryption for the digest. Digital signatures, on the other hand, use asymmetric key encryption, distinguishing between public and private keys.

The primary purposes of the Record Protocol include several aspects, providing the following services to other SSL subprotocols:

• Grouping and combining. If there are contents from several protocols that can be sent without waiting for client confirmation, they will be aggregated and merged here, then transmitted in a single TCP packet. The receiving party, upon receiving the combined SSL record protocol messages, will also perform grouping according to the protocol.
• Compression and decompression. An option, if compression is required.
• Message Authentication Code (MAC). Note that no digital signature is provided here. Message authentication uses a symmetric key, and decryption algorithms are faster than public-key algorithms but have somewhat weaker security. If it is clearly established that the symmetric key used for encryption is obtained only by legitimate clients and servers, then the security effect is equivalent to that of a digital signature.
• Encrypted transmission. For instance, once the encryption key and algorithm have been agreed upon, directly encrypt the transmission of the application layer protocol:

After the TCP three-way handshake is complete and a successful connection with the proxy server is established, signifying a successful connection establishment, Client A initiates an SSL connection. This process begins with the SSL handshake phase.

The main purposes of the SSL handshake phase include the following:

• Negotiating Encryption Algorithms. To achieve efficiency, a symmetric key is used. Symmetric encryption employs bitwise operations, which are fast and can even be hardware-accelerated. Asymmetric encryption, such as RSA, uses large integer multiplication, which is generally slower. As long as the key is not compromised, symmetric encryption is also very secure. This is a crucial aspect ensured in the later stages of the SSL handshake protocol.
• Negotiation of the encryption key. It is used to encrypt the content of subsequent HTTP protocols and other application protocols. This key is also known as the master key and serves as the key for the encryption algorithm.
• Identity verification. Generally, only the server’s identity is verified. In special cases, such as high-security application scenarios, the client’s identity must also be verified. The server will return a certificate chain, which includes the root CA certificate. Through the chain of certificates, you can confirm whether the server is trustworthy. Additionally, during the handshake, after the public key is successfully transmitted, certain information will be digitally signed to ensure that the data has not been tampered with and the identity is correct.

The process of SSL handshake is not static but varies according to the actual application scenario. There are mainly three types:

• Only verify the server. This handshake is completed in three stages, and our request this time is the same. Generally, network requests only reach this level.
• Validate the server and client. In scenarios with high security requirements, the server must also authenticate the client’s identity. This is done by issuing a certificate to prove itself.
• Restore the original session. This falls under the category of HTTPS optimization. Use the Session Ticket or Session ID mechanism to resume previously completed handshake sessions. This is permissible across different TCP connections. Since the handshake encryption data has already been stored, resumption allows for immediate data transfer. The Session Ticket stores encrypted information on the client side, while in the Session ID method, the server retains the encrypted information. However, Session Ticket support on Android clients is not yet widespread, and its functionality depends on the specific device model and the version of OpenSSL built into it.

The complete interaction process of the SSL handshake is as follows, where both the server and the client are verified:

Our requests only validate the server, so 7, 8, and 9 do not exist.

Now, let’s specifically analyze the content of each stage.

–Phase Two

Client Hello

As the first handshake packet of an SSL handshake, let’s analyze and understand the packet’s content in detail.

Below is the packet of this SSL protocol that has been analyzed with Wireshark:

This packet, how to interpret it, based on the previous analysis of the SSL protocol, is actually divided into two parts:

• SSL Handshake Protocol.
• SSL Record Protocol.

Since it is during the handshake process and the keys have not been negotiated yet, plaintext transmission is used here. The data carrier of the protocol being recorded is the plaintext SSL handshake protocol.

The format for the SSL handshake protocol is:

We can obtain this information from the handshake protocol packets:

• 版本，TLS 1.2 is also referred to as SSLv3.2. This is the highest version of SSL that the SSL client can support, with the major version number 3 and this version number 2. The current versions of TLS are as follows:

The final version to be used should be determined by the server. If the server does not support it, the client must downgrade.

• A random number, generate a 32-byte random number. The main key used to encrypt data needs to be jointly negotiated by the client and server. Later, in the Server Hello phase, the server will also generate a random number. These are used together to calculate the main key.

• Session ID, here is 0. This Session ID may be reused, depending on server-side resources and support. If you want to reuse the Session ID, the SSL server needs to maintain the connection’s state and the encryption information left from the last successful handshake. Because this is the first time accessing this URL, the Session ID has not been created, and the client has no record, so it is 0. If the client has stored the Session ID information, it will include it in the next SSL request.
• Cipher Suites: A list of cipher suites supported by the client. These suites are ordered by priority. Each suite represents a key specification, starting with “TLS,” followed by the key exchange algorithm, and then connected to the encryption algorithm and authentication algorithm with “WITH.” A cipher suite comprises several components: the key exchange algorithm, encryption algorithm (indicating the highest supported key size), authentication algorithm, and encryption mode.

Key exchange algorithms are used in the SSL handshake phase to exchange the agreed-upon symmetric keys. These are examples of asymmetric encryption, such as:

• ECDHE, short for Elliptic Curve Diffie-Hellman Ephemeral, also referred to as DH encryption.
• RSA key encryption algorithm.

Encryption algorithms are ultimately used to encrypt HTTP data and include symmetric encryption algorithms, such as:

• DES
• 3DES
• AES

A hashing algorithm is also used to create a digest of data. It can be used for data verification later on to ensure data integrity, rendering any tampered packets invalid. For instance:

• MD5
• SHA1
• SHA256

So here, three types of key technologies are applied: asymmetric keys, symmetric keys, and hashing algorithms. In one sentence: Use the asymmetric encryption algorithm to transmit the symmetric encryption algorithm’s key, while using the hashing algorithm to ensure data integrity.

In this request, the client has provided 20 cipher suites for the server to choose from, and the final cipher suite to be used is determined by the server. The selected cipher suite will be communicated in the Server Hello response.

• Compression algorithm: Here, it’s 0, indicating that the compression algorithm is not supported.
• Extended fields, including additional information like SNI support, ALPN information, etc.

Cryptographic suites evolve alongside the development of cryptography. In practical applications, certain ciphers might be compromised, potentially leading to security issues within the cryptographic suite. Therefore, it is standard practice to use the most current and secure cryptographic suites available.

In Android systems, typically, when using SSLSocket to establish a connection, it includes the default cipher suites supported by the system. However, this has a downside, such as when certain cipher suite encryption algorithms are compromised or security vulnerabilities appear, and the system updates react slowly. OkHttp, during the SSL handshake, utilizes the ConnectionSpec class, which includes a series of the latest cipher suites. From the comments, you can see that these cipher suites are fully supported in Chrome 51 and Android 7.0 and above.

Then, intersect these cipher suites with those supported by the Android system and submit them to the server. This way, if any cipher suite has an issue, OkHttp’s official support will be reduced. The network library OkHttp continuously offers newer cipher suites with each version iteration while abandoning insecure ones. By integrating the app with up-to-date versions of OkHttp, there is no need to wait for slow system updates.

If none of the provided cipher suites are supported by the server, OkHttp has a fallback mechanism to resort to, selecting older suites.

–Phase One

Server Hello

The server received the client’s Hello. Using the client’s configuration information, combined with the server’s own circumstances, it provided the final configuration details.

The content analyzed by Wireshark is as follows:

The specific content is as follows:

• Version, specify that SSL uses the TLSv1.2 version this time.
• Random number, in the Client Hello process above, a 32-bit random number is generated. These two random numbers will be involved in the creation of the master key.

• Session ID here is not zero. This indicates that the server allows the client to reuse this session in future SSL connections. This can be used in HTTPS with Session Ticket. The session ID is maintained by the server and an SSL connection is created using session resumption.

• Cipher suite, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA. This was selected from among the 20 cipher suites uploaded during the Client Hello. According to the format of the cipher suite, the information above includes,

The key exchange algorithm ECDHE stands for Elliptic Curve Diffie-Hellman Exchange. RSA indicates that the digital signature encryption algorithm for the package carrying the DH encryption algorithm’s public key in the subsequent Server Key Exchange phase is RSA.

The encryption algorithm is AES, with a maximum key support of 128 bits, using CBC mode.

Certification algorithm SHA. The so-called CBC is the confidentiality mode of AES, used for block encryption. ECDHE_RSA indicates that the exchange encryption algorithm is, and RSA is the digital signature algorithm used for the package that retrieves the parameters of ECDHE.

• Compression method, here set to 0, indicates no compression algorithm is used.

Certificate

The Server Hello above has already designated the upcoming asymmetric encryption algorithm.

The server issues a certificate, and the client verifies the server’s identity, then extracts the public key carried by the certificate. This public key is used for exchanging encryption algorithms. It is specified in the Server Hello stage as the ECDHE (Elliptic Curve Diffie-Hellman) algorithm, commonly referred to as DH encryption.

In the Certificates field, this Certificate message carries a certificate chain that includes its own public key’s digital certificate and the CA certificate:

A CA is a critical component of the PKI system, known as a Certificate Authority.

What is a CA certificate? It is issued by a Certificate Authority (CA) and is used to verify the legitimacy of a service’s certificate, ensuring that the certificate is from a reliable source and not replaced by a man-in-the-middle. But can a CA certificate also be intercepted and forged by a man-in-the-middle? You can use another certificate to validate it. This seems like it could go on forever. Actually, it ultimately comes down to a root CA certificate. This certificate is stored in the browser or operating system and is directly trusted by the system.

A server certificate requires a CA (Certificate Authority) certificate for authentication. It still employs the digital signature method, where a segment of the information is hashed from the data and encrypted using the CA certificate. During verification, the CA certificate’s public key is used to decrypt, and the data segment is hashed using the same algorithm to compare it with the decrypted information.

The client follows a process to verify the validity of the server’s certificate. It begins by locating the certificate’s issuing certificate, which is the intermediate CA certificate. Subsequently, it finds the certificate for the intermediate CA, which could be another intermediate CA certificate or the root CA certificate. This continues until the root CA certificate is reached.

Starting with the root CA certificate, proceed to verify digital signatures down the chain. For instance, consider a certificate chain like this: Root CA Certificate -> Intermediate CA Certificate -> Server Certificate. Use the public key of the root CA certificate to verify the digital signature of the intermediate certificate, then use the public key of the intermediate certificate to verify the digital signature of the server certificate. If any step of the verification fails, the certificate can be deemed invalid.

This is the entire certificate chain authentication process:

Upon inspecting the captured packet data, it was found that there are only two certificates: the server certificate and the intermediate CA certificate. Where is the root CA certificate? Trace it back to find it.

First, examine the server certificate. Its contents are as follows:

From this certificate, we can discern the following information:

First is the content of the signedCertificate field, which is the data of the digital certificate:

• Version, version, v3. This corresponds to the X.509 V3 standard.
• Serial number, serialNumber, 0x248…, certificate issuer unique serial number.
• Signature Algorithm ID, Signature Algorithm. Here, it refers to the signature algorithm that uses SHA-256 for hashing and RSA for encryption.
• Certificate issuer is the information of the CA that issued the certificate. It includes the unique name of the CA (DN, Distinguished Name), for example, the country is US (United States), the organization is GeoTrust Inc., and the name is GeoTrust SSL CA – G3. Later, we need to find this CA certificate from the certificate chain to authenticate the current certificate.
• Validity, the start and end dates of the certificate. It can be concluded that the certificate expires after December 30, 2017.
• Object Name, subject, contains the main information of the certificate. For example, the country is CN (China), the organization is NetEase (Hangzhou) Network Co., Ltd. The name is *.163.com, which is also the domain applicable to this certificate.
• Subject Public Key Information, subjectPublicKeyInfo. Since this is a server-side certificate, this public key will later be used in the master key exchange process. From this, it can be seen that the public key uses RSA encryption. The public key content is 3082010a0282010100ce9da6fb3a940ae04a939b85a1a961…
• Extensions section, some extended information. For example, the aliases of the object. If this pertains to a CDN server certificate, there will be numerous aliases. It includes the domain names of all the websites utilizing the CDN. For instance, with a previously captured CDN certificate from CDNetworks, its extension field subjectAltName contains the following information:

The CDN provides services for 314 domains.

Then there is the certificate authority’s signature information:

• Signature Algorithm, algorithmIdentifier. It is concluded here that the signature algorithm used is SHA-256 hash combined with RSA encryption. This is the signature algorithm used by the CA certificate to authenticate this certificate.
• The signature information, encrypted, contains data where the CA certificate performs a hash on the data section using SHA-256, and then it is encrypted with an RSA private key. Later, during the certificate authentication process, the CA certificate’s public key is used to decrypt the signature information. The same algorithm is then used to generate a hash of the data, and the two hashes are compared to verify if they are identical.

From the issuer mentioned above, we can deduce that the CA certificate authenticating this server certificate is GeoTrust SSL CA – G3. We found the corresponding intermediate certificate under Certificates, as detailed below (there can be multiple levels of intermediate certificates, in this case, there’s only one level):

It is possible to obtain an intermediate certificate named GeoTrust SSL CA – G3, with the certificate organization being GeoTrust Inc.

Authenticate this CA certificate? Still look at the issue field, certify that the certificate name is GeoTrust Global CA, and the organization is also GeoTrust Inc.

In fact, this is the root CA certificate. It wasn’t found in this request, but it can be located in the browser or operating system. Most browsers and systems have this CA certificate built in. Therefore, the root certificate is trusted by the browser or operating system and does not require other certificates for assurance.

If you want your system to trust certain non-standard root CA certificates from some authorities, then go ahead and install them.

For example, my Windows system has the GeoTrust Global CA certificate installed:

https://ask.qcloudimg.com/http-save/yehe-919048/pczzkbkoeh.jpeg

Just as we usually use Charles to intercept HTTPS traffic, this principle involves installing the CA certificate from Charles on a mobile device, making it a trusted root CA certificate.

The fundamental principle is that Charles Proxy acts as an SSL tunnel, not transparently transmitting, but instead intercepting the SSL handshake information as a man-in-the-middle and modifying the CA certificates within. It impersonates the mobile client to establish a connection with the real server to obtain the master key, and then masquerades as the server to establish an SSL connection with the mobile client, altering the server certificate’s CA and digital signature. This way, Charles can decrypt the encrypted HTTP content.

The modified server certificate is shown below. You can see that the issuer has been replaced with Charles’s certificate.

At this stage, we’ve come up with a small idea: is it possible for us to create our own root CA and then distribute it across various devices for use?

Theoretically, it is possible. However, the promotion costs are too high, and in the market, only these few institutions are recognized by most systems:

• Symantec (VeriSign/GeoTrust)
• Comodo
• GoDaddy

Even major enterprises like BAT need to purchase their certificates. For example, this time I captured a package from NetEase, which also uses GeoTrust. However, there are exceptions. Our 12306 tends to be more unconventional and hasn’t purchased certificates from these authorities, so accessing this website often triggers untrusted warnings in browsers like Chrome.

Server Key Exchange

Key exchange phase, this step is optional, supplementing the Certificate phase, and only occurs in the following scenarios:

Negotiation adopted RSA encryption, but the server certificate did not provide an RSA public key.

The negotiation adopted DH (EC Diffie-Hellman) encryption, but the server certificate did not provide DH parameters.

Negotiation to use fortezza_kea encryption is underway, but the server certificate does not provide parameters.

Which scenario did we meet?

It is evident that we previously discussed using the EC Diffie-Hellman algorithm, and since it comes without parameters, this packet is sent from the server to negotiate the DH key parameters.

This package transmits the public key needed for the DH algorithm, i.e., Pubkey: 04a10ad7a23135095205caf7ca8e4c838728e877dbcb23c3…

Similarly, this package also carries a digital signature Signature: 8c7c51f60574144e9e1385a534e12f85911e8dc7cd40dc04…, which can be used to verify its integrity and origin using the public key brought over by the server certificate.

Server Hello Done

Inform the client that the version and cipher suite negotiation has concluded.

This Server Hello Done is akin to the ACK acknowledgment packet in the TCP protocol. Here, the server also provides a confirmation, notifying the client that it is prepared to proceed to the next stage.

During packet capture with Wireshark, a phenomenon was observed where the Server Key Exchange and Server Hello Done messages were included in the same SSL record protocol. This occurs because the SSL record protocol has a combining function. When the client receives such a packet, it processes it into two separate protocol messages, showcasing the SSL record protocol’s segmentation function.

The benefit of doing this is that it can reduce the number of TCP packets sent and decrease the time spent on SSL handshakes.

–Phase Three

In scenarios with high-security levels, the server may also require the client to report a certificate, known as mutual authentication, resulting in a Certificate Request SSL handshake message. In such cases, there will be a comprehensive client certificate submission process to the server. This process is similar to Phase Two.

–Phase Four

Because we only need to verify the server’s certificate here, we can directly proceed to phase four and begin the final handshake. The main purpose of this phase is to generate encryption keys and facilitate their secure transmission.

Client Key Exchange

In this case, the client does not directly generate the encryption key. Instead, a new random number is generated through previously established random numbers between the client and server. This value is then encrypted using the EC Diffie-Hellman algorithm, which was agreed upon earlier, and transmitted to the server. This value is also known as the “premaster secret.”

When the server receives this packet, it will use its private key to decrypt this random number.

After this phase, both the server and the client possess three random numbers: client random number, server random number, and pre-master secret.

After receiving the Client Key Exchange message on the server side, both ends generate the master key according to the corresponding algorithm, completing the encryption key exchange.

Once the exchange is complete, since the master key is generated by both parties according to an agreed-upon algorithm, how can you ensure that this master key is correct?

At this point, it will proceed to the next stage. The client and server will use SHA to create a digest of the handshake information, encrypt it using the AES encryption algorithm and the master key, and transmit it to each other for verification. This method is also known as message authentication. The process is as follows:

Change Cipher Spec (Client)

The client notifies the server that subsequent messages will be encrypted.

Encrypted Handshake Message (Client)

Here is the Client Finished message of the client.

It is also the first encrypted message sent to the server during the entire SSL process.

After receiving the data, the server calculates the digest of the handshake messages that have been exchanged in the same manner. It then compares it with the message decrypted using the master key. If they match, it indicates that the master keys generated on both ends are identical, thus completing the key exchange.

Change Cipher Spec (Server)

The server notifies the client that subsequent messages will be encrypted.

Server Encrypted Handshake Message

Here is the Server Finish message from the server side.

Like the Encrypted Handshake Message from the client above, this is the first encrypted message sent by the server.

After the client decrypts using the agreed-upon master key and verifies its correctness, the SSL handshake phase is completed.

–TLS/SSL Handshake Summary

The main objectives of the entire SSL handshake are to achieve the following:

• Authentication of the server-side identity, or the client side.
• Negotiate a symmetric encryption algorithm and a symmetric encryption key

The entire process of this request took approximately 380ms. It is evident that the SSL handshake significantly consumes request time. Therefore, optimizing the handshake, such as using Session ID or Session Ticket, is advisable. This is similar to the use of Session and Cookie in the HTTP protocol.

3

Data Transmission

After the SSL handshake, the server’s identity authentication succeeded, and the encryption algorithm negotiated is AES, with the key set to xxxxx (calculated by both the client and server using three random values with the same algorithm, without being transmitted in plaintext). Everything is ready.

SSL handshake successful, enabling encryption of subsequent data. Various application layer protocols can now be transmitted securely.

–Application Data

Apply data transmission messages. Since this is HTTPS, the HTTP application protocol data can be encrypted and then transmitted.

From here, without knowing the key, it’s impossible to know what data is being transmitted, or even what protocol is being used for the transmission.

So, to establish an SSL tunnel previously and let the proxy server blindly forward HTTPS data, you need to use the CONNECT method to tell the proxy server which host and port number to connect to; otherwise, the proxy server would be completely bewildered.

Therefore, the SSL protocol is quite independent. It encrypts HTTP here, but it can also encrypt other protocols. It acts like an intermediary layer between TCP and application-layer protocols, providing encrypted data transmission for the upper layer protocols.

–Encryted Alert

SSL warning message: Since the content is encrypted, Wireshark alone cannot reveal the details of the alert.

However, because alert messages are often just used by the client to indicate the end of an SSL transmission to the server, the captured packet content confirms this. Therefore, this is merely a signal indicating the end of SSL transmission.

After issuing the Encrypted Alert, the client data transfer is complete, and preparation begins for the four-way handshake to terminate the TCP connection.

4

I’m here to help with text in the context of WordPress posts structured in HTML. If you could provide me with more context or a post that needs translation, I’d be glad to assist!

After the client sends the HTTP data and correctly receives the server’s response, the next step after completing the HTTPS request is to close the TCP connection. Closing the TCP connection is carried out in four steps, which are also known as the four-way handshake. The corresponding packets are as follows:

You can see that these four packets are only 66 bytes each. This is because they contain no data and consist solely of the TCP header.

–Round 1

Client A sends a FIN + ACK packet, informing the server that data transmission is complete and the connection can be closed. Similarly, this is an ACK acknowledgment packet, indicating that the desired sequence number of the next packet is 3213.

After this phase, the client enters the FIN_WAIT_1 state and stops sending data to the server. However, if the server is still transmitting data, the client’s ACK acknowledgment packets will still be sent.

The server enters the CLOSE_WAIT state. This state is terminated after sending an ACK acknowledgment packet.

–Round 2

Once Receiver B gets the first FIN packet from the sender, it will reply with an ACK acknowledgment packet, which constitutes the second step in the four-way handshake.

Here’s a question: since receiving A’s termination message, why not end it immediately? Because A completes data transmission, but B might still have data that hasn’t been sent, so there will be one more step compared to the three-way handshake.

Upon receiving the client’s FIN packet, the server recognizes that the client has finished transmitting data. Therefore, once the server completes its own data transmission, it can directly inform the client that the connection can be terminated.

After the client receives the ACK acknowledgment packet, it enters the FIN_WAIT_2 state.

–Round 3

If the server-side data has not been completely transmitted, it will continue to be sent to the client.

Once the server has completely transmitted the data, it will send a FIN + ACK packet to notify the client that the data transmission is complete.

After this stage, the server enters the LAST_ACK state, which means it is waiting for the client’s final ACK message.

–Round 4

Sender A, upon receiving the FIN + ACK from B, enters the TIME_WAIT state. The server, after receiving the FIN, transitions to the CLOSED state to terminate the connection.

After the 2MSL time has elapsed without issues, the connection will close. It also enters the CLOSED state.

Why is there a TIME_WAIT?

The reason might be that the server hasn’t received the FIN + ACK, which could trigger a timeout and retransmission, causing the server to send another FIN to the client. The client then has to resend the last packet.

The timing diagram for the TCP four-way handshake is shown below:

5