Understanding Network Events in Nginx: A Deep Dive into Event-Driven Architecture and Packet Handling

Overview

Nginx is an event-driven framework, where events refer to network events. Each connection in Nginx corresponds to two network events, namely read events and write events.

To understand the principles of Nginx and how it handles various extreme scenarios, one must first understand network events.

Network Transmission

Assume host A is your own computer, and host B is a server running Nginx.

What major events occur when host A sends an HTTP GET request to host B?

As shown in the data flow above:

TCP Streams and Packets

What role do network packets play in the above process?

This is a packet, meaning our sent HTTP protocol will be fragmented into many small packets. At the network layer, these fragments are called MTUs, each Ethernet MTU is 1500 bytes;

At the TCP layer (transport layer), the largest MTU value at each intermediate point is considered. At this point, each packet is often only a few hundred bytes, which we call the MSS size. Therefore, receiving a packet smaller than an MSS is effectively a network event.

MTU: Maximum Transmit Unit, the largest transmission unit. It refers to the maximum size of a data packet that can be sent at one time by the physical interface (data link layer) to its upper layer (usually the network layer). An Ethernet frame has a minimum size of 64 bytes and a maximum of 1518 bytes, with data frames smaller or larger than this being considered erroneous. Ethernet forwarding devices typically discard these data frames.Because the Ethernet frame header is 14 bytes and the frame tail CRC check is 4 bytes, taking up 18 bytes in total, the maximum space left for the upper layer protocol, the Data field, is 1500 bytes.

MSS: Maximum Segment Size, a transport layer concept, is the maximum amount of data a TCP packet can transmit at one time. To achieve optimal transmission efficiency, the TCP protocol usually negotiates the MSS value with both parties when establishing a connection. This value is often replaced with the MTU value during TCP protocol implementation (subtract the 20 bytes of the IP header and 20 bytes of the TCP header), so the MSS is often 1460. The smallest MSS value provided by both communicating parties determines the maximum MSS value for this connection.

MTU is the largest transmission unit and refers to the maximum datagram size (in bytes) that a certain layer of a communication protocol can pass. It is often closely related to the link layer protocol.

Due to Ethernet’s electrical transmission limits, each Ethernet frame has a minimum size of 64 bytes and cannot exceed 1518 bytes; any frames below or above this limit are considered erroneous. General Ethernet forwarding devices tend to discard these frames.

The maximum frame size for Ethernet II is 1518 bytes, excluding the 14 bytes of the Ethernet frame header and 4 bytes of the CRC check, leaving a maximum of only 1500 bytes for the Data field, which is known as the MTU.

This MTU is a critical point of concern for the network layer protocols, as they decide whether to fragment the data from the upper layer based on this value. It’s akin to cutting a large piece of bread into smaller slices to fit into several boxes. When two remote PCs are interconnected, their data needs to traverse many routers and various types of network media to reach the opposite endpoint. The MTU differs across various network media, much like a long segment of water pipes composed of different diameters (different MTUs). The maximum flow through this segment is determined by the narrowest pipe.

Non-blocking Interfaces and TCP Protocol

Let’s explore how many events in the TCP protocol are linked to some common interfaces we call, such as Accept, Read, Write, and Close.

Read Events

Write Events

So, what is a write event?

When the response needs to be sent to the browser, the message needs to be written to the operating system, requiring the operating system to send it to the network, which constitutes a write event.

Such network read and write events are typically part of any asynchronous event processing framework, including Nginx, and involve a component known as an event loop.

It defines consumers for each type of event, meaning events are producers that are generated automatically into Nginx via the network, and we must establish a consumer for each type of event.

For example, the event consumer for establishing connections involves calling Accept, the HTTP module will establish a new connection. There are also many read or write messages, where different methods will be called at different times within the HTTP state machine for each consumer to process.

This is an overview of event distribution and consumption, including asynchronous I/O like asynchronous disk read/write events and timer events, like determining whether a timeout has occurred (worker_shutdown_timeout).

Packet Capturing and Analysis with WireShark

WireShark Setup

First, download WireShark and configure it to capture the target IP and port, as shown below.

Click Start

Currently, there is no traffic. We will now access nginx.

Check wireshark

Then

Proceed with the analysis

Packet Analysis

Local IP

The three-way handshake means Windows first sends a [SYN] to Nginx, and conversely, the server hosting Nginx also sends a [SYN] to Windows.

At this point, Nginx is not yet aware of this, as the connection is still in a half-open state. It’s not until the Windows server sends an [ACK] again to the server hosting Nginx that the operating system notifies Nginx of receiving a read event associated with establishing a new connection, prompting Nginx to call the Accept method to establish a new connection.