A colleague, who had just returned from a 50-day business trip, approached me because, on his first day back at work, his PC couldn’t access the Internet. He asked for help in identifying the issue. I inquired about the physical status of the network, and he confirmed there were no apparent problems. However, he mentioned that there was a yellow exclamation mark on the network card icon. This situation often points to ARP issues, so we focused on how to troubleshoot ARP issues to restore his Internet connection.
Possible reasons for not being able to access the Internet:
a. There is a physical fault in the network card connection
b. Unable to obtain the gateway MAC address. ARP -a cannot query the gateway MAC address, and the Internet packet cannot be sent. It may be that DHCP fails to obtain the IP, or there is an IP conflict, or the network card is not in the same broadcast domain as the gateway.
c. DNS failed and could not resolve the destination domain name.
He said he had set a fixed IP address, and asked arp -a to check the MAC address of the gateway 192.168.209.1. The feedback was that it was not found. It was suspected that the core switch did not respond to the ARP request of the PC. The company’s switch had such a problem. When asked about the MAC address of the PC, the answer was as shown below:
I decided to check the Mac address and ARP table on the switch. The known network is as follows:
In the network, switch 209 is used as a Layer 2 access switch, starting with vlan209. The core optical switch 3048 starts with the svi sub-interface address 192.168.209.1 of vlan209 as the gateway for the devices under switch 209.
It is found that this MAC address exists on port 22 of the switch. Clear the MAC address and then search for the corresponding MAC address on the core network switch.
This proves that the Layer 2 broadcast channel is normal, and excludes the possibility that the PC’s network cable and gateway are not in the same broadcast domain. However, the MAC address cannot be found in the ARP cache, as shown below:
It is confirmed that port 6 of the switch is the docking port of the 209 switch. To confirm the problem, mirror the message of port 1/1/6:
Wireshark does pre-filtering and filters ether hw D8CB.8A29.26FC.
It was found that this MAC address used the IP address 169.254.163.76 to send an ARP message to obtain the MAC address of the gateway 192.168.205.1, but there was no response message. This is a failure caused by the failure to obtain the gateway MAC address.
169.254.0.0/16 is a link-local address is an Internet Protocol address used only for communication within the local network (link) or segment of a point-to-point connection that a host is connected to. Routers do not forward packets with link-local addresses. Link-local addresses can be assigned manually by an administrator or by an operating system program. They are mostly assigned automatically using stateless addresses. In IPv4, they usually only appear when no external, stateful mechanism for address configuration exists, such as the Dynamic Host Configuration Protocol (DHCP fails), or when another primary configuration method fails to work (IP conflict).
The network card IP becomes 169.254.xx. One reason is that when the PC network card is started or the DHCP lease is renewed successfully, an IP conflict is detected in the environment, and the operating system allocates a reserved IP address for protection. This address is not legal, so the gateway MAC address cannot be obtained, and operations such as Internet access cannot be achieved. The router does not respond to the ARP request for this type of message, and this address cannot pass through the router.
It should be caused by IP conflict. Can you try to change it to automatic acquisition?
See the captured packets as follows:
It was found that the DHCP process passed normally, the IP obtained was 192.168.209.97, and the core switch responded to the ARP request:
Problem solved.
When the PC is set with a fixed IP or DHCP, the network card will activate the IP, or after the DHCP obtains the IP, it will send three ARP probe messages to see if there is the same IP in the broadcast domain to detect IP conflicts. The characteristic is that the source IP and destination MAC of the ARP are 0. If there is no response, a free ARP message is sent. Look for it in this captured packet, as shown below:
The characteristics of free arp are as follows:
The characteristic of free ARP is that the source and destination IP addresses are the same, and the destination MAC address is 0.
Verify it with your own fixed IP of pc192.168.205.201:
Check the system to see if there is an IP 192.168.205.5 with a MAC address of 0030.18CC.7B40 online. Set it to capture the packet with this IP.
After setting, it is found that the network card logo in the system tray becomes:
Check the network status:
Check and filter the MAC address in the captured packet eth.addr== 44:37:e6:8d:95:27 && arp
It was found that when the ARP probe was performed for the first time, a device with the IP address 0030.18CC.7B40 responded. It was determined that there was a conflict and this IP address could not be used. The system assigned an IP address of 169.254.57.43 and used this IP address to send the same ARP probe message.
Now that I think about it, my colleague’s screenshot at the beginning had the IP address of 169.254.163.76 and the fixed IP 192.168.209.78 he configured as other MAC addresses when queried in the core switch. This way, it can be directly determined that it was an IP conflict problem, and the packet capture took a detour.
It is known that the local Mac address 44-37-E6-8D-95-27 automatically obtains the IP address 192.168.205.57. Then set another laptop in the broadcast domain to this ip address. After changing this local laptop to automatically obtain the IP address, the captured packets are as follows:
After DHCP, 192.168.205.27 is obtained. After ARP detection, it is found that there is a conflict in the environment. This is a temporary use of the local link address 169.254.57.45. The PC sends a DHCP decline message to tell the server that this IP is unavailable and then re-initiates DHCP to obtain the IP address 192.168.205.55. Similarly, no conflict is found in the three detections, and this address is used.
Summary About How to Troubleshoot ARP Issues
a. The gateway’s ARP cannot be obtained. The PC may use the local link address 169.254.xx.xx. There may be an IP conflict in the environment or a conflict in the address obtained by DHCP.
For DHCP servers, some will send an ARP search message to the broadcast domain to address the ability to troubleshoot ARP issues before making an offer, checking for any conflicts. If no conflict is detected, it proceeds with the offer. Alternatively, some servers directly select an unused IP address from the IP pool to make an offer. If the PC receives an IP address and detects a conflict during its checks, it sends a decline message to reject the IP address, then re-initiates the DHCP process until a valid and available IP address is obtained.
When the PC powers on its network card, it initiates the ARP issues troubleshooting process by sending three ARP probe messages with its IP address. If no conflict is detected, it sends a gratuitous ARP message. However, if a conflict arises, it switches to using the local link address 169.254.xx.xx and repeats the process.
d. The gateway does not process the ARP message of the IP address 169.254.xx.xx.
e. If two automatically configured IPv4 addresses appear in the local connection details of the network card, and one of them starts with 169.254, it means that there is an IP conflict.
f. If the device initiates a DHCP message and there is no response in the environment, the DHCP fails to time out and the system will also assign an IP address starting with 169.254. It will also initiate up to three conflict detections, but will not be able to obtain the gateway MAC address.
After the DHCP server assigns an IP address to a MAC address, it establishes an automatic binding relationship between the IP and the MAC address. In scenarios where to troubleshoot ARP issues is necessary, maintaining this binding is crucial to ensure proper network communication. If the user shuts down or disconnects during the lease period and then reconnects within the same period, the server will verify that the IP is not already occupied and will reassign the IP to the MAC address device. This correspondence will become invalid after the server is restarted or the service is terminated, potentially necessitating further ARP issues troubleshooting. When the server receives a release message from the MAC or a discover message again, the binding relationship is canceled. Additionally, if the lease is not renewed upon expiration, the binding relationship is canceled.