Troubleshooting Video Call Packet Loss: How to Resolve tcpdump Capture Issues

Recently, I’ve been working on video call-related tasks. While analyzing issues like screen flickering and stuttering, my first thought was to capture packets, so I used tcpdump to capture the packets. The command is as follows:

tcpdump -i any -w rtp.pcap

Using wireshark to analyze the RTP stream resulted in the following:

I was surprised at the time. Despite being within the local network, why was there a 10% packet loss in the video?

I quickly turned to Google for answers and found the following potential reasons:

• Firewall
• Insufficient UDP buffer size
• High system load
• Packet loss within the application

We can first rule out the first and third reasons since the test was conducted on a local virtual machine with all firewalls disabled, and the system load was confirmed to be low.

I then verified the fourth reason, packet loss within the application. Here’s the test scenario: 192.168.0.103 is the FreeSWITCH‘s IP. 192.168.0.102 is the IP of the softphone. FreeSWITCH first calls the softphone, and once it connects, FreeSWITCH calls itself and plays an mp4 file (self-calling is a business requirement). The command is as follows:

originate user/1005 &bridge(sofia/internal/[email protected]:5060)

The route is as follows:

From the wireshark analysis, it was clear that there was significant packet loss from 103 to 103. This is easy to test: since tcpdump didn’t capture the packets sent by FreeSWITCH, I just need to verify whether FreeSWITCH actually sent them to confirm if it’s an internal application packet loss.

Thus, after the call began, I used uuid_debug_media xxxxx vwrite on to enable media send statistics and compared the logs with the tcpdump packet capture sequence numbers.

Clearly, FreeSWITCH had already sent the packets, but they weren’t captured, ruling out internal application packet loss.

Analyzing further, the only suspicious reason seemed to be the “insufficient UDP buffer size.” Therefore, I immediately adjusted the kernel’s buffer size:

sysctl -w net.core.rmem_default=16777216
sysctl -w net.core.wmem_default=16777216
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216

After testing, packet loss still occurred.

At this point, I began to suspect that perhaps there wasn’t any actual packet loss in the call, but rather tcpdump failed to capture the packets, thus “showing” a packet loss.

Have any of you ever observed tcpdump‘s output after completing a capture? I certainly hadn’t noticed. I need to self-criticize here; if I had paid attention sooner, it wouldn’t have taken two days to resolve this issue. It really shows how important attention to detail is. 🙂

According to the message, 1503 packets were discarded by the kernel; why were so many packets dropped by the kernel?

You can find some answers here: https://unix.stackexchange.com/questions/144794/why-would-the-kernel-drop-packets (click ‘Read Original’ to view).

It turns out, tcpdump captures raw packets from the network interface, first placing them into a buffer before reading and analyzing them. If tcpdump processes packets slowly, it causes the buffer to overflow, leading to dropped packets.

tcpdump offers a parameter to set buffer size, like this:

tcpdump -i any --buffer-size=409600 -w rtp.pcap

The buffer-size is measured in KB, so the parameter above sets a buffer size of 400MB.

Some older versions of tcpdump allow buffer size configuration using -B 409600.

After testing, wireshark indeed showed no “packet loss.”

The default buffer size for tcpdump is 2MB, which is far from sufficient for capturing video packets, so adding -B is essential.