Peer Exchange in BitTorrent: Understanding its Role and Security Considerations

0. Review

Previous articles:

• BitTorrent Protocol Analysis (Part 1) Metadata File https://cloud.tencent.com/developer/article/2332701
• BitTorrent Protocol Analysis (Part 2) Tracker and Peers https://cloud.tencent.com/developer/article/2333043
• BitTorrent Protocol Analysis (Part 3) Peer Data Transfer Example https://cloud.tencent.com/developer/article/2333677
• BitTorrent Protocol Analysis (Part 4) Distributed Hashing https://cloud.tencent.com/developer/article/2334440
• BitTorrent Protocol Analysis (Part 5) Extension Protocol and Metadata Transfer Extension https://cloud.tencent.com/developer/article/2334776

Review of previous content:

BitTorrent is a protocol used for distributing files. Metadata files are encoded with bencode, with pieces being checked using SHA-1 hashing. The structure of metadata files is introduced, and node information is exchanged via HTTP request from the Tracker, enabling direct node communication.

In the Distributed Hash Table (DHT), each node has its own ID and routing table. Through KRPC, information corresponding to a specific hash can be retrieved in the DHT, and the metadata transfer extension allows the transfer of metadata information between peers, making it possible to obtain metadata via magnet links (using info hashes and other data).

Since the analysis of DHT is complete, let’s also discuss PEX and Local Service Discovery, which often appear alongside DHT. Although the content of IPv6 has not been discussed, this section does not have special handling for IPv6, and everyone should be able to understand given their knowledge.

This content is based on BEP11 (version 7ad40d1b62068015b7b846e7d248f3770d0e9b6a) and BEP14 (version 023256c7581a4bed356e47caf8632be2834211bd). It should be noted that this protocol may change, and it is recommended to analyze it with the latest protocol.

1. Security Issues

Security issues should be addressed at the very beginning. Whether it’s DHT, PEX, or Local Service Discovery, they all have potential risks of being exploited maliciously. Attackers can use these protocols to maliciously announce node addresses to execute DDoS attacks. Therefore, the obtained node information should be considered untrustworthy and potentially malicious. To mitigate these situations, avoid getting all peer connection information from a single node, ignore different ports on the same address, and handle it with techniques from BEP40.

2. Peer Exchange (PEX) Extension

After connecting to other nodes, Peer Exchange is another peer discovery mechanism besides Tracker and DHT. It is implemented based on the extension protocol mentioned in the previous article, with its extension marker as ut_pex. Below is a handshake message supporting the PEX extension, declaring the use of extension information ID 1 for PEX:

Code language: jsonCopy

 {    "m": {        "ut_pex", 1,    },    "p": 6881,    "v": "QCloud_rand 1",}

After completing the extension handshake, nodes can send PEX information, with its message payload as follows:

Code language: jsonCopy

{  "added": "",  "added.f": "", // BitFlag  "added6":"",  "added6.f": "", // BitFlag  "dropped": "",  "dropped6": "",}

At least one of the four flags added, added6, dropped, dropped6 should be present. Other than the flag bits, the content is in a compact string format. When a connection is successfully established, the node is added to “added”, and when the connection breaks, the node is moved to “dropped”. “Added” and “dropped” should be sent promptly, and not sending “dropped” is non-compliant, which helps improve the effectiveness of PEX and reduce the risk of DDoS attacks. The BitFlags content is explained as follows:

Identifier	Description
0x01	Suggest an encrypted connection
0x02	Upload only (seeding state)
0x04	Supports UTP
0x08	Supports ut_holepunch
0x10	Outgoing connection and peer reachable

The frequency of PEX messages does not need to be high, usually not more than once per minute, and PEX messages do not need to be sent immediately after the handshake. Duplicate items should not be included in “added” and “dropped”. Other than the initial data sent, the number of nodes in “added” and “dropped” should not exceed 50, and the client should actively disconnect connections that seriously violate the rules.

Obviously, when a resource is inactive, has many seeders, or when IPv4 and IPv6 selectively disconnect, it leads to low PEX efficiency. Therefore, if less than 25 clients are connected, the criteria for activity can be relaxed, and qualified connections can be placed in “added” or “added6”, even if disconnected:

• Connecting through different addresses with the same peer ID
• Nodes that are deemed unnecessary to connect based on their status and availability
• Connections that are disconnected due to exceeding local resource limits

The rule of tracing the most recently disconnected nodes list to populate PEX applies separately to both IPv4 and IPv6 lists.

Below is an example of a PEX message during the download process of an Ubuntu official image torrent mentioned previously. After initial and extension handshakes, the other party (Transmission) returned PEX information. In the image, the PEX message highlighted by the red box follows two HAVE messages (marked in green):

/>PEX Message

3. Local Peer Discovery

The implementation of Local Peer Discovery essentially utilizes two multicast addresses, 239.192.152.143:6771 (org-local) and ff15::efc0:988f:6771, to broadcast LSD announcement messages on IPv4 and IPv6 networks, similar to SSDP:

Code language: textCopy

BT-SEARCH * HTTP/1.1\r\nHost: \r\nPort: \r\nInfohash: \r\ncookie: \r\n\r\n\r\n

One multicast can include multiple continuous infohashes, allowing the announcement of participation in multiple torrents. However, note the packet length to avoid fragmentation due to MTU. The cookie is a private value to help the sending client filter its own multicast data. Also, note that the multicast address has a broader scope than local multicast, so IP_MULTICAST_TTL should be set appropriately.

Clients participating in multicast should send an LSD announcement every 5 minutes on each interface for listening to BitTorrent connections, and no more than one announcement per minute. If more than five torrents are active, apart from the multiple infohash method mentioned, active torrents can be announced cyclically (some early implementations do not support multiple infohashes in the same announcement). After receiving a multicast announcement, the client must determine which IP address can contact the remote client based on the UDP source address.

4. Private Torrents

Private torrents use private trackers for access control. Torrents with restricted access are known as private torrents, while all others are public. To promote sharing, private trackers usually maintain statistics on registered users and may restrict access or block users who exhibit abnormal data behaviors, only download without uploading, or stop seeding, to ensure a thriving community environment. In the example of creating metadata files mentioned earlier, private torrents are indicated by adding a private=1 key-value pair in the info section.

When a client obtains a declared private metadata file, it must only send announce messages to private trackers and only establish connections with peers returned by the private tracker. Even if a private torrent declares multiple trackers, the client should use only one at a time, and should disconnect all current connections when switching due to failure.

Note: If everyone followed BitTorrent protocols and became excellent downloaders, the need for private trackers would be less significant. However, due to certain behaviors by some downloaders in the past, the current situation arose.

5. Multi-Tracker Extension

Since multiple trackers have been mentioned, it is worth discussing the Multi-Tracker extension, an extension to BitTorrent. In metadata, besides announce, there can also be an announce-list, containing a list of tracker URL strings. If the client is compatible and there is an announce-list, announce should be ignored, even if not present in the announce-list.

More Content

If there are no surprises, this series of articles will continue, with some more content to share. Updated links will be placed here once available:

BitTorrent Protocol Analysis (Part 7) uTorrent Transfer, Hole Punching Extension, and UDP Tracker

Finally, a promotional message for the essay contest this article is participating in:

I am participating in the 2023 Tencent Technology Creation Training Camp Phase 2 Essay Contest, sharing thousands of prize pools and keyboard watches

“`