NAT and Peer-to-peer networking

 

Peer-to-peer is a style of networking in which a group of computers communicate directly with each other, rather than through a central server. This is often used for multiplayer online games, such as Activision's Battlezone, to avoid the expense and delay of handling all that traffic at the server. However, this style of networking often has problems dealing with Network Address Translators (NATs). In this page, I describe a way to solve these problems. Products that use this technique now work properly through several commercial NATs.

The situation

The Internet is based on 32-bit IP (Internet Protocol) addresses, which means the theoretical maximum number of computers on the Internet is 4 billion or so. The practical limit is much lower, due to inefficiences in how IP addresses are used. In fact, the Internet may be only a few years away from running out of IP addresses.

As IP addresses become scarce, a technique known as Network Address Translation, or NAT, was developed to allow the use of a single IP address for a whole network of computers.

A NAT sits inbetween the public Internet and the network it serves, and works by rewriting IP addresses and port numbers in IP headers on the fly so the packets all appear to be coming from (or going to) the single public IP address of the NAT device instead of the actual source or destination.

NAT is now commonly employed in small home-office routers and in software used by consumers to connect several personal computers to a single cable modem. It is even used by some Internet Service providers.

(NAT is not the only possible solution; proxy servers are also commonly used, but require more configuration, and sometimes require custom client software. Eventually, we'll all switch to IPv6, which will have 128-bit addresses, and will solve the problem once and for all, but that's probably not going to be commonplace for many years.)

Some Protocols Aren't NAT-Friendly

Some applications send IP addresses or port numbers hidden inside their datapackets, where NAT can't properly rewrite them - so those applications don't work when you try to use them on computers behind NATs.

Some NATs, for security reasons, only allow incoming traffic from an outside address if an outgoing packet has already been sent to that outside address. This means that two people behind different NATs can't open up connections to each other in the usual way - ever!

The solution

Peer-to-peer protocols that wish to be NAT-friendly must be aware that any addresses they embed in their data packets may be invalid once the packets pass through the NAT, and compensate accordingly. One way to do it is as follows:

All traffic between the peers is done via a single UDP port. There is an address server which is not behind any NAT. Users connect to the address server first, and send it the IP address they think they have; the server notes both that address and the address it sees in the UDP header. The server then sends both addresses to the other peers. At this point, everyone knows everyone else's address(es).

To open up peer-to-peer connections, all old peers send a UDP packet to the new peer, and the new peer sends a UDP packet to each of the old peers. Since nobody knows at first whether they are behind the same NAT, the first packet is always sent to both the public and the private address.

This causes everyone's NAT to open up a bidirectional hole for the UDP traffic to go through. Once the first reply comes back from each peer, the sender knows which return address to use, and can stop sending to both addresses.

Compatibility Requirements

Above and beyond the basic NAT RFC, a NAT device that wants to support this scheme should have the following desirable property:

NATs should not change the number of UDP ports used by a stream of packets.

Correlary:

If a host behind a NAT sends a series of packets from a single UDP port, the packets as relayed by the NAT should also appear to come from a single host and UDP port.

Draft RFC

I'm working on a draft RFC describing this technique in more detail. Contact me if you're interested.

Compatibility Test Results

I am testing the compatibility of this approach with several NAT implementations. Here are partial results:

Known Compatible NAT Implementations

• NAT1000 - fully compatible. Thanks to Nevod for their early help doing compatibility testing. They didn't need to change anything; the technique just plain worked with their NAT. No longer available; Nevod was bought out by Microsoft.
• Win98 SE contains Internet Connection Sharing software which traces its origins back to NAT1000, so it should work fine.
• SyGate - fully compatible.
• NAT32 - the beta test version released Jan 5th, 1999 is fully compatible. Current releases are probably also fully compatible.
• Linux IP Masquerading - kernel 2.2.1 and later should work fine. See Juanjo's page for the patch for earlier 2.1 kernels. For 2.0 kernels, see the backport to kernel 2.0.36 by Glenn Lamb; Glenn's port makes it a configure-time option CONFIG_IP_MASQ_LOOSE_UDP, which is a nice touch.
• WinNAT - their current release works fine.

NAT Implementations Soon to be Compatible

• Arescom Apex 1100 ISDN Router - Arescom has a firmware patch that fixes the problem as of January 1999; presumably it's in their standard firmware now, but I haven't confirmed it.
• Vicomsoft Softrouter Plus Vicom has since released a version that should fix the problem, but I haven't had a chance to test it. - Note: you may have to disable all native TCP bindings on the gateway machine except for one pointing to the inside ethernet adapter for this implementation to work. Be sure to read their doc all the way through before using their Setup Assistant.

Not yet known to be compatible

• Cisco IOS has a built-in NAT capability that might or might not be compatible.
• We have tested all the win32 software NATs we can get our hands on. We have not yet tested the NATs built into many small home office routers, nor the standalone NATs such as the SonicWall.

Not compatible

• Any pure proxy server solution, such as Wingate 2 or PPPShar, will not work properly.

Software using this technique

The following peer-to-peer network software packages are known to support operation from behind NATs:

• Civilization: Call To Power
• Heavy Gear 2

Discussion Area

I'd love to hear what other developers think about this technique, or about how Masq could be rewritten to reuse UDP ports properly. Join the NAT-peer-games mailing list and let's talk.

Wierd problems

 

While testing Sygate, and later NAT1000, I had some wierd problems. Everything was fine if the machine running the gateway is connected to the Internet via a modem. But if the the gateway machine was connected to the Internet by Ethernet, clients couldn't access other hosts on that outer Ethernet. It's as if packets from the gateway were being utterly ignored by the other hosts on the outside Ethernet. Routers didn't ignore the packets, though, so connecting to distant hosts was no problem. Go figure. See my Usenet post for more info.

I'm inclined to believe this was hardware trouble, but who knows...

Links

• IETF Working Group on NAT - new draft RFC's and mailing list. One of the documents even mentions this technique; search there for "Activision".
• The NAT Page - lists some available NAT implementations
• Linux IP Masquerading - all about Linux's 'Masq' implementation of NAT
• Searchable Index of Masq mailing list
• Linux IP NAT Forum - A paper about NAT, an old and unsupported implementation of NAT for Linux, and a discussion area. Mostly of historical interest.

History

This is to my knowledge a novel technique. I started working with it in 1997, and shipped my first product using it in 1998. This technique was developed while working on multiplayer games at Activision.

---

Copyright 1999 Dan Kegel
dank@alumni.caltech.edu