BitTorrent in Linux

 

          How BitTorrent Works

BitTorrent has three distinct components: the client, the web server, and the tracker. The client is the person/machine that downloads the content. The web server provides a link to a file called a torrent. The torrent is a specially created file that describes the shared file and the location of the tracker. This third component is a service that waits for a connection from a client. It sits on a user-assigned socket that can be either on the same machine as the web server or at another location. The tracker not only supervises the sharing of the content between multiple clients, but also logs all downloading activities. The tracker can manage many files at the same time from many different torrents on many different web servers. You can even refer to the tracker by a torrent that you have downloaded as a file on your machine, eliminating the need for the web server.

 

Beware the trap of false assumptions: surfing and downloading is so familiar that people take it for granted that supplying a file via BitTorrent is pretty much a case of uploading it to some server. No! Peer-to-peer file sharing means that files come from other clients and not from a server. Instead, the server manages the mechanics of sharing the file between clients.

 

Suppose you wish to share an ISO image and you're the only person who has it. Others can have it only if you are running a client yourself. This first client is the seed. If another client comes along and wants to download your content, the tracker guides it over to the first one and his download begins. Now suppose a third client shows up and wants the same content; the tracker coordinates three clients together providing the new content to client No. 3. As more clients connect, the faster the connection becomes, because each client provides additional bandwidth.

 

          The Steps

             Install

If there is not BitTorrent in your system, you should download the source code of Bittorrent-4.4.0 (I have tried some later edition like 5.2.2, but have some import error) from http://download.bittorrent.com/dl/archive/. You need the python 2.x (If use python 2.6 above there may be some warnings). In the follows, I introduce the step of building a BitTorrent environment by using the source code.

             create the torrent

The first step in sharing your file is to create the torrent. The utility is the Python script maketorrent-console.py.

 

Usage: maketorrent-console [OPTION] TRACKER_URL FILE [FILE]

 

arguments are -

--piece_size_pow2 <arg>

          which power of two to set the piece size to (defaults to 18)

 

--tracker_name <arg>

          default tracker name (defaults to 'http://my.tracker:6969/announce')

 

--use_tracker, --no_use_tracker

          if false then make a trackerless torrent, instead of announce URL,

          use reliable node in form of <ip>:<port> or an empty string to pull

          some nodes from your routing table (defaults to True)

 

--data_dir <arg>

          directory under which variable data such as fastresume information

          and GUI state is saved. Defaults to subdirectory 'data' of the

          bittorrent config directory. (defaults to '')

 

--filesystem_encoding <arg>

          character encoding used on the local filesystem. If left empty,

          autodetected. Autodetection doesn't work under python versions older

          than 2.3 (defaults to '')

 

--language <arg>

          ISO Language code to use: af, bg, ca, cs, da, de, el, en, es, es_MX,

          fr, he, hu, is, it, ja, ko, nb_NO, nl, pl, pt, pt_BR, ro, ru, sk, sl,

          sv, tr, vi, zh_CN, zh_TW (defaults to '')

 

--comment <arg>

          optional human-readable comment to put in .torrent (defaults to '')

 

--target <arg>

          optional target file for the torrent (defaults to '')

 

where FILE is the name of the file that I want to share and  TRACKER_URL is the location where I'll install the tracker service.

 

Because this was a test platform, I decided to use the local host URL. The tracker can listen on any socket. The default is port number 80, but the BitTorrent documentation recommends using port 6969. I chose port 8099. You can use either a domain name or an IP address as your URL to the tracker's server. Remember, you must be root to be able to set up a service that listens on any ports less than 1024.

 

From the directory containing the shared file koala, I used the command:

python maketorrent-console.py koala http://localhost:8099/announce

This produced a torrent file named koala.torrent in the same directory of the shared file.

 

The maketorrent-console.py utility creates a hash used to verify the data's integrity as clients download it. The larger the file, the longer it takes to generate the torrent. The resulting torrent file size varies according to the size of the file you want to share. For example, an 11MB tar ball will result produce a torrent of approximately 1.1K. On the other hand, a large ISO of 380MB will increase the torrent to 31K.

 

Notice that I appended the path announce to my local host's URL. This is a hard coded value in BitTorrent that must always be present in the tracker's URL. A real directory by the name of announce is not required to exist on your web server.

             Installing the torrent onto the web server

I have not build a web server and just have an FTP to let the clients download the torrent file.

             Setting up the tracker

Think of the tracker as the middleman to a financial transaction; he doesn't add anything to the product itself, but he makes it possible for both seller and the buyer to meet and carry out the transaction.

 

The utility is the Python script bittorrent-tracker.py.

 

Usage:  bittorrent-tracker.py [OPTION]

--port <arg>

          Port to listen on. (defaults to 80)

 

--dfile <arg>

          file to store recent downloader info in

 

--bind <arg>

          ip to bind to locally (defaults to '')

 

--socket_timeout <arg>

          timeout for closing connections (defaults to 15)

 

--close_with_rst <arg>

          close connections with RST and avoid the TCP TIME_WAIT state

          (defaults to 0)

 

--save_dfile_interval <arg>

          seconds between saving dfile (defaults to 300)

 

--timeout_downloaders_interval <arg>

          seconds between expiring downloaders (defaults to 2700)

 

--reannounce_interval <arg>

          seconds downloaders should wait between reannouncements (defaults to

          1800)

 

--response_size <arg>

          default number of peers to send an info message to if the client does

          not specify a number (defaults to 50)

 

--timeout_check_interval <arg>

          time to wait between checking if any connections have timed out

          (defaults to 5)

 

--nat_check <arg>

          how many times to check if a downloader is behind a NAT (0 = don't

          check) (defaults to 3)

 

--log_nat_checks <arg>

          whether to add entries to the log for nat-check results (defaults to

          0)

 

--min_time_between_log_flushes <arg>

          minimum time it must have been since the last flush to do another one

          (defaults to 3.0)

 

--min_time_between_cache_refreshes <arg>

          minimum time in seconds before a cache is considered stale and is

          flushed (defaults to 600.0)

 

--allowed_dir <arg>

          only allow downloads for .torrents in this dir (and recursively in

          subdirectories of directories that have no .torrent files

          themselves). If set, torrents in this directory show up on

          infopage/scrape whether they have peers or not (defaults to '')

 

--parse_dir_interval <arg>

          how often to rescan the torrent directory, in seconds (defaults to

          60)

 

--allowed_controls <arg>

          allow special keys in torrents in the allowed_dir to affect tracker

          access (defaults to 0)

 

--hupmonitor <arg>

          whether to reopen the log file upon receipt of HUP signal (defaults

          to 0)

 

--show_infopage <arg>

          whether to display an info page when the tracker's root dir is loaded

          (defaults to 1)

 

--infopage_redirect <arg>

          a URL to redirect the info page to (defaults to '')

 

--show_names <arg>

          whether to display names from allowed dir (defaults to 1)

 

--favicon <arg>

          file containing x-icon data to return when browser requests

          favicon.ico (defaults to '')

 

--only_local_override_ip <arg>

          ignore the ip GET parameter from machines which aren't on local

          network IPs (0 = never, 1 = always, 2 = ignore if NAT checking is not

          enabled). HTTP proxy headers giving address of original client are

          treated the same as --ip. (defaults to 2)

 

--logfile <arg>

          file to write the tracker logs, use - for stdout (default) (defaults

          to '')

 

--allow_get <arg>

          use with allowed_dir; adds a /file?hash={hash} url that allows users

          to download the torrent file (defaults to 0)

 

--keep_dead <arg>

          keep dead torrents after they expire (so they still show up on your

          /scrape and web page). Only matters if allowed_dir is not set

          (defaults to 0)

 

--scrape_allowed <arg>

          scrape access allowed (can be none, specific or full) (defaults to

          'full')

 

--max_give <arg>

          maximum number of peers to give with any one request (defaults to

          200)

 

--max_incomplete <arg>

          max number of outgoing incomplete connections (defaults to 100)

 

--twisted <arg>

          Use Twisted network libraries for network connections. 1 means use

          twisted, 0 means do not use twisted, -1 means autodetect, and prefer

          twisted (defaults to -1)

 

--pid <arg>

          Path to PID file (defaults to '/var/run/bittorrent-tracker.pid')

 

This invocation puts the tracker on port 8090, recording all download activity in the file mydownloadlogfile.txt:

python bittorrent-tracker.py. --dfile mydownloadlogfile.txt --port 8090

             Setting up the seed

The last piece of the puzzle is running the client itself. As I mentioned earlier, you need to run a client yourself if you are preparing to share a file for the first time; otherwise there will be no sharing.

 

The utility is the Python script bittorrent-curses.py.

 

Usage: bittorrent-curses [OPTIONS] TORRENTFILE

 

arguments are -

--ip <arg>

          ip to report to the tracker (has no effect unless you are on the same

          local network as the tracker) (defaults to '')

 

--forwarded_port <arg>

          world-visible port number if it's different from the one the client

          listens on locally (defaults to 0)

 

--minport <arg>

          minimum port to listen on, counts up if unavailable (defaults to

          6881)

 

--maxport <arg>

          maximum port to listen on (defaults to 6999)

 

--bind <arg>

          ip to bind to locally (defaults to '')

 

--display_interval <arg>

          seconds between updates of displayed information (defaults to 0.5)

 

--rerequest_interval <arg>

          minutes to wait between requesting more peers (defaults to 300)

 

--min_peers <arg>

          minimum number of peers to not do rerequesting (defaults to 20)

 

--max_initiate <arg>

          number of peers at which to stop initiating new connections (defaults

          to 60)

 

--max_incomplete <arg>

          max number of outgoing incomplete connections (defaults to 100)

 

--max_allow_in <arg>

          maximum number of connections to allow, after this new incoming

          connections will be immediately closed (defaults to 80)

 

--check_hashes, --no_check_hashes

          whether to check hashes on disk (defaults to True)

 

--max_upload_rate <arg>

          maximum kB/s to upload at, 0 means no limit (defaults to 20)

 

--min_uploads <arg>

          the number of uploads to fill out to with extra optimistic unchokes

          (defaults to 2)

 

--max_files_open <arg>

          the maximum number of files in a multifile torrent to keep open at a

          time, 0 means no limit. Used to avoid running out of file

          descriptors. (defaults to 50)

 

--start_trackerless_client, --no_start_trackerless_client

          Initialize a trackerless client. This must be enabled in order to

          download trackerless torrents. (defaults to True)

 

--upnp, --no_upnp

          Enable automatic port mapping (UPnP) (defaults to True)

 

--save_as <arg>

          file name (for single-file torrents) or directory name (for batch

          torrents) to save the torrent as, overriding the default name in the

          torrent. See also --save_in (defaults to '')

 

--max_uploads <arg>

          the maximum number of uploads to allow at once. -1 means a

          (hopefully) reasonable number based on --max_upload_rate. The

          automatic values are only sensible when running one torrent at a

          time. (defaults to -1)

 

--save_in <arg>

          local directory where the torrent contents will be saved. The file

          (single-file torrents) or directory (batch torrents) will be created

          under this directory using the default name specified in the .torrent

          file. See also --save_as. (defaults to '')

 

--responsefile <arg>

          deprecated, do not use (defaults to '')

 

--url <arg>

          deprecated, do not use (defaults to '')

 

--ask_for_save <arg>

          whether or not to ask for a location to save downloaded files in

          (defaults to 0)

 

--spew, --no_spew

          whether to display diagnostic info to stdout (defaults to False)

 

--data_dir <arg>

          directory under which variable data such as fastresume information

          and GUI state is saved. Defaults to subdirectory 'data' of the

          bittorrent config directory. (defaults to '')

 

--filesystem_encoding <arg>

          character encoding used on the local filesystem. If left empty,

          autodetected. Autodetection doesn't work under python versions older

          than 2.3 (defaults to '')

 

--language <arg>

          ISO Language code to use: af, bg, ca, cs, da, de, el, en, es, es_MX,

          fr, he, hu, is, it, ja, ko, nb_NO, nl, pl, pt, pt_BR, ro, ru, sk, sl,

          sv, tr, vi, zh_CN, zh_TW (defaults to '')

 

--keepalive_interval <arg>

          number of seconds to pause between sending keepalives (defaults to

          120.0)

 

--download_slice_size <arg>

          how many bytes to query for per request. (defaults to 16384)

 

--max_message_length <arg>

          maximum length prefix encoding you'll accept over the wire - larger

          values get the connection dropped. (defaults to 8388608)

 

--socket_timeout <arg>

          seconds to wait between closing sockets which nothing has been

          received on (defaults to 300.0)

 

--timeout_check_interval <arg>

          seconds to wait between checking if any connections have timed out

          (defaults to 60.0)

 

--max_slice_length <arg>

          maximum length slice to send to peers, close connection if a larger

          request is received (defaults to 16384)

 

--max_rate_period <arg>

          maximum time interval over which to estimate the current upload and

          download rates (defaults to 20.0)

 

--max_rate_period_seedtime <arg>

          maximum time interval over which to estimate the current seed rate

          (defaults to 100.0)

 

--max_announce_retry_interval <arg>

          maximum time to wait between retrying announces if they keep failing

          (defaults to 1800)

 

--snub_time <arg>

          seconds to wait for data to come in over a connection before assuming

          it's semi-permanently choked (defaults to 30.0)

 

--rarest_first_cutoff <arg>

          number of downloads at which to switch from random to rarest first

          (defaults to 4)

 

--upload_unit_size <arg>

          how many bytes to write into network buffers at once. (defaults to

          1380)

 

--retaliate_to_garbled_data, --no_retaliate_to_garbled_data

          refuse further connections from addresses with broken or

          intentionally hostile peers that send incorrect data (defaults to

          True)

 

--one_connection_per_ip, --no_one_connection_per_ip

          do not connect to several peers that have the same IP address

          (defaults to True)

 

--peer_socket_tos <arg>

          if nonzero, set the TOS option for peer connections to this value

          (defaults to 8)

 

--bad_libc_workaround, --no_bad_libc_workaround

          enable workaround for a bug in BSD libc that makes file reads very

          slow. (defaults to False)

 

--tracker_proxy <arg>

          address of HTTP proxy to use for tracker connections (defaults to '')

 

--close_with_rst <arg>

          close connections with RST and avoid the TCP TIME_WAIT state

          (defaults to 0)

 

--twisted <arg>

          Use Twisted network libraries for network connections. 1 means use

          twisted, 0 means do not use twisted, -1 means autodetect, and prefer

          twisted (defaults to -1)

 

The TORRENTFILE can be either the URL or a file path. The --saveas switch indicates the file path of the existing file I want to share with the world:

python bittorrent-curses.py --save_as ./koala koala.torrent

The client now connects with the tracker, informing it that it is ready to share a copy of koala.

 

The client connects to the tracker, where it will synchronize its data. The larger the file, the longer it takes.

             Setting up the peer

Run the command as follow:

python bittorrent-curses.py koala.torrent --save_as ./koala

It is the same as the seed.

             Stats reporting from the tracker

Getting tracker statistics from my tracker is easy. I just direct my browser to the tracker's port, http://localhost:8099. The resulting HTML page returns data similar to:

 

BitTorrent download info

 

* tracker version: 3.4.2

* server time: 2005-03-29 13:50 UTC

 

               info hash                                    complete      downloading      downloaded

4e98ea442573f5b8868537e970fd3ce6321e9e81            1                  0                          0

                                 0 files            1/1               0/0                       0/0

                                                          

* info hash: SHA1 hash of the "info" section of the metainfo (*.torrent)

* complete: number of connected clients with the complete file

  (total: unique IPs/total connections)

* downloading: number of connected clients still downloading

  (total: unique IPs/total connections)

* downloaded: reported complete downloads (total: current/all)