Seeding Torrents with Amazon S3 and s3cmd on Ubuntu

Amazon Web Services is such a huge, complex service with so many products and features that sometimes very simple but powerful features fall through the cracks when you’re reading the extensive documentation.

One of these features, which has been around for a very long time, is the ability to use AWS to seed (serve) downloadable files using the BitTorrentâ„¢ protocol. You don’t need to run EC2 instances and set up software. In fact, you don’t need to do anything except upload your files to S3 and make them publicly available.

Any file available for normal HTTP download in S3 is also available for download through a torrent. All you need to do is append the string ?torrent to the end of the URL and Amazon S3 takes care of the rest.

Steps

Let’s walk through uploading a file to S3 and accessing it with a torrent client using Ubuntu as our local system. This approach uses s3cmd to upload the file to S3, but any other S3 software can get the job done, too.

  1. Install the useful s3cmd tool and set up a configuration file for it. This is a one time step:

     sudo apt-get install s3cmd
     s3cmd --configure
    

    The configure phase will prompt for your AWS access key id and AWS secret access key. These are stored in $HOME/.s3cmd which you should protect. You can press [Enter] for the encryption password and GPG program. I prefer “Yes” for using the HTTPS protocol, especially if I am using s3cmd from outside of EC2.

  2. Create an S3 bucket and upload the file with public access:

     bucket=YOURBUCKETNAME
     filename=FILETOUPLOAD
     basename=$(basename $filename)
     s3cmd mb s3://$bucket
     s3cmd put --acl-public $filename s3://$bucket/$basename
    
  3. Display the URLs which can be used to access the file through normal web download and through a torrent:

     cat <<EOM
     web:     http://$bucket.s3.amazonaws.com/$basename
     torrent: http://$bucket.s3.amazonaws.com/$basename?torrent
     EOM
    

Notes

  1. The above process makes your file publicly available to anybody in the world. Don't use this for anything you wish to keep private.

  2. You will pay standard S3 network charges for all downloads from S3 including the initial torrent seeding. You do not pay for network transfers between torrent peers once folks are serving the file chunks to each other.

  3. You cannot throttle the rate or frequency of downloads from S3. You can turn off access to prevent further downloads, but monitoring accesses and usage is not entirely real time.

  4. If your file is not popular enough for other torrent peers to be actively serving it, then every person who downloads it will transfer the entire content from S3's torrent servers.

  5. There is no way to force people to use the Torrent URL. If they know what they are doing, they can easily remove "?torrent" and download the entire file direct from S3, perhaps resulting in a higher cost to you.