Reading the fileset from a torrent

Question

I want to (quickly) put a program/script together to read the fileset from a .torrent file. I want to then use that set to delete any files from a specific directory that do not belong to the torrent.

Any recommendations on a handy library for reading this index from the .torrent file? Whilst I don't object to it, I don't want to be digging deep into the bittorrent spec and rolling a load of code from scratch for this simple purpose.

I have no preference on language.

Answer 1

Effbot has your question answered. Here is the complete code to read the list of files from .torrent file (Python 2.4+):

import re

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data

if __name__ == "__main__":
    data = open("test.torrent", "rb").read()
    torrent = decode(data)
    for file in torrent["info"]["files"]:
        print "%r - %d bytes" % ("/".join(file["path"]), file["length"])

Answer 2

I would use rasterbar's libtorrent which is a small and fast C++ library.
To iterate over the files you could use the torrent_info class (begin_files(), end_files()).

There's also a python interface for libtorrent:

import libtorrent
info = libtorrent.torrent_info('test.torrent')
for f in info.files():
    print "%s - %s" % (f.path, f.size)

Answer 3

bencode.py from the original Mainline BitTorrent 5.x client (http://download.bittorrent.com/dl/BitTorrent-5.2.2.tar.gz) would give you pretty much the reference implementation in Python.

It has an import dependency on the BTL package but that's trivially easy to remove. You'd then look at bencode.bdecode(filecontent)['info']['files'].

Answer 4

Expanding on the ideas above, I did the following:

~> cd ~/bin

~/bin> ls torrent*
torrent-parse.py  torrent-parse.sh

~/bin> cat torrent-parse.py
# torrent-parse.py
import sys
import libtorrent

# get the input torrent file
if (len(sys.argv) > 1):
    torrent = sys.argv[1]
else:
    print "Missing param: torrent filename"
    sys.exit()
# get names of files in the torrent file
info = libtorrent.torrent_info(torrent);
for f in info.files():
    print "%s - %s" % (f.path, f.size)

~/bin> cat torrent-parse.sh
#!/bin/bash
if [ $# -lt 1 ]; then
  echo "Missing param: torrent filename"
  exit 0
fi

python torrent-parse.py "$*"

You'll want to set permissions appropriately to make the shell script executable:

~/bin> chmod a+x torrent-parse.sh

Hope this helps someone :)

Answer 5

Here's the code from Constantine's answer above, slightly modified to handle Unicode characters in torrent filenames and fileset filenames in torrent info:

import re

def tokenize(text, match=re.compile("([idel])|(\d+):|(-?\d+)").match):
    i = 0
    while i < len(text):
        m = match(text, i)
        s = m.group(m.lastindex)
        i = m.end()
        if m.lastindex == 2:
            yield "s"
            yield text[i:i+int(s)]
            i = i + int(s)
        else:
            yield s

def decode_item(next, token):
    if token == "i":
        # integer: "i" value "e"
        data = int(next())
        if next() != "e":
            raise ValueError
    elif token == "s":
        # string: "s" value (virtual tokens)
        data = next()
    elif token == "l" or token == "d":
        # container: "l" (or "d") values "e"
        data = []
        tok = next()
        while tok != "e":
            data.append(decode_item(next, tok))
            tok = next()
        if token == "d":
            data = dict(zip(data[0::2], data[1::2]))
    else:
        raise ValueError
    return data

def decode(text):
    try:
        src = tokenize(text)
        data = decode_item(src.next, src.next())
        for token in src: # look for more tokens
            raise SyntaxError("trailing junk")
    except (AttributeError, ValueError, StopIteration):
        raise SyntaxError("syntax error")
    return data

n = 0
if __name__ == "__main__":
    data = open("C:\\Torrents\\test.torrent", "rb").read()
    torrent = decode(data)
    for file in torrent["info"]["files"]:
        n = n + 1
        filenamepath = file["path"]     
        print str(n) + " -- " + ', '.join(map(str, filenamepath))
        fname = ', '.join(map(str, filenamepath))

        print fname + " -- " + str(file["length"])