[tahoe-dev] new improved README

zooko zooko at zooko.com
Wed Oct 3 01:29:28 UTC 2007


I've edited the README to take into account user testing feedback  
from Nathan, Darius, and some guy named Peter.  (Sorry if I left  
anyone out of that list.)

Mike Booker is going to be trying out the new README.  Everyone else,  
please feel free to inspect this file for errors and to try following  
its instructions on any platform and reporting if it worked.  (It is  
already believed to work on all Linuxes, Mac OS X, Windows, cygwin,  
and Solaris.)




------- begin appended README
Welcome to the Allmydata-Tahoe project.  This project implements a  
distributed, fault-tolerant storage grid.  All of the source code is  
under a Free Software licence.

The basic idea is that the data in this storage grid is spread over all
participating nodes, using an algorithm that can recover the data  
even if
some of the nodes are not available.

The interface to the storage grid allows you to store and fetch  
files, either
by self-authenticating cryptographic identifier or by filename and path.

See the web site for all kinds of information, news, and community



Pre-compiled binaries are available for Debian or Ubuntu.  Please see  
following web page for instructions:



If you aren't getting a pre-compiled binary, then you'll have to  
ensure that
the following packages are installed before you install Tahoe.

There are two kinds of dependencies, "manual dependencies" and
"easy_install-able dependencies".  The latter kind are normally  
satisfied for you when you install Tahoe, but if something goes  
wrong, please

Note: All of the manual dependencies can probably be installed  
through your
standard package management tool if you are running on a modern Unix
operating system.  For example, on an debian-like system, you can do  
apt-get install gcc make python-dev python-twisted python-pyopenssl".

+ a C compiler (language)

+ GNU make (build tool)

+ Python 2.4 or newer (tested against 2.4, and 2.5.1 ), including
    development headers (language)


+ Twisted Python (tested against 2.2.0, 2.4.0, and 2.5.0) (network and
    operating system integration library)


    You need the following subpackages, which are included in the  
    Twisted distribution:

    * core (the standard Twisted package)
    * web, trial, conch

    Twisted requires zope.interface, a copy of which is included in the
    Twisted distribution. Note that Twisted does *not* require the  
entire Zope
    distribution, merely the much smaller zope.interface component.

+ OpenSSL, including development headers (cryptography library)


+ Python PyOpenSSL (0.6 or later) (secure transport layer)


    To install PyOpenSSL on Windows-native, download this:

    or for Python 2.4, this:


    To install PyOpenSSL on Windows-cygwin, install the OpenSSL  
    libraries with the cygwin package management tool, then get the  
    source code, cd into it, and run "python ./setup.py install".

+ the pywin32 package (210 or later); required only on native Windows  
    required on cygwin)



You need the source code if you are going to install The Debian Way, The
Setuptools Way, or The Running-In-Place Way (see below).  You do not  
need the
source code if you are getting precompiled binaries for Debian or  
Ubuntu (see
above), or if you are going to install The easy_install Way (see below).

The code is available via darcs by running the following command:

darcs get http://allmydata.org/source/tahoe/trunk tahoe

This will create a directory named "tahoe" in the current working  
and put a copy of the latest source code into it.  Later, if you want  
to get
any new changes, then cd into that directory and run the command "darcs

Tarballs of sources are available at:



There are four ways to do it: The easy_install Way, The Setuptools  
Way, The
Running-In-Place Way, and The Debian Way.  Choose one:

The easy_install Way:

   You don't need to download the source code first.

   Tahoe is registered with the Python Package Index (PyPI), so the
   'easy_install' tool can download and install it for you. Just type
   'easy_install allmydata-tahoe' from any shell. That will download  
the most
   recent Tahoe source tarball, unpack it in a temporary directory,  
install it
   to the standard location, then download and install any  
   dependencies that you need (setuptools, zfec, foolscap,  
simplejson, and
   nevow).  (This will work only if you already have the dependencies  
   in the MANUAL DEPENDENCIES section, above.)

The Setuptools Way:

   Get the source code (see above).

   Run 'python setup.py install'. This will compile and install the  
Tahoe code
   to the standard location for your operating system (on unix, that is
   somewhere inside /usr/lib/). It will also acquire and install the
   easy_install-able dependencies (setuptools, zfec, foolscap,  
simplejson, and
   nevow) to the same place.

   (To install it to a non-standard location, see
   http://allmydata.org/trac/tahoe/wiki/SetuptoolsAndGNUStow .)

The Running-In-Place Way:

   You can use Tahoe without installing it.  The steps are these:

   1. Get the source code (see above).

   2. Run "make build-deps" to install the easy_install-able  
      (setuptools, zfec, foolscap, simplejson, and nevow) into a local
      subdirectory of the Tahoe source distribution.

   3. Build Tahoe by running "make".

   4. Once you've built it then you can execute "./bin/allmydata- 
tahoe". (When
      the allmydata-tahoe script is in a Tahoe source distribution,  
it adds
      the necessary directory to the Python "sys.path".  It also  
looks for any
      dependencies that you installed by "make build-deps" and  
includes them
      in the sys.path.)  See the RUNNING section, below.

The Debian Way:

   The Debian Way is to build .deb files which you can then install with

   This requires certain debian packages (build-essential, fakeroot,
   devscripts, debhelper, cdbs) to be installed first, since they are  
used to
   construct the Tahoe .deb files. A full list of these required  
packages can
   be found in the "Build-Depends" line in the misc/DIST/debian/ 
control in the
   top-level tahoe directory (replacing the word DIST with etch,  
dapper, edgy,
   or feisty as appropriate).

   Get the source code (see above).

   If you're running on a debian system, run 'make deb-etch', 'make  
   'make deb-edgy', or 'make deb-feisty' from within the tahoe top-level
   directory to construct a debian package named 'allmydata-tahoe'  
which you
   can then install with dpkg.


'make check-deps' checks that all of the required Python package
dependencies are installed.

'make test' runs the unit test suites.  (This can take a long time on
slow computers.  There are a lot of tests and some of them do a lot of
public-key cryptography.)

Executing the allmydata-tahoe script from the "bin" subdirectory will  
only if Tahoe itself is installed, either because it is installed  
into the
local subdirectory (as per "The Running-In-Place Way") or because it is
installed into your system (as per the other three ways of installing).


Run the "allmydata-tahoe" executable.

If you installed "The Running-In-Place Way", then it is in your  
source tree,
in the "bin" subdirectory thereof.  If you installed in one of the other
three ways, then it has been installed into your operating system's
filesystem, perhaps in "/usr/bin" on Unix, or in "C:\Python25 
\Scripts" on

The "allmydata-tahoe" utility is used to create, start, and stop nodes.
Each node lives in a separate base directory, inside of which you can  
files to configure and control the node.  Nodes also read and write  
within that directory.

A grid consists of a single central 'introducer and vdrive' node and  
one or
more 'client' nodes.  If you are joining an existing grid, the
introducer-and-vdrive node will already be running, and you'll just  
need to
create a client node.  If you're creating a brand new grid, you'll  
need to
create both an introducer-and-vdrive and a client (and then invite other
people to create their own client nodes and join your grid).

The introducer (-and-vdrive) node is constructed by running  
create-introducer --basedir $HERE'.  Once constructed, you can start the
introducer by running 'allmydata-tahoe start --basedir $HERE' (or, if  
are already in the introducer's base directory, just type 'allmydata- 
start').  Inside that base directory, there will be a pair of files
'introducer.furl' and 'vdrive.furl'.  Make a copy of these, as  
they'll be
needed on the client nodes.

To construct a client node, pick a new working directory for it, then  
'allmydata-tahoe create-client --basedir $HERE'.  Copy the two .furl  
from the introducer into this new directory, then run 'allmydata- 
tahoe start
--basedir $HERE'.  After that, the client node should be off and  
The first thing it will do is connect to the introducer and introduce  
to all other nodes on the grid.  You can follow its progress by  
looking at
the $HERE/logs/twistd.log file.

To actually use the client, enable the web interface by writing a port
number (like "8123") into a file named $HERE/webport and then  
restarting the
node with 'allmydata-tahoe restart --basedir $HERE'. This will prompt  
client node to run a webserver on the desired port, through which you  
view, upload, download, and delete files. This 'webport' file is  
actually a
"strports specification", defined in
, so you can have it only listen on a local interface by writing
"tcp:8123:interface=" to this file, or make it use SSL by  
"ssl:8123:privateKey=mykey.pem:certKey=cert.pem" instead.

A client node directory can also be created without installing the code
first.  Just use 'make create-client', and a new directory named  
will be created inside the top of the source tree.  Copy the  
relevant .furl
files in, set the webport, then start the node by using 'make start- 
To stop it again, use 'make stop-client'.  Similar makefile targets  
for making and running an introducer node.

If you are behind a firewall and you can configure your firewall to  
TCP connections on a port to the computer running your Tahoe node,  
then you
can configure the Tahoe node to announce itself as being available on  
IP address and port.  The way to do this is to create a file named
$HERE/advertised_ip_addresses, in which you can put IP addresses and  
numbers in "dotted-quad:port" form, e.g. "".  You  
can put
multiple IP-address-and-port-number entries into this file, on separate

There is a public grid available for testing.  See
http://allmydata.org/trac/tahoe/wiki/TestGrid for the necessary .furl  


This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the  
Software Foundation; either version 2 of the License, or (at your  
any later version, with the added permission that, if you become  
to release a derived work under this licence (as per section 2.b),  
you may
delay the fulfillment of this obligation for up to 12 months.  If you  
obligated to release code under section 2.b of this licence, you are
obligated to release it under these same terms, including the 12- 
month grace
period clause.  See the COPYING file for details.


The following Python packages are required, but normally they are
automatically installed as a side-effect of installing Tahoe.

+ Python setuptools (build and distribution tool)


    The Tahoe install process will automatically download and install
    setuptools if it is not present.  However, if an old,  
incompatible version
    of setuptools is present (< v0.6c6 on Cygwin, or < v0.6a9 on other
    platforms), then the install will fail.

    If the install fails due to your current version of setuptools being
    incompatible, please either upgrade or uninstall your version of
    setuptools and re-run the install.

+ zfec (erasure coding library)


    zfec is packaged in a setuptools-compatible way and included in  
the Python
    Package Index (PyPI), so it will be automatically installed when you
    install Tahoe (see INSTALLING).  It can be manually installed by  
    "easy_install zfec".

+ Python foolscap (secure remote object library)


    foolscape is packaged in a setuptools-compatible way and included  
in the
    Python Package Index (PyPI), so it will be automatically  
installed when
    you install Tahoe (see INSTALLING).  It can be manually installed by
    running "easy_install foolscap".

+ Python simplejson (JSON parser)


    simplejson is packaged in a setuptools-compatible way and  
included in the
    Python Package Index (PyPI), so it will be automatically  
installed when
    you install Tahoe (see INSTALLING).  It can be manually installed by
    running "easy_install simplejson".

+ Python Nevow (0.6.0 or later) (web presentation language)


    Note that the current version of Nevow (0.9.18) requires Twisted  
2.4.0 or

    Nevow is packaged in a setuptools-compatible way and included in the
    Python Package Index (PyPI), so it will be automatically  
installed when
    you install Tahoe (see INSTALLING).  It can be manually installed by
    running "easy_install nevow".

More information about the tahoe-dev mailing list