[Tahoe-dev] announcing Allmydata-Tahoe v0.3
Zooko O'Whielacronx
zooko at zooko.com
Tue Jun 12 17:25:30 UTC 2007
Dear Folks of tahoe-dev:
I posted this announcement to the p2p-hackers mailing list and there
is already some discusson of it: [1].
Regards,
Zooko
[1] http://lists.zooko.com/pipermail/p2p-hackers/2007-June/001074.html
From: zooko at zooko.com
Subject: [p2p-hackers] announcing Allmydata-Tahoe v0.3
Date: June 11, 2007 11:11:37 AM PDT
To: p2p-hackers at lists.zooko.com
Reply-To: p2p-hackers at lists.zooko.com
Dear p2p-hackers:
Please notice the section "WHAT IS IT GOOD FOR?" in the following
release announcement. :-)
Regards,
Zooko
NEW VERSION RELEASED
We are pleased to announce the release of version 0.3.0 of Allmydata-
Tahoe, a
secure, decentralized storage grid under a free-software licence.
This is
the follow-up to v0.2 which was released May 2, 2007 (see [1]).
Since then we've made several improvements, including:
* fix IP address configuration so that you can run a node behind a
firewall
or NAT box (ticket #22)
* better packaging, build process, command-line, documentation
(tickets #8,
#21, #23, #35, #46, #51)
* handle the loss of peers during the upload process by uploading to the
remaining peers (ticket #17)
* display size of files (ticket #37) (thanks to Faried Nawaz)
* private storage -- upload a file without publishing its decryption key
(ticket #33) (thanks to Faried Nawaz)
* add Merkle-Tree authentication on ciphertext and on plaintext
(ticket #64)
* enable SSL encryption on the connection between web browser and node
(ticket #55)
* improved unit tests including memory-usage tests
* faster unit tests (ticket #20)
* upgrade to a newer version of Foolscap, the remote object protocol
For complete details (there are other improvements that are not in
the above
list), see this web page which shows all ticket changes, repository
checkins,
and wiki changes from May 3 to today: [2].
Allmydata-Tahoe v0.3 is incompatible with v0.2 due to the new Merkle-
Tree
authentication among other things.
WHAT IS IT GOOD FOR?
The source code that we are releasing is the current working
prototype for
Allmydata's next-generation product. This release is targeted at
hackers and
users who are willing to use a minimal, text-oriented web user
interface.
This software is not yet recommended for storage of highly
confidential data
nor for important data which is not otherwise backed up, but it is
useful for
experimentation, prototyping, and extension.
This release of Allmydata-Tahoe marks the first version which is be
suitable
for Use Case #2: "groups of friends who want to share backup and
file-sharing" (see the wiki page "UseCases": [3]). It is easy to set
up a
private grid which is securely shared among a specific, limited set of
friends. Files uploaded to this shared grid will be available to all
friends, even when some of the computers are unavailable.
LICENCE
Tahoe is offered under the GNU General Public License (v2 or later),
with the
added permission that, if you become obligated to release a derived work
under this licence (as per section 2.b), you may delay the
fulfillment of
this obligation for up to 12 months.
INSTALLATION
This release of Tahoe works on Linux/x86, Linux/amd64, Mac/Intel, Mac/
PPC,
Windows-native, and Cygwin.
To install, download the tarball [4], untar it, go into the resulting
directory, and follow the directions in the README [5].
USAGE
Once installed, create a "client node". Instruct this client node to
connect
to a specific "introducer node" by means of config files in the
client node's
working directory. To join a public grid, copy in the .furl files
for that
grid. To create a private grid, run your own introducer, and copy
its .furl
files. See the README for step-by-step instructions.
Each client node runs a local webserver (enabled by writing the
desired port
number into a file called 'webport'). The front page of this
webserver shows
the node's status, including which introducer is being used and which
other
nodes are connected. Links from the status page lead to others that
give
access to a shared virtual filesystem, in which each directory is
represented
by a separate page. Each directory page shows a list of the files
available
there, with download links, and forms to upload new files.
Other ways to access the filesystem are planned, as well as other
structures
than the single globally-shared namespace implemented by this
release: please
see the roadmap.txt [6] for some rough details.
HACKING AND COMMUNITY
Please join the mailing list [7] to discuss the ideas behind Tahoe and
extensions of and uses of Tahoe. Patches that extend and improve
Tahoe are
gratefully accepted -- roadmap.txt shows the next improvements that
we plan
to make. You can browse the revision control history, source code,
and issue
tracking at the Trac instance [8]. Please see the buildbot [9],
which shows
how Tahoe builds and passes unit tests on each checkin, and the code
coverage
results [10] and percentage-covered graph [11], which show how much
of the
Tahoe source code is currently exercised by the test suite.
NETWORK ARCHITECTURE
Each peer maintains a connection to each other peer. A single distinct
server called an "introducer" is used to discover other peers with
which to
connect.
To store a file, the file is encrypted and erasure coded, and each
resulting
share is uploaded to a different peer. The secure hash of the
encrypted file
and the encryption key are packed into a URI, knowledge of which is
necessary
and sufficient to recover the file.
To fetch a file, starting with the URI, a subset of shares is
downloaded from
peers, the file is reconstructed from the shares, and then decrypted.
A single distinct server called a "vdrive server" maintains a global
mapping
from pathnames/filenames to URIs.
We are acutely aware of the limitations of decentralization and
scalability
inherent in this version. In particular, the completely-connected
property
of the grid and the requirement of a single distinct introducer and
vdrive
server limits the possible size of the grid. We have plans to loosen
these
limitations (see roadmap.txt). Currently it should be noted that the
grid
already depends as little as possible on the accessibility and
correctness of
the introduction server and the vdrive server. Also note that the
choice of
which servers to use is easily configured -- you should be able to
set up a
private grid for you and your friends almost as easily as to connect
to our
public test grid.
SOFTWARE ARCHITECTURE
Tahoe is a "from the ground-up" rewrite, inspired by Allmydata's
existing
consumer backup service. It is primarily written in the Python
programming
language.
Tahoe is based on the Foolscap library [12] which provides a remote
object
protocol inspired by the capability-secure "E" programming language
[13].
Foolscap allows us to express the intended behavior of the
distributed grid
directly in object-oriented terms while relying on a well-engineered,
secure
transport layer.
The network layer is provided by the Twisted library [14].
Computationally
intensive operations are performed in native compiled code, such as the
"zfec" library for fast erasure coding (also available separately:
[15]).
Tahoe is sponsored by Allmydata, Inc. [16], a provider of consumer
backup
services. Allmydata, Inc. contributes hardware, software, ideas, bug
reports, suggestions, demands, and money (employing several Allmydata-
Tahoe
hackers and allowing them to spend part of their work time on the
next-generation, free-software project). We are eternally grateful!
Zooko O'Whielacronx
on behalf of the Allmydata-Tahoe team
June 11, 2007
San Francisco
[1] http://allmydata.org/trac/tahoe/browser/relnotes.txt?rev=686
[2] http://allmydata.org/trac/tahoe/timeline?from=06%2F11%
2F2007&daysback=39&changeset=on&milestone=on&ticket=on&wiki=on&update=Up
date
[3] http://allmydata.org/trac/tahoe/wiki/UseCases
[4] http://allmydata.org/source/tahoe/tahoe-0.3.tar.gz
[5] http://allmydata.org/trac/tahoe/browser/README?rev=748
[6] http://allmydata.org/trac/tahoe/browser/roadmap.txt
[7] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
[8] http://allmydata.org/trac/tahoe
[9] http://allmydata.org/buildbot
[10] http://allmydata.org/tahoe-figleaf/figleaf/
[11] http://allmydata.org/tahoe-figleaf-graph/hanford.allmydata.com-
tahoe_figleaf.html
[12] http://twistedmatrix.com/trac/wiki/FoolsCap
[13] http://erights.org/
[14] http://twistedmatrix.com/
[15] http://allmydata.org/trac/tahoe/browser/src/zfec
[16] http://allmydata.com
More information about the tahoe-dev
mailing list