[tahoe-dev] the future of Tahoe-LAFS

Zooko O'Whielacronx zookog at gmail.com
Sat Feb 13 06:28:08 UTC 2010


Folks:

Tahoe-LAFS v1.6 is an excellent release. It has garnered widespread
interest, for example articles in Boing Boing [1], reddit [2],
habrahabr.ru [3], and ossblog.it [4]. It has been accepted into Ubuntu
10.04 LTS "Lucid" [5] and NixOS [6] and Ruben Kerkhof is hard at work
packaging it for Fedora (calling all Fedora hackers: please help him
out!).

Above all, Tahoe-LAFS v1.6 is a solid product on which our users can
rely. Way to go, folks!

I predict that Tahoe-LAFS has reached a "tipping point" where our user
base is going to expand rapidly. This means more successes: more
users, more data, more new ideas built on top of Tahoe-LAFS. It also
means some failures are coming. I learned long ago from Ian Grigg [7]
that you can recognize a successful system by the fact that it is
failing. That's because whenever a system works and is useful and
valuable then people continue to pile more and more load onto it until
it starts breaking. Not just "load" in the sense of a higher volume of
usage (although that too) but also in the sense of expanding the
boundaries of what the thing is used for—we bring in a wider and wider
variety of use cases as we get further and further from what it was
originally designed for.

(When Ian Grigg told me this story, we were discussing this new,
successful, protocol called "HTTP". And by "successful" I mean that it
was breaking.)

My general strategy is to try to make the inevitable failures as
understandable and diagnosable as possible — not just to try to
predict the future use-cases and satisfy them.

So what's coming? Generally speaking, three things: Tahoe-LAFS v1.7,
new applications, and Tahoe-LAFS v2.0.

 * Tahoe-LAFS v1.7: We have a lot of good energy from contributors
going into near-term improvements that will form, in about three
months, the Tahoe-LAFS v1.7 release. I'll write a separate letter
about Tahoe-LAFS v1.7, but in general if you have been following this
mailing list you know what I mean—it looks like our team of volunteers
is already on track to kick out another stellar release featuring
usability and performance improvements, bugfixes, interfaces like SFTP
and FUSE, and forward-compatibility features.

 * new applications: web hosting, file-sharing, "grid-apps" such as my
blog [8], NOSQL-style tools such as hadoop-lafs, a Tahoe-LAFS port of
CouchDB's Javascript-based views, distributed secure objects [9], etc.
Tahoe-LAFS v1.6 looks like a strong foundation on which many
interesting new things could be built, and I expect that a lot more of
those things will be built and deployed in the coming year.

 * Tahoe-LAFS v2.0: We have a few ideas worked out for "great leaps
forward" in Tahoe-LAFS technology that will probably require more than
one standard release cycle (three months) to complete. These include
replacing the storage server protocol (which is currently layered on
top of Foolscap) with a dumber and less Python-specific protocol,
possibly one based on HTTP. This would make it easier for people to
write compatible implementations of the LAFS protocols in other
programming languages. Other big ideas include the New Cap design [10]
and 100-Year Security [11]. My guess is that Brian and David-Sarah and
I (at least) will spend part of our Tahoe-LAFS hacking time on v2.0
work and part of it on v1.7 work this year. Whatever happens
Tahoe-LAFS will of course maintain its strong tradition of
compatibility and stability.

The list of potential improvements above is omitting a lot of
interesting possibilities. For example, I am applying for a grant to
pay Kevan Carstensen to implement the "Redundant Array of Inexpensive
Clouds" idea [12]. That idea is to maintain all of the Tahoe-LAFS
security and reliability properties while delegating the job of bulk
storage to cloud service providers like Amazon S3, Rackspace Cloud
Files, Microsoft Azure, Google Docs (Don't Call It GDrive), etc. If
corporate sponsors decide to give us this grant then Kevan will
definitely implement RAIC this year.

Okay that's my general imagination of what might come in the future.
Please let me know what you think—feedback from users and hackers is
an integral part of our process.

Regards,

Zooko

[1] http://www.boingboing.net/2010/02/05/tahoe-lafs-a-p2p-fil.html
[2] http://www.reddit.com/r/programming/comments/b0by5/tahoelafs_a_p2p_filesystem_that_lets_you_use_the/
[3] http://habrahabr.ru/blogs/p2p/83744/
[4] http://www.ossblog.it/post/5744/tahoe-lafs
[5] https://launchpad.net/ubuntu/lucid/+source/tahoe-lafs
[6] http://hydra.nixos.org/job/nixpkgs/trunk/tahoelafs/all
[7] https://financialcryptography.com
[8] http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:j74uhg25nwdpjpacl6rkat2yhm:kav7ijeft5h7r7rxdp5bgtlt3viv32yabqajkrdykozia5544jqa/wiki.html
[9] http://cap-lore.com/BigStore/Tahoe.html
[10] http://allmydata.org/trac/tahoe-lafs/wiki/NewCapDesign
[11] http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:j74uhg25nwdpjpacl6rkat2yhm:kav7ijeft5h7r7rxdp5bgtlt3viv32yabqajkrdykozia5544jqa/wiki.html#[[can%20we%20build%20a%20crypto%20system%20to%20last%20for%20a%20hundred%20years%3F]]
[12] http://allmydata.org/~zooko/RAIC.png



More information about the tahoe-dev mailing list