[tahoe-dev] documenting tahoe's design in standardizable form

[Zooko O'Whielacronx]

Folks:

Here are some notes from my blog [1] about a conversation I had on the phone
with Brian on Friday:

"""
add the following to the list of Things To Do After 1.3.0 release: write up
the notes from the discussion with Brian about how to document
standardizable explanation of tahoe:

   - document 1: file formats: integrity, confidentiality, erasure-coding;
   immutable and mutable files (this would basically be my
lafs.pdf<http://allmydata.org/%7Ezooko/lafs.pdf>with enough detail to
facilitate compatible implementation)
   - document 2: storage server protocol: how to connect to storage servers
   and upload/download files or parts thereof; foolscap protocol (extant), HTTP
   protocol (future work)
   - document 3: peer selection ; This is the hard one. We can document the
   "TahoeTwo" system, which works okay for the allmydata.com use
case<http://allmydata.org/trac/tahoe/wiki/UseCases>,
   but there are all sorts of use cases that are ill-served by the "TahoeTwo"
   design — more than a few hundred servers per grid, non-clique grids,
   secure decentralized introduction, different needs for reliability,
   availability, efficiency, etc. (some user wants Tahoe to store K+1 shares of
   each file in each of their several separate data centers, Shawn wants it to
   store exactly K shares of each file on his mom's computer, if that file is
   tagged as being a family photo), how to tell which grid or grids or
   set-of-servers a given cap can be found in.
   - document 4: directories ; How to encode a set of caps, each annotated
   with metadata such as a name, timestamp, etc., in a file. Traversal caps?

This project is very exciting because it could facilitate other people
re-using tahoe, re-implementing it, or at least stealing its best ideas. The
walls of separation between these four documents are there to limit the
damage from design mistakes, and by the same token to explicitly identify
what each part requires of the other parts.
In particular I am eager to wall off the designs in document 3 from the
others, as it is a field littered with the corpses of attractive but failed
designs.
"""


[1]
http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:j74uhg25nwdpjpacl6rkat2yhm:kav7ijeft5h7r7rxdp5bgtlt3viv32yabqajkrdykozia5544jqa/wiki.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20090207/c89161de/attachment.html>