[tahoe-dev] [tahoe-lafs] #869: Allow Tahoe filesystem to be run over a different key-value-store / DHT implementation

tahoe-lafs trac at tahoe-lafs.org
Thu Jan 6 19:48:54 UTC 2011

#869: Allow Tahoe filesystem to be run over a different key-value-store / DHT
     Reporter:  davidsarah   |       Owner:  nobody                                                                                                         
         Type:  enhancement  |      Status:  new                                                                                                            
     Priority:  major        |   Milestone:  undecided                                                                                                      
    Component:  unknown      |     Version:  1.5.0                                                                                                          
   Resolution:               |    Keywords:  scalability performance  forward-compatibility backward-compatibility availability newcaps docs anti-censorship
Launchpad Bug:               |  

Comment (by warner):

 Replying to [comment:6 davidsarah]:
 > As far as performance is concerned, signature ''verification'' is fast
 > RSA, ECDSA or hash-based signatures (and the hashing can be done
 > incrementally as the share is received, so no significant increase in
 > latency). I don't think this is likely to be a performance bottleneck.

 I'd want to test this with the lowliest of our potential storage servers:
 embedded NAS devices like Pogo-Plugs and !OpenWRT boxes with USB drives
 attached (like Francois' super-slow ARM buildslave). Moving from Foolscap
 HTTP would help these boxes (which find SSL challenging), and doing less
 per share would help. Ideally, we'd be able to saturate the disk bandwidth
 without maxing out the CPU.

 Also, one of our selling points is that the storage server is low-impact:
 want to encourage folks on desktops to share their disk space without
 worrying about their other applications running slowly. I agree that it
 not be a big bottleneck, but let's just keep in mind that our target is
 than 100% CPU consumption.

 Incremental hashing will require forethought in the CHK share-layout and
 the write protocol (the order in which we send out share bits): there are
 plenty of ways to screw it up. Mutable files are harder (you're updating
 existing merkle tree, reading in modified segments, applying deltas,
 rehashing, testing, then committing to disk). The simplest approach would
 involve writing a whole new proposed share, doing integrity checks, then
 replacing the old one.

 > The compatibility impact of changes in the mutable share format would be
 > that an older server is not able to accept mutable shares of the newer
 > version from a newer client. The newer client can still store shares of
 > older version on that server. Grids with a mixture of server and client
 > versions (and old shares) will still work, subject to that limitation.

 Hm, I think I'm assuming that a new share format really means a new
 protocol, so everything about the share is different, and the filecaps
 necessarily change. It wouldn't be possible to produce both "old" and
 shares for a single file. In that case, clients faced with older servers
 either have to reencode the file (and change the filecap, and find
 the old cap was used and replace it), or reduce diversity (you can only
 shares on new servers).

 Migrating existing files to the new format can't be done in a simple
 rebalancing pass (in which you'd only see ciphertext); you'd need
 closer to a {{{cp -r}}}.

 My big concern is that this would slow adoption of new formats like MDMF.
 Since servers should advertise the formats they can understand, I can
 a control panel that shows me grid/server-status on a per-format basis:
 you upload an SDMF file, you can use servers A/B/C/D, but if you upload
 you can only use servers B/C". Clients would need to watch the control
 and not update their config to start using e.g. MDMF until enough servers
 were capable to provide reasonable diversity: not exactly a flag day, but
 a painless upgrade either.

 > On the other hand, suppose that the reason for the change is migration
 to a
 > new signing algorithm to fix a security flaw. In that case, a given
 > can't expect any improvements in security until all servers have

 Incidentally, the security vulnerability induced by such a flaw would be
 limited to availability (and possibly rollback), since that's all the
 can threaten anyways. In this scenario, a non-writecap-holding attacker
 be able to convince the server to modify a share in some invalid way,
 will either result in a (detected) integrity failure or worst-case a
 rollback. Anyways, it probably wouldn't be a fire-drill.

Ticket URL: <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/869#comment:8>
tahoe-lafs <http://tahoe-lafs.org>
secure decentralized storage

More information about the tahoe-dev mailing list