[tahoe-dev] Tahoe-LAFS v1.6: only one ticket left open
zooko at zooko.com
Thu Jan 28 16:19:39 UTC 2010
We just committed the fix for #833 -- thanks to David-Sarah for
writing the patch and Kevan for reviewing it. #833 makes it so that
you have the "deep-immutable" guarantee that you would expect from an
immutable directory: if you start in an immutable directory then
*everything* you can reach from that starting point is also
immutable, and you can rely on the fact that anyone else who starts
in the same directory will also see the same things.
#833 also adds some very nice forward-compatibility features, so that
when future versions of Tahoe-LAFS start uploading new kinds of
capabilities, people using Tahoe-LAFS v1.6 will get a reasonable
indicator showing that there is something here that is immutable, but
is from the future so you can't actually see it, instead of getting a
traceback or some sort of behavior that violates the deep-
immutability property. (This is important because Tahoe-LAFS v1.6 is
going into Ubuntu 10.04, and hopefully also into the next release of
Fedora, so there will be many Tahoe-LAFS v1.6 users for a long time
to come.) See the "Memory Lane" matrix of features vs. Tahoe-LAFS
versions on The TestGrid Page  for an example of what a cap from
the future looks like to Tahoe-LAFS v1.6.
That leaves only one ticket open for v1.6, #928. See The Roadmap 
for the 44 closed tickets.
#928 is a good robustness, performance, and load-balancing
improvement for download. In current trunk, whenever you download a
file your Tahoe-LAFS storage client starts by querying every storage
server on the grid to find out what shares each server has. Then it
waits for every query to be answered (or for the storage server to
drop its TCP connection instead of answering the query). Then it
chooses what is basically a random K shares from the set of all
shares that it learned about and downloads those. After #928, it
will not wait to hear the answer to *every* query, but will wait only
until it has learned about K shares. As soon as it learns about K
shares it will proceed to download the file using those.
This has three beneficial effects: first, if there is a server on the
grid which is in a hung state such that it keeps its TCP connections
open but doesn't answer queries, then with current trunk all
downloads from that grid will stall indefinitely. With #928, that
hung server will have no deleterious effect on downloads at all
(provided that K shares can be found on other servers). Second, the
servers which answer the query fastest are the ones that the
downloader uses, which hopefully means that the servers that have a
faster connection to the downloader are the ones that get used, so:
faster downloads! This effect should be more pronounced on grids with
heterogeneous and geographically dispersed servers than on grids with
a bunch of homogeneous servers on the same LAN. Third, servers that
are busier will tend to answer the queries later, so this patch makes
downloaders automatically load-balance to download shares from
servers that are less busy.
It is a nice simple patch, written by me, and reviewed and approved
by Brian, but it isn't in trunk yet because it doesn't yet have
thorough unit tests. (David-Sarah has posted the beginnings of unit
tests and they or I will finish them soon.) If you know how to use
darcs, you can get the most recent darcspatch from #928, apply it,
try it out, and tell us whether downloads go any faster for you with
#928 than without.
As soon as we finish thorough unit tests for #928 I'll commit it to
trunk. In the meantime any reports from the field of success or
failures with current trunk or with current trunk+#928 would be good
to have. Tahoe-LAFS v1.6 is going to be an extremely reliable,
stable new release with major feature and performance improvements.
After #928, the only remaining tasks before release are to write up
relnotes.txt, build binary packages for as many platforms as we can
(help!!), and send the release announcement to mailing lists and news
sites that might be interested.
Thank you all very much for your support of this project.
http://allmydata.org/trac/tahoe/ticket/833# reject mutable children
when *reading* an immutable dirnode
http://allmydata.org/trac/tahoe/ticket/928# start downloading as soon
as you know where to get K shares
More information about the tahoe-dev