[tahoe-dev] atlasgrid performance testing results
warner at lothar.com
Fri Sep 23 06:36:59 UTC 2011
I've spent the last week running some performance tests on trunk (which
will behave very much like the upcoming 1.9 release), using four
LAN-connected hosts generously donated by Atlas Networks.
The report is up on the wiki here:
* MDMF is nice and fast: 4MBps over LAN connections
* my new CHK downloader (which appeared in 1.8.0) is not as fast
(2.6MBps) as the old downloader (from 1.7.1) (4.4MBps)
* high values of "k" really hurt the new CHK downloader (but not MDMF)
* both CHK and MDMF do efficient partial reads (if you only read half
the file, it takes half the time
This analysis reveals a number of fixes that we should experiment with.
The first big one is to implement a scatter/gather readv() method for
immutable-share downloads, to bring them more in line with mutable
retrievals. The more-experimental one is to play with plain HTTP access
for share download: the data suggests that Foolscap message
serialization is pretty expensive.
Tools and raw data are included on that wiki page.
Thanks again to Atlas for lending us thie grid: it makes doing this kind
of testing sooo much easier when you've got four quiet+dedicated
machines with fast network links to each other.
More information about the tahoe-dev