tests now run 40% faster

Brian Warner warner at lothar.com
Sat Aug 6 18:11:40 UTC 2016


FYI, I just pushed a branch to master which speeds up the unit test
suite by roughly 40%. There were a couple of unnecessary things we were
doing:

* test_mutable was uploading around 900kB, I reduced that to 400kB
* test_mutable had some tests which exercised k=127/N=255, which took
  forever and seemed unnecessary, so I removed them
* most tests that use GridTestMixin were changed to use k=N=1, instead
  of the default k=3/N=10. (some tests exercise partially-uploaded
  files, and thus require more than one share)
* we turned off the (new) control/log tubs in all GridTestMixin tests,
  since they aren't used and their RSA key generation consumes
  significant time
* I also split some large tests into multiple files, which doesn't
  affect the speed but should make things more manageable in the future

The results are pretty cool:

* travis (linux) "trial" time went from 1711s to 1057s (a 40% speedup).
  This is a 'tox -e coveraage' run, which includes branch coverage
  checking, which is the most work that our tests can do. travis(OS-X),
  which does a non-coverage test, had a similar speedup (from 2031s to
  1369s, 32% speedup).

* our benchmark Ubuntu 15.10 buildslave (which runs on a pretty decent
  Linode host) went from 1312s to 727s (45% speedup)

* The Raspberry Pi 3 buildslave, which couldn't even finish the test
  suite before, now finishes in a mere 1.5 hours (5821s). A full 'tox -e
  coverage' on the rpi3 (with all timeouts disabled) used to take 16412s
  (4.5 hours). When I get a chance, I'll make a more direct comparison.

Some things we can do to further speed things up:

* make the mutable publish/update code accept a custom segment size. The
  default 128KiB requires at least 384kB to exercise three-or-more
  segments, but if tests could reduce that to e.g. 128 bytes, we could
  exercise the same code paths with a tiny tiny amount of data. The
  immutable code already does this, but mutable does not. (it's
  non-trivial, but important, since mutations that modify just one
  segment must obviously retain the uploaded file's segsize, and I'm not
  sure we currently have code to modify mutable files with non-default
  segsizes)
* test_system is still the longest single test: we should be able to
  reduce its scope to make it faster

This all falls under ticket 20 [1].

cheers,
 -Brian


[1]:  https://tahoe-lafs.org/trac/tahoe-lafs/ticket/20



More information about the tahoe-dev mailing list