devchat notes 11-Jul-2017

[Brian Warner]

Tahoe-LAFS devchat 11-Jul-2017

Attendees: warner, str4d, dawuud, exarkun

* PR415 / I2P Options

https://github.com/tahoe-lafs/tahoe-lafs/pull/415 would add a new
--i2p-i2cp-options=" CLI argument to the "tahoe create-node", which
would get stored in the tahoe.cfg file as "[i2p] i2cp.options=", and
which modify the way the i2p "SAM" connection is established.

This was complicated by the fact that the SAM connection could also be
established by the construction of the server's listening port, via
"[node] tub.port=". Whichever side builds the SAME connection first gets
to set the options, so we have to make sure that both pathways will do
exactly the same thing. That means duplicating the options in two
different places. This was further complicated by the rules for parsing
"tub.port": tahoe splits on commas, then Twisted's
"endpoints.serverFromString" splits on colons. The I2P endpoint parser
can accept extra options (as a dictionary, joined with commas and
colons), but the commas/colons must be escaped to avoid being mistaken
by the earlier two parsers.

We decided to simplify this by letting the i2p-specific code build the
"tub.port" value at runtime, instead of trying to squeeze these extra
options into the normal "tub.port" string. The plan is:

** we'll commandeer the "listen" endpoint type for this purpose
** if "[node] tub.port" has "listen:i2p" as a member, then that means
   the "create_main_tub()" code should ask the "i2p provider" object for
   a list of Endpoints to submit to tub.listenOn, as well as a list of
   strings to append to tub.location
** we'll modify allmydata.util.i2p_provider to parse the "[i2p]" section
   of the config file, and use the options there to build an appropriate
   endpoint for the Foolscap tub to listen on. This will be a txi2p SAM
   endpoint

Basically, since Foolscap can be given a server Endpoint directly
(instead of only being able to build one from a descriptor string), we
can apply the SAM options outside of Foolscap's awareness.

We'll create a trac ticket to explain this issue (and our plan), then
close PR415 in favor of a future PR that implements the scheme in the
ticket.

* travis-ci is currently failing Tor integration test

This showed up on PR 422, which should have passed CI but didn't.

Our Travis-CI builds are defaulting to the old "Precise" (Ubuntu 12.04)
release, which was end-of-lifed a few months ago. The integration test
is trying to download Tor packages from torproject.org, which is
failing, probably because the Tor project no longer supports Precise.

The fix should be to change our .travis.yml to opt-in to a newer Ubuntu
release, probably Xenial (Ubuntu 16.04 LTS).

* PR 422

This PR changes setup.py to allow the use of Python3 in the "sdist"
command. The issue is that most recent Ubuntu systems use a
python3-based Tox, and users who follow our recommended test procedure
("git clone; tox") will get an error, because our setup.py loudly
complains when run with python3. This also causes our xenial buildbot to
be red.

This complaint was added because:

** tox builds an sdist of the package being installed using its own
   python, rather than the python of the virtualenv it just created
** zfec's setup.py uses a python2-style 'print "something"' statement
   (instead of the py2/py3 'print("something")' style)
** so users who do "pip install tahoe-lafs" with a python3-based pip
   will get a mysterious and inscrutable error message

JP pointed out that this is really a bug in Tox, which ought to use the
virtualenv's python to build the sdist, rather than tox's python. They
might have good reasons for doing it that way, though.

Our plan is:

** file a Tox bug
** if they're willing to fix it, great, although we must wait for a full
   release cycle before our buildbot can be green again
** if not, let's get them to commit to exactly which setup.py verbs will
   be run by the tox python (instead of the virtualenv), and land a
   setup.py patch that allows only those specific verbs to pass

Also, we should figure out who owns zfec these days and make a release
that fixes the py2-only syntax in setup.py .

Incidentally, we should also port zfec to py3, as part of a larger
effort to port all of Tahoe to py3. Twisted is now ready, so Foolscap is
really the largest blocker, but we'll need zfec and pycryptopp to be
ported too.

* magic-wormhole / SPAKE2 standardization

LAE and Gridsync are working on a magic-wormhole -based setup flow, and
JP had questions about whether SPAKE2 has been properly standardized yet
(e.g. by the IETF). The answer is no, although Watson Ladd wrote a
(now-expired) Internet-Draft to begin the process. I (warner) am working
on a blog post that describes the decisions one needs to make when
writing an implementation, and how they affect compatibility. I'm also
working on a Rust implementation, and interoperability with
python-spake2 is a top goal.

The eventual RFC is unlikely to be compatible with python-spake2==0.7
(the current version). My plan is:

** assist / wait-for an RFC
** python-spake2 1.0 will have two modes: legacy/0.7 and RFC-compatible
** magic-wormhole will spend an extra round trip figuring out whether
   its peer is capable of using the RFC-compatible SPAKE2 format or not:
   if both sides can speak it, then they'll use the modern one instead
   of the old one

JP was also investigating performance problems when testing their
magic-wormhole system, which might possibly be related to crypto code.
warner identified an import-time slowdown (about 500ms on his laptop)
that will be fixed in an upcoming release, but that didn't seem to the
the same problem. JP will do more research.

cheers,
 -Brian