how to land magic-folders?

[Brian Warner]

On the devchat this morning, meejah and I talked about the status of the
"magic folders" project, specifically trying to work out how we could
start landing some of it, to reduce the increasing workload of
maintaining a long-running branch (while trunk has moved out from under
it).

I'd like to hear from Daira and Zooko (and everyone else involved), but
it sounds like the magic-folders branch might be 1: sufficiently
isolated, and 2: sufficiently functional, that we could land it now and
just mark it as "Experimental" without causing anything to explode. If
so, that'd be great, as then the folks hacking on it could do their
continued work on a much shorter branch that should be a lot easier to
maintain (and merge from trunk, or rebase, or however they choose to
manage it).

I've taken the liberty of rebasing the most recent branch that meejah
and I could find, "2438.magic-folder-stable.13", and rebasing it up to
current master. I fixed the merge conflicts along the way (mostly in the
docs, where I'd changed the format of inter-document links, and the 2438
branch had added some text around those links).

I pushed my work to github.com/warner/2438-13-rebased .

The branch is 224 patches long, which is a lot, but I'm willing to land
it (as a merge commit) as-is, if:

* the magic-folder folks say that it makes sense to do so
* it doesn't break anything outside of magic-folders

It looks like the branch removes the "drop-upload" feature (I imagine
magic folders is meant to replace drop-upload). If magic-folders remains
experimental for a while (in particular, if we make the next release
before it's ready), is it going to be a problem to have removed
drop-upload? I want trunk to remain releaseable, and if the current
magic-folder code is not a suitable replacement, then we need to change
this branch to retain drop-upload and provide both features in parallel
for a while.

Looking at the diff (git diff master..warner/2438-13-rebased --stat), I
see the new files, and the expected smaller changes to wires things in
(client.py, interfaces.py, web/root.py, etc). And then I see a bunch of
small changes that don't feel related, like immutable/upload.py . I'm
guessing these are cleanups or refactorings that were done along the
way. It'd be great if we could land these first, so the main "add magic
folders" merge patch only touches the things that are really related to
magic folders.

The way I usually do this (on my own work) is to squash down my feature
branch into a single/few commits, then split them back out into commits
that do cleanups followed by commits that add the feature. For example,
I'll check out the parent commit, perform my cleanup changes, commit,
then overwrite all files with the feature+cleanup versions, then commit
again. Then we can land the first commit on trunk now, and keep working
on the feature branch. This way, the branch gets smaller and smaller
over time until the only thing left on it is the feature itself.

I can try to do this work for the 2438 branch, if you want. If the
cleanup patches are relatively isolated, I can rearrange the branch to
put them near the front. But with such a long branch, that might not be
possible, so we may just have to review those cleanups along with
everything else, and land the whole thing at once.

Anyways, I'm concerned about the maintenance overhead of having those
long-lived large-feature branches hanging around outside the main tree
for a long time, and I'd like to start bringing them in. Let me know
what I can do to help.

cheers,
 -Brian