new WAPI+WUI proposal

Fri Jul 1 13:41:37 UTC 2016

Hi Brian,

your plans sound very reasonable and exciting!

One option to separate responsibilities very clearly would be to
introduce another port (oh well…) for the separation of the (new) WAPI
and the (new) WUI.

It is surely a matter of taste and semantics but one could argue that
it's a bit funny that a Web API delivers fancy JS-based HTML pages.

Hence, I could imagine creating the following three listeners/ports:

* WAPI (HTTP/CBOR only): note control, directory browsing
* WUI (HTML/JS): HTML user interface to WAPI
* WCD (Web content delivery? However you call it):
  delivery of grid-stored files

That way, the WUI could be handled as separate "app", connecting to the
WAPI like everything else. Maybe there are even scenarios where you
want to distribute those services over multiple machines?

Just 0.02.

Cheers,

Lukas

On 06/29/2016 01:12 AM, Brian Warner wrote as excerpted:
> To support some new accounting / grid-setup / magic-folders workflows
> (in which Tahoe nodes get invited to join a grid using a
> Magic-Wormhole[1] -based protocol), I'd like to add a new web API
> ("WAPI"). I wanted to describe the new protocol and see what people
> think. I also have plans for a new directory-browsing / node-controlling
> web UI ("WUI") that would sit on top of it, to replace the
> server-generated HTML -based WUI we currently have.
> 
> == How the old WAPI/WUI works:
> 
> The old (machine-oriented) WAPI works by making HTTP GET/PUT/POST
> requests to the web.port, generally putting the filecap in the URL path,
> and selecting the operation to perform (upload, download, check, verify,
> repair) by adding query args (like "&t=check") to the URL. To fetch a
> directory, you GET a dircap with "&t=json".
> 
> The old (human-oriented) WUI renders a directory listing when you do a
> GET of a dircap without t=json. The directory page has HTML forms which
> let you upload a file (adding it to the rendered directory),
> rename/delete files, and trigger check/verify/repair operations.
> 
> There's also a "Welcome page" which shows grid membership,
> storage-server status, and lets you perform upload/download operations
> that aren't tied to a specific directory. This page also links to
> "Recent Uploads And Downloads", which presents performance and timing
> data about recent operations, some of which is presented graphically
> (the d3.js-based download timeline). A few of these pages have JSON
> representations, but many do not, and as a result Cypher's "Grid-Sync"
> project has resorted to scraping the HTML to determine e.g. the list of
> connected servers.
> 
> == Problems with the old WAPI/WUI
> 
> There are three main problems:
> 
> * Origin Contamination: All pages retrieved from the same Tahoe
>   client/gateway will share a single "web origin" (usually
>   http://localhost:3456), which means any active Javascript content in
>   those pages shares access to cookies, same-origin tabs, and
>   same-origin browser history. This allows (the code inside) one
>   document to learn dircaps/filecaps of other documents. To address
>   this, we recommend disabling JS for the gateway origin, which
>   obviously limits the kinds of appplications that can be built on top
>   of Tahoe-backed storage. See tickets #615, #821, #1859, and everything
>   with the "websec" keyword.
> 
> * Ambient Authority: We don't like ambient authority, and we assume that
>   an attacker can probably find a way to cause your browser to visit a
>   URL of their own choosing. So the current WAPI is intentionally
>   limited in power: you must give it a filecap before it can do anything
>   with files, and it offers no control over non-filecap things (node
>   admin, storage server controls). It'd be nice if we could use a
>   browser-based UI to e.g. configure periodic backup, or magic-folder
>   directories, but there's not currently a way for the node to
>   distinguish attacker's requests from legitimate ones.
> 
> * No Async Notifications: The WUI only does things in response to a POST
>   or a GET. But there are features we'd like to add that would benefit
>   from asynchronous user notification, like magic-folder conflict
>   detection, completion of long-running check/repair operations, backup
>   progress, etc. The current WUI check/repair has a hacked-up
>   poll-until-complete scheme, but it's pretty ugly and only helps with
>   long-running (but user-triggered) operations, nothing else.
> 
> == New WAPI/WUI
> 
> So I'm looking to build the following:
> 
> * new WAPI, listening on a new local port (so a different HTTP origin)
> * this hosts a WebSocket at a path of "/v1"
> * the websocket accepts CBOR[2]-encoded requests, and returns CBOR
>   responses
> * we'll define new capability tokens which authorize node-admin actions;
>   CLI tools will read these from the node's basedir/private/, and will
>   submit them in the CBOR requests (see [3] for one CLI approach) (see
>   #674)
> 
> * the same port will also serve a new WUI static-HTML directory
> * the port will never provide grid-stored files/directories via GET or
>   POST. The *only* thing retrievable by normal HTML operations will be
>   the static WUI pages, to keep the HTML origin safe
> * the new WUI will be a JS-based single-page app, which provides
>   node control and grid-stored directory navigation
> * when asked to show a grid-stored document, the WUI will render it into
>   a sandboxed iframe[4], like ZeroNet[5] does (which was proposed in
>   ticket #1797)
> 
> We get to define the runtime environment of Tahoe-stored HTML/JS
> document/applications, but I think a reasonable approach would be to
> give the doc/app access to it's parent directory and anything reachable
> from there. I think, with the right combination of ServiceWorkers and
> intercepted requests, we could arrange it so that when you open
> DIRCAP/foo/bar.html, the sandboxed bar.html can retrieve "../other.html"
> (e.g. DIRCAP/other.html) but not "../../outside.html". This would enable
> Zooko's TiddlyWiki and similar apps, but protect against attacks.
> 
> This is a shift from my previous no-JS stance (I've always encouraged
> users to disable javascript for their Tahoe clients on localhost:3456,
> to prevent the one-document-eats-the-other attacks, e.g. #615 and #821).
> It's been about 10 years since I established that opinion.. what's
> changed?:
> 
> * I've learned a bit more JS and see how useful it can be
> * sandboxed iframes are a thing now
> * ServiceWorkers are a thing now
> 
> Also, Accounting needs storagecaps and retrievalcaps (which are
> basically private signing keys that allow the node to make authorized
> upload and download requests), and these ought to be passed into the
> WAPI rather than being ambiently available to all WAPI clients (see
> #587), and I think a JS-based frontend is the only practical way to pass
> in multiple caps (directory writecap + storagecap) at the same time.
> 
> The new WAPI port is also a good place to offer control of
> non-file-based resources. Imagine an "admincap" (just an unguessable
> token) which enables the frontend to add a new local directory to the
> periodic "tahoe backup" list. Or which authorizes an API call that can
> generate or accept Wormhole-style "invitation codes" to exchange
> introducer.furl and accounting keys and an initial shared directory with
> a new node.
> 
> So anyways, I plan to start implementing this at some point, and wanted
> to describe my ideas with enough detail to give people a chance to
> complain first (or, who knows, maybe offer ideas and feedback instead
> :-).
> 
> I'll give folks a chance to chime in here on the mailing list first,
> then I'll write this up in a proper ticket.
> 
> let me know what you think!
>  -Brian
> 
> 
> 
> 
> [1]: http://magic-wormhole.io
> [2]: CBOR is like JSON but includes bytestrings and has a very compact
>      encoding. It actually looks a lot like Banana, the encoding used by
>      Foolscap. https://tools.ietf.org/html/rfc7049 is the spec, and
>      there are multiple libraries for both Python and JS (and others)
> [3]:
> https://github.com/warner/petmail/blob/master/docs/steal_this_software.md#secure-all-http-frontend-access-tokens-event-channels
> [4]: https://html.spec.whatwg.org/#attr-iframe-sandbox
> [5]: http://zeronet.readthedocs.io/en/latest/
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at tahoe-lafs.org
> https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
> 