[tahoe-dev] Getting my root writecap for the production grid

Brian Warner warner-tahoe at allmydata.com
Fri Aug 8 07:28:07 UTC 2008


> After the big discussion about accounting and garbage collection
> that you and me and Greg Hazel had two weeks ago in Boulder, we
> agreed that we needn't start by implementing time-based deletion of
> expired data, but instead we could start with an active "decref this
> now" command to delete data.
> ...
> Did you change your mind or forget our plan about expiry vs. decref  
> since you wrote that message on the 21st?  Or do I misunderstand
> your comments about using expirations in garbage collection in this
> post yesterday?

Nope, I still think active-decref is the best way to start.

It's just that, eventually, we might need to be able to say "ok everybody,
renew your shares, because we suspect that there's a bunch of garbage out
there that we need to get rid of, and on tuesday we're going to expire
everything that hasn't been renewed in a month", or something. We can handle
this for everything that allmydata customers put in their allmydata-visible
directory hierarchy, but anyone else will need to participate in the process
to keep their files alive.

We don't yet know how accurate the active-decref will be.. what rate of
garbage will be generated by things like:

 1: servers being offline when the user removes a file from a directory
 2: users shutting off their computers before the refs can be decremented
 3: bugs
 4: race conditions

So we can't yet rule out timer-based leases. We just know that we're going to
keep them in reserve as a fallback mechanism, and hope that active-decref
works well enough that we never have to expire old shares.

(I plan for the new lease format to include a creation/last-renewal
timestamp, rather than an expiration time, to support this sort of process..
expiring shares will be a manual process, some sort of script we run on the
backend, rather than an automatic 'expire after one month' kind of thing).

cheers,
 -Brian



More information about the tahoe-dev mailing list