[tahoe-dev] timestamps on mutable files

Jody Harris imhavoc at gmail.com
Mon Jan 4 18:10:46 UTC 2010


Ack!

So, I'm NOT backing up my data!? (I'm just backing up the first 2MB of each
compressed table?)

Okay, if immutable files are my only choice, how do I do garbage collection
to insure that a minimum of dead space is taken up by "yesterday's" files?

jody
----
- Think carefully.
- Contra mundum - "Against the world" (St. Athanasius)
- Credo ut intelliga - "I believe that I may know" (St. Augustin of Hippo)


On Mon, Jan 4, 2010 at 11:05 AM, Peter Secor <secorp at allmydata.com> wrote:

> Mutable files currently have a small (2MB?) limit and cannot be used for
> larger files. Immutable files (the default) have a much higher limit
> (12GB now?) and are much better suited for the purpose you described of
> large snapshots of data that won't change later.
>
> There are ideas behind larger mutable files (#393) but they are not
> implemented yet.
>
> Ps
>
> On 1/4/10 7:37 AM, Jody Harris wrote:
> > Oh, I forgot to answer the "how?"
> >
> > ... put --mutable ...
> >
> > jody
> > ----
> > - Think carefully.
> > - Contra mundum - "Against the world" (St. Athanasius)
> > - Credo ut intelliga - "I believe that I may know" (St. Augustin of
> Hippo)
> >
> >
> > On Sun, Jan 3, 2010 at 10:59 PM, Jody Harris <imhavoc at gmail.com
> > <mailto:imhavoc at gmail.com>> wrote:
> >
> >     I have a script that:
> >       - iterates though my MySQL tables
> >       - dumps each table
> >       - pipes the dump through gzip
> >       - write the file to a temp file
> >       - copies the file to [remote storage] (now Tahoe grid)
> >       - deletes temp file
> >
> >     This gives me a daily snapshot of all of the active databases on my
> >     server every night.
> >
> >     The current backup is ~ 300 MB/night, which isn't bad, but after one
> >     year, that's 110+GB x 3.3 and counting of consumed tahoe space.
> >     Unless tahoe has a better garbage collection system than I have come
> >     to believe, that's going to be a problem on a 500 GB grid.
> >
> >     In my reading of the tahoe docs, I have not come across any alarms
> >     concerning mutable files. Is there something I should be made aware
> of?
> >
> >     jody
> >     ----
> >     - Think carefully.
> >     - Contra mundum - "Against the world" (St. Athanasius)
> >     - Credo ut intelliga - "I believe that I may know" (St. Augustin of
> >     Hippo)
> >
> >
> >     On Sun, Jan 3, 2010 at 10:50 PM, Zooko O'Whielacronx
> >     <zookog at gmail.com <mailto:zookog at gmail.com>> wrote:
> >
> >         On Sun, Jan 3, 2010 at 10:33 PM, Jody Harris
> >         <havoc at harrisdev.com <mailto:havoc at harrisdev.com>> wrote:
> >          > I have set up my web server to store backup files on my tahoe
> >         grid, shifting
> >          > from the use of a server at my house. One thing that
> >         initially alarmed me
> >          > was that the timestamps of the backed up files is not updated
> >         in the
> >          > tahoe-lafs interface. I did confirm that the shares
> >         timestamps are updated.
> >
> >         Tahoe-LAFS doesn't store timestamps or other metadata on files,
> only
> >         on links to files, which links are stored in directories.  So if
> you
> >         have a directory which contains a link named "README.txt" to
> >         version 1
> >         of your file, and then you upload version 2 of your file and you
> >         change that directory to link to version 2, then Tahoe-LAFS will
> >         update the timestamp on that link named "README.txt" when it
> updates
> >         the link to point to the new file.
> >
> >         Did you say "mutable files"?  You should probably not be using
> >         mutable
> >         files for anything.  Immutable files (plus directories) probably
> fit
> >         all your needs for backups, and mutable files have significant
> >         performance problems.
> >
> >         How do you trigger these backups -- are you using "tahoe cp" or
> >         "tahoe
> >         backup" or sending HTTP requests to the WAPI or something else?
> >
> >         Regards,
> >
> >         Zooko
> >         _______________________________________________
> >         tahoe-dev mailing list
> >         tahoe-dev at allmydata.org <mailto:tahoe-dev at allmydata.org>
> >         http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
> >
> >
> >
> >
> >
> > _______________________________________________
> > tahoe-dev mailing list
> > tahoe-dev at allmydata.org
> > http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
> _______________________________________________
> tahoe-dev mailing list
> tahoe-dev at allmydata.org
> http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tahoe-lafs.org/pipermail/tahoe-dev/attachments/20100104/e3502f06/attachment.html>


More information about the tahoe-dev mailing list