|Subject:||Re: [ZODB-Dev] Restoring from repozo and reusing an index file?|
|Date:||Fri, 11 Jun 2010 17:24:24 -0400|
On Fri, Jun 11, 2010 at 11:48 AM, Paul Winkler <slinkp@xxxxxxxxx> wrote:
FWIW total time for the second `rsync -z --append Data.fs` was:
It seems that a second rsync isn't exactly blazing fast with a few changes on the end of theÂ 32G Data.fs. Near as I can tell, it spends a good 10 minutes or so just comparing the files to see if it has any work to do.
Once that phase is done, it seems to spend a lot of its time in IO since by default it builds a new file and replaces the existing file when it's done.Â Total time ~ 25 minutes.
The rsync man page paid off though: Using the --append option (or --append-verify on recent enough versions of rsync) seems to reduce the IO a lot, as it's tailor-made for this use case: updating in-place when the source file has only been appended to and potentially losing the target file on failure is OK. (We can manually make a pristine copy prior to starting our downtime, just in case we need to do it over for any reason).
Last time I had to rebuild the index file it took ~ 30 minutes,Â so this looks like a win.Â We'll go with rsync.
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@xxxxxxxx https://mail.zope.org/mailman/listinfo/zodb-dev
|<Prev in Thread]||Current Thread||[Next in Thread>|
|Previous by Date:||Re: [ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?, Christian Theune|
|Next by Date:||Re: [ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?, Chris Withers|
|Previous by Thread:||Re: [ZODB-Dev] Restoring from repozo and reusing an index file?, Paul Winkler|
|Next by Thread:||[ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?, Alan Runyan|
|Indexes:||[Date] [Thread] [Top] [All Lists]|