[email protected]
[Top] [All Lists]

Bug#303741: marked as forwarded (mirrordir should not expand sparse file

Subject: Bug#303741: marked as forwarded mirrordir should not expand sparse files while copying
From: Debian Bug Tracking System
Date: Sun, 24 Apr 2005 15:33:09 -0700
Your message dated Sun, 24 Apr 2005 18:20:28 -0400
with message-id <[email protected]>
has caused the Debian Bug report #303741,
regarding mirrordir should not expand sparse files while copying
to be marked as having been forwarded to the upstream software
author(s) [email protected], [email protected]

(NB: If you are a system administrator and have no idea what I am
talking about this indicates a serious mail system misconfiguration
somewhere.  Please contact me immediately.)

Debian bug tracking system administrator
(administrator, Debian Bugs database)

---------------------------------------
Received: (at 303741-forwarded) by bugs.debian.org; 24 Apr 2005 22:21:02 +0000
>From [email protected] Sun Apr 24 15:21:02 2005
Return-path: <[email protected]>
Received: from rwcrmhc11.comcast.net [204.127.198.35] 
        by spohr.debian.org with esmtp (Exim 3.35 1 (Debian))
        id 1DPpTK-0003vs-00; Sun, 24 Apr 2005 15:21:02 -0700
Received: from vanzandt.comcast.net 
(c-24-61-94-90.hsd1.nh.comcast.net[24.61.94.90](misconfigured sender))
          by comcast.net (rwcrmhc11) with ESMTP
          id <2005042422203001300rssrde>; Sun, 24 Apr 2005 22:20:31 +0000
Received: from jrv by vanzandt.comcast.net with local (Exim 3.36 #1 (Debian))
        id 1DPpSm-0004A5-00; Sun, 24 Apr 2005 18:20:28 -0400
To: [email protected], [email protected]
Cc: Jan Bretschneider <[email protected]>,
   [email protected]
Subject: [[email protected]: Bug#303741: mirrordir should not expand sparse files 
while copying]
Message-Id: <[email protected]>
From: "James R. Van Zandt" <[email protected]>
Date: Sun, 24 Apr 2005 18:20:28 -0400
Delivered-To: [email protected]
X-Spam-Checker-Version: SpamAssassin 2.60-bugs.debian.org_2005_01_02 
        (1.212-2003-09-23-exp) on spohr.debian.org
X-Spam-Status: No, hits=-11.0 required=4.0 tests=BAYES_00,HAS_BUG_NUMBER,
        HAS_PACKAGE autolearn=ham version=2.60-bugs.debian.org_2005_01_02
X-Spam-Level: 


Paul -

Please consider this patch for mirrordir.

                - Jim Van Zandt


------- Start of forwarded message -------
X-Originating-IP: [140.211.166.43]
X-Loop: [email protected]
Subject: Bug#303741: mirrordir should not expand sparse files while copying
Reply-To: Jan Bretschneider <[email protected]>, [email protected]
Resent-From: Jan Bretschneider <[email protected]>
Resent-To: [email protected]
Resent-CC: [email protected] (James R. Van Zandt)
X-Debian-PR-Message: report 303741
X-Debian-PR-Package: mirrordir
X-Debian-PR-Keywords: patch
Date: Fri, 8 Apr 2005 15:58:17 +0200
From: Jan Bretschneider <[email protected]>
To: [email protected]
Content-Disposition: inline
X-Info: valid message
X-Info: original Date
X-Spam-Checker-Version: SpamAssassin 2.60-bugs.debian.org_2005_01_02 
        (1.212-2003-09-23-exp) on spohr.debian.org
X-Spam-Status: No, hits=-8.0 required=4.0 tests=BAYES_00,HAS_PACKAGE 
        autolearn=no version=2.60-bugs.debian.org_2005_01_02
X-Spam-Level: 
Resent-Sender: Debian BTS <[email protected]>


- --Qxx1br4bt0+wmkIi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Package: mirrordir
Version: 0.10.49-intl-1.1
Severity: wishlist
Tags: patch


Hi James,

I have patched mirrordir to handle sparse files. If it encounters a
sparse file, the mirror file will now be sparse, too. So the file won't
get expanded with zero bytes that waste valuable disk space. Instead, a
hole is created as in the original file. This at least works for me and
my local backups.

That might be interesting for those who backup their sparse files with
mirrordir and don't want to spend extra disk space on zero bytes;
someone like me. :-)

The patch is attached. (I couldn't quite grasp the indentation style of
mirrordir's code. Also because spaces and tabs are heavily mixed.)

Maybe you find this worth incorporating; I do. :-) Perhaps you can
forward this to upstream, too.


Ciao, Jan.

- --Qxx1br4bt0+wmkIi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="mirrordir_sparse_files.patch"

Lets mirrordir handle sparse files, i.e., also create sparse files in mirror
path (and don't expand them). Works at least locally. With the changed
buffer size the sparse files in the mirror path take up the same amount of
blocks as the control files. A larger buffer size may yield larger mirror
files because holes of size st_blksize are not found.

diff -ru mirrordir-0.10.49-intl/src/mirrordir.c 
mirrordir-0.10.49-intl.new/src/mirrordir.c
- --- mirrordir-0.10.49-intl.old/src/mirrordir.c        2005-04-08 
14:45:45.737822904 +0200
+++ mirrordir-0.10.49-intl/src/mirrordir.c      2005-04-08 14:25:37.797457752 
+0200
@@ -832,9 +832,11 @@
 int copy_regular_file (char *p, char *q, struct stat *s, struct stat *t)
 {
     long f = 0, g = 0;
- -    char buf[COPY_BUF_SIZE];
+    size_t buf_size = s->st_blksize;
+    char* buf = (char*) alloca (buf_size);
     off_t togo, offset;
     int gzip = 0;
+    int last_write_made_hole = 0;
 
     if (gzip_backups) {
        if (ends_in_dot_gz (q) && !ends_in_dot_gz (p))
@@ -888,7 +890,7 @@
            char *b;
            be_nice_to_cpu ();
            do {
- -             count = gzip < 0 ? gzread ((gzFile) f, buf, COPY_BUF_SIZE) : 
mc_read ((int) f, buf, COPY_BUF_SIZE);
+               count = gzip < 0 ? gzread ((gzFile) f, buf, buf_size) : mc_read 
((int) f, buf, buf_size);
            } while (count < 0 && errno == EINTR);
            if (count <= 0) {
                if (!count)
@@ -904,7 +906,39 @@
            offset += count;
            while (count > 0) {
                do {
- -                 c = gzip > 0 ? gzwrite ((gzFile) g, b, count) : mc_write 
((int) g, b, count);
+            if (gzip > 0)
+            {
+                       c = gzwrite ((gzFile) g, b, count);
+            }
+            else
+            {
+                /* Make a hole iff all bytes in this block are zero. */
+
+                int i;
+                int make_hole = 1;
+                for (i = 0; i < count; ++i)
+                {
+                    if (b[i] != 0)
+                    {
+                        make_hole = 0;
+                        break;
+                    }
+                }
+
+                if (make_hole)
+                {
+                    if (mc_lseek((int) g, count, SEEK_CUR) == (off_t)-1)
+                        progmess_strerror("error trying to lseek file", p);
+
+                    c = count;
+                    last_write_made_hole = 1;
+                }
+                else
+                {
+                    c = mc_write ((int) g, b, count);
+                    last_write_made_hole = 0;
+                }
+            }
                } while (c < 0 && errno == EINTR);
                if (c <= 0) {
                    if (!count)
@@ -920,6 +954,21 @@
            }
        }
     }
+    
+    /* If the file ends with a `hole', something needs to be written at
+       the end.  Otherwise the kernel would truncate the file at the end
+       of the last write operation.  */
+
+    if (gzip <= 0 && last_write_made_hole)
+    {
+        /* Seek backwards one character and write a null. */
+        if (mc_lseek((int)g, -1, SEEK_CUR) == (off_t)-1 ||
+            mc_write((int)g, "", 1) != 1)
+        {
+            progmess_strerror("error trying to lseek or write to file", p);
+        }
+    }
+
     mad_check (__FILE__, __LINE__);
     if (!only_test) {
        f = gzip < 0 ? gzclose ((gzFile) f) : mc_close ((int) f);       /* 
locks are removed when we close the file */

- --Qxx1br4bt0+wmkIi--
------- End of forwarded message -------


-- 
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]

<Prev in Thread] Current Thread [Next in Thread>
  • Bug#303741: marked as forwarded (mirrordir should not expand sparse files while copying), Debian Bug Tracking System <=