xfs
[Top] [All Lists]

Re: seeking advice on sparse files on xfs

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: seeking advice on sparse files on xfs
From: Joe Landman <joe.landman@xxxxxxxxx>
Date: Wed, 19 Dec 2012 20:30:58 -0500
Cc: "xfs@xxxxxxxxxxx" <xfs@xxxxxxxxxxx>
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=fpJUFmjvTLh/EV4qyl+fn3nu6tBdJeL7R3AmlsbiYN4=; b=nQTcYol1J62/4kVgUtDtPPV1vUvVMVLvxNjdnR4tPMc8r+y6s6jl1xucrcR+b50aKr Do+OTaUoKur5laHZM2WTwjNVmNLrOSOx90EXK/d+JEc0hDZbhNai0qHKRV4yl4MaeT13 b3MxO7XmtTE2918Zcqzh9ugo9MZMmFIrSKZYm0U39gU0Sd+9a0B6/GOWtVwHB9XrI7Dl 49maPfTzVl09z29qeX6kuonPCVDdo/3kQlWySAK0bHiUWt0hKG7RHc7Fd3HvJetdvUEl yCR5jeq8tDNc0ZMyb4WDE1WQwocyQ4V2WQ3x7qGrmenNIgrYG3QO87wV23HovJ4aty1B SsIg==
In-reply-to: <20121219225406.GO15182@dastard>
References: <50D23D09.3080708@xxxxxxxxx> <20121219225406.GO15182@dastard>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/17.0 Thunderbird/17.0
On 12/19/2012 05:54 PM, Dave Chinner wrote:
On Wed, Dec 19, 2012 at 05:17:45PM -0500, Joe Landman wrote:

[...]

   Pointers appreciated.  I am looking at the copy routines in
coreutils now, looking to see if we can increase its intelligence
somewhat w.r.t. sparse files.

Here's a good overview of the state of play:

http://www.linuxplumbersconf.org/2012/wp-content/uploads/2012/08/sparse-improvements-LPC-2012.pdf


I git cloned the coreutils to look at current state of the code, and saw exactly what is represented on slide 14.

"Brute force – read each sector in full, before
skipping while writing the copy"

There has to be a better way.

And what you really want is a version of cp that supports these:

$ man lseek
....
    Seeking file data and holes
        Since version 3.1, Linux supports the following additional
        values for whence:

        SEEK_DATA
              Adjust  the  file  offset to the next location in the
              file greater than or equal to offset containing data.
              If offset points to data, then the file offset is set
              to offset.

        SEEK_HOLE
              Adjust the file offset to the next hole in the file
              greater than or equal to offset.  If offset points
              into the middle of a hole, then the file offset  is
              set to offset.  If there is no hole past offset, then
              the file offset is adjusted to the end of the file
              (i.e., there is an implicit hole at the end of any
              file).
.....

Something akin to this. Actually would like to be able to have it pull bmap data so that reading over a file only reads populated extents, so that anything that is not populated is nulled out by definition. I am guessing that these are the abstraction above bmap type data?


I'm pretty sure coreutils support is in the pipeline right now....

Hopefully, but it doesn't appear to be in the repository I cloned.  :(

Was thinking of hacking something up at a much higher level (cheating by parsing bmap data and stuff like that).


Cheers,

Dave.


Thanks for the pointers. If I come up with anything marginally useful prior to the SEEK_{HOLE,DATA} implementation, I'll update.

--
Joe

<Prev in Thread] Current Thread [Next in Thread>