Thanks Eric.
I ran xfs_bmap -v and it returned extents 0-19999, alternating data
with holes. The holes and data were various sizes, I suppose for xfs
alignment reasons, but everything was there.
Running fiemap again after xfs_bmap still returned 1364 extents.
Fiemap only returns data extents, with holes implied, and comparing
this to a seek_data/seek_hole map, they are identical as far as they
go. The fiemap just poops out early, around the 5GB mark as I recall,
but the file is 41GB. It appears broken to me.
I appreciate the advice about things changing after the fiemap call.
That's true of any backup that isn't of a snapshot, so it's usually a
crap shoot in that regard.
Jim
On 2/15/16, Eric Sandeen <sandeen@xxxxxxxxxxx> wrote:
> On 2/15/16 11:26 AM, Jim Wilcoxson wrote:
>> I'm developing a backup program, HashBackup, and recently added sparse
>> "hole skipping" using fiemap.
>
> That is probably not safe. Mapping can change immediately after fiemap
> returns, and you could miss data. cp went down that path a few years ago,
> and it caused pain.
>
> SEEK_HOLE/SEEK_DATA would be a better choice.
>
>> Today I ran into a weird issue with xfs
>> on Linux 3.10.
>>
>> A test program creates a sparse file with 10000*(4K data, 4M hole).
>> HB calls fiemap with start=0, length=0xFFFF...FFFF, mapped_extents=0
>> to get the number of extents. Fiemap is coming back with
>> mapped_extents=1364 instead of 10000.
>
> Take a look at xfs_bmap -v output to get a clear picture of what
> is actually on disk. It's possible that some combination of
> speculative preallocation or other things has merged extents.
>
> In general, the kernel has plenty of leeway in terms of how it
> chooses to lay out a file. "seek/write" doesn't necessarily
> guarantee a hole.
>
>> The fiemap.txt file says:
>>
>> "If fm_extent_count is zero, then the fm_extents[] array is ignored
>> (no extents will be returned), and the fm_mapped_extents count will
>> hold the number of extents needed in fm_extents[] to hold the file's
>> current mapping."
>>
>> It doesn't say the filesystem can choose to return fewer extents if it
>> wants, but maybe xfs interprets it this way and fiemap has to be
>> called in a loop until the extent_last flag is set. If that's the
>> case, fiemap.txt should be updated.
>
> It's not clear that it is returning fewer extents. See what xfs_bmap
> says, and compare.
>
> But also, don't use fiemap results for anything that requires
> data integrity...
>
> -Eric
>
>> Thanks,
>> Jim
>> (not on the xfs mailing list)
|