xfs
[Top] [All Lists]

Re: Contiguous file sequences

To: Eric Sandeen <sandeen@xxxxxxxxxxx>
Subject: Re: Contiguous file sequences
From: Daire Byrne <daire.byrne@xxxxxxxxx>
Date: Mon, 27 Sep 2010 17:30:35 +0100
Cc: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=nKbIhf10iKI72TSSGvT24OvhJE0IGFnqWyXXiqgiulo=; b=lhzfJnHFTapTdrxIAtYu+AiWCJz530L9QkV6z2WTaETJ1LaVPFWEpIAJjCK9GkxjuN Fj878fve0StLnKfIReE9UodKSEyhJcXGyqF7LLySAXmCrBn0VWqQZZmEWebs06g1tnFS SC2xMhXZpF5gfWXM20B66kZpHpQYpC5gP/V7o=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=VitVzkI1yPC/yYlTU7ALhMJdNSIr9Co6JDX8fvdK/iRWpjrLjxYbnHnGGbqfs8d8Do IN44Y86hN75cKTLGwJh0E4Ob70+Rwkzzb6yZVnWUwp1/4KqltEE6n2qReYpqGzE+Uxbk OBiEHMTCvsdQaJOeoj/I4ALuzFjdWAk8m+MqQ=
In-reply-to: <4CA018D9.1030803@xxxxxxxxxxx>
References: <AANLkTikHqjvEGJb0XnNy+nz7+nHLVLwjF_wp5RZdk_1-@xxxxxxxxxxxxxx> <4C9A6298.106@xxxxxxxxxxx> <AANLkTinz4rYCb8cHH69pYy-oPT-041y+DmVaWm3N_1hu@xxxxxxxxxxxxxx> <4CA018D9.1030803@xxxxxxxxxxx>
Eric,

On Mon, Sep 27, 2010 at 5:08 AM, Eric Sandeen <sandeen@xxxxxxxxxxx> wrote:
> Daire Byrne wrote:
>>> Why is this the goal, what are you trying to achieve?
>>
>> I am essentially trying to play back a large frame sequence and trying
>> to minimise seeks as it can lead to sporadic slowdowns on a SATA based
>> RAID.
>
> Ok - and you've really seen allocation patterns that cause the playback
> to slow down?  xfs_bmap information for a few sequential files that were
> this far off would be interesting to see.
>
> Are you certain that it's seekiness causing the problem?  A great way
> to visualize it would be to use the seekwatcher application while you
> run a problematic file sequence.

I'm certain that the seekiness is the culprit. The image files are
pretty big and require 400MB/s+ speeds to play them back at full rate.
I can play a sequence which is aligned perfectly on disk just fine
(readahead) but when seeks are required between frames the framerate
drops noticeably. I'm using SATA disks which probably doesn't help
matters.

>>> You can't specify a starting block for any given file I'm afraid.
>>
>> Somebody pointed me at this which looks fairly promising:
>>
>>   http://oss.sgi.com/archives/xfs/2006-07/msg01005.html
>
> Yeah, that never got merged, but I think it still could be.
>
> It's only half your battle though, you need to find that contiguous
> space first, then specify the start block for it with the interface
> above.

I played around with the patch and I think I have a way to do what I
want using something like:

# allocate a big file that all the frames can fit into and hope it is contiguous
BLOCK=`xfs_io -f -c "resvsp 0 $TOTALSIZE" -c "freesp $FRAMESIZE 0" -c
"pwrite 0 1" -c "bmap" $DIR/test.0 | grep "0: \[" | sed 's/\../ /g' |
cut -f5 -d" "`
for x in `seq 1 $FRAMES`; do
    allocnear $DIR/test.$x $BLOCK
    BLOCK=`xfs_io -f -c "bmap" $DIR/test.$x | grep "0: \[" | sed
's/\../ /g' | cut -f5 -d" "`
    dd if=/dev/zero of=$DIR/test.$x bs=1M count=13 conv=notrunc,nocreat
    sync
done

where "allocnear" just creates a new file with the near block hint. It
isn't pretty atm but it does a better job of allocating files without
any block gaps between them. FYI the allocation patch is bypassed on
newer kernels and is useless without modification thanks to:

  http://www.mail-archive.com/ocfs2-devel@xxxxxxxxxxxxxx/msg04387.html

>> I'm still trying to get my head around how I would actually write a
>> userspace app/script to use it but I think it should allow me to do
>> what I want. It would be good if I could script it through xfs_io. I'd
>> really like a script where I could point it at a directory and it
>> would do something like:
>>
>>   1. count total space used by file sequence
>>   2. find start block for that much contiguous space on disk (or as
>> much of it as possible)
>>   3. allocate the files using the start block one after another on disk
>>
>>>> Another option might be to create a single contiguous large file,
>>>> concatenate all the images into it and then split it up on disk using
>>>> offsets but I don't think such a thing is even possible? I always know
>>>> the image sequence size beforehand, all images are exactly the same
>>>> size and I can control/freeze the filesystem access if needed.
>>>>
>>>> Anybody got any suggestions? It *seems* like something that should be
>>>> possible and would be useful.
>>> This would be pretty low-level control of the allocator by userspace.
>>>
>>> I'll just go back and ask what problem you're trying to solve?  There
>>> may be a better (i.e. currently existing) solution.
>>
>> The "realtime" option is sometimes suggested as a way to do sequence
>> streaming but I'd really rather avoid that. It seems to me like the
>> option to allocate a sequence of files end on end in a known chunk of
>> contiguous space is something that would be useful in the normal
>> operating mode.
>
> It would be, but it's not there now.  Also, without some more complexity
> it'd still probably end up being a best effort rather than a guarantee,
> but some hints from userspace might be better than nothing.

I'm pretty sure I can do what I need to do now. Just a case of writing
a userspace application to "defrag" a directory of images now ....

Thanks for the feedback,

Daire

<Prev in Thread] Current Thread [Next in Thread>