<div>Hi, Dave,</div><div><br></div><div>Thanks for the answer, it's great, and I apologize for the terrible format.</div><div><br></div><div>>You can't, directly. If you have enough contiguous free space in the</div>
<div>>AG that you are allocating in, then you will get contiguous files if</div><div>>the allocation size lines up with the filesystem geometry:</div><div>></div><div>>$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0 512m" foo.$i ; done</div>
<div>>$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"</div><div>> EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS</div><div>> sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"</div>
<div>> 0: [0..1048575]: 8096..1056671 0 (8096..1056671) 1048576 10000</div><div>> 0: [0..1048575]: 1056672..2105247 0 (1056672..2105247) 1048576 10000</div><div>> 0: [0..1048575]: 2105248..3153823 0 (2105248..3153823) 1048576 10000</div>
<div>> 0: [0..1048575]: 3153824..4202399 0 (3153824..4202399) 1048576 10000</div><div>> 0: [0..1048575]: 4202400..5250975 0 (4202400..5250975) 1048576 10000</div><div>> 0: [0..1048575]: 5250976..6299551 0 (5250976..6299551) 1048576 10000</div>
<div>> 0: [0..1048575]: 6299552..7348127 0 (6299552..7348127) 1048576 10000</div><div>> 0: [0..1048575]: 7348128..8396703 0 (7348128..8396703) 1048576 10000</div><div>> 0: [0..1048575]: 8396704..9445279 0 (8396704..9445279) 1048576 10000</div>
<div>> 0: [0..1048575]: 9445280..10493855 0 (9445280..10493855) 1048576 10000</div><div>></div><div>>So all those files are contiguous both internally and externally. If</div><div>>there isn't sufficient contiguous freespace, or there is allocator</div>
<div>>contention, this won't happen - it's best effort behaviour....</div><div><br></div><div>I believe you got these in a single AG, but I do the allocation in filesystem </div><div>with multi-AGs, specifically, it is a 6T storage space, and I run the mkfs.xfs </div>
<div>without setting the AG number/size, it ends up with 32 AGs.</div><div>My files layout:</div><div> - 0 - dir</div><div> | - 0 - dir</div><div> | | - 1 - file</div>
<div> | | - 2 - file</div><div> | | - 3 - file</div><div> | | - 4 - file</div><div> | | - 5 - file</div><div> | | - ... - file</div>
<div> | | - 128 - file</div><div> | - 1 - dir</div><div> | | - 1 - file</div><div> | | - 2 - file</div><div> | | - 3 - file</div>
<div> | | - 4 - file</div><div> | | - 5 - file</div><div> | | - ... - file</div><div> | | - 128 - file</div><div> | - ... - dir</div>
<div>Every file is 512MB, every directory holds 512MB*128=64GB. </div><div><br></div><div>According to your advice and XFS document, I tried to set the AG size to 64GB, </div><div>for avoiding the allocator contention and keeping all files in single directory </div>
<div>fall in the same AG, but it didn't work. The files are still in different AGs.</div><div><br></div><div>My xfs_info:</div><div>meta-data=/dev/sdc2 isize=256 agcount=96, agsize=16777216 blks</div>
<div>
= sectsz=512 attr=0</div><div>data = bsize=4096 blocks=1610116329, imaxpct=25</div><div> = sunit=0 swidth=0 blks, unwritten=1</div>
<div>naming =version 2 bsize=4096</div><div>log =internal log bsize=4096 blocks=32768, version=1</div><div> = sectsz=512 sunit=0 blks, lazy-count=0</div><div>
realtime =none extsz=4096 blocks=0, rtextents=0</div><div><br></div><div>The files:</div><div>$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0 512m" foo.$i ; done</div>
<div>$ sudo xfs_bmap -vp *| grep " 0:"</div><div> 0: [0..1048575]: 2147483712..2148532287 16 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 3355443264..3356491839 25 (64..1048639) 1048576 10000</div>
<div> 0: [0..1048575]: 2281701440..2282750015 17 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 2415919168..2416967743 18 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 2550136896..2551185471 19 (64..1048639) 1048576 10000</div>
<div> 0: [0..1048575]: 2684354624..2685403199 20 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 2818572352..2819620927 21 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 2952790080..2953838655 22 (64..1048639) 1048576 10000</div>
<div> 0: [0..1048575]: 3087007808..3088056383 23 (64..1048639) 1048576 10000</div><div> 0: [0..1048575]: 3221225536..3222274111 24 (64..1048639) 1048576 10000</div><div><br></div><div>Any way I can specify which AG a file should be allocated? </div>
<div><br></div><div>Again, Thanks for your time.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Nov 7, 2012 at 11:19 AM, Dave Chinner <span dir="ltr"><<a href="mailto:david@fromorbit.com" target="_blank">david@fromorbit.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Wed, Nov 07, 2012 at 09:02:11AM +0800, huubby zhou wrote:<br>
> Hi, folks,<br>
><br>
</div>> I'm using *CentOS5.8*, with *XFS* filesystem extend storage disks. What I<br>
<div class="im">> want to do is, pre-allocating many files, with continuous blocks in<br>
> filesystem, both single file and crossing files. That is the start block ID<br>
> of the next file is following the end block ID of current file.<br>
><br>
</div>> I could do space pre-allocation by *posix_allocate()*, problem is the API<br>
<div class="im">> zeros all disk space, I can't afford time consuming(I'm not sure if this<br>
> API could make the blocks continuous).<br>
<br>
</div>it doesn't on more recent releases - it uses fallocate() in the back<br>
end, which does the same as:<br>
<br>
> then I tried<br>
> *xfsctl()*<<a href="http://linux.die.net/man/3/xfsctl" target="_blank">http://linux.die.net/man/3/xfsctl</a>>,<br>
> with *XFS_IOC_RESVSP* flag, I can reserve space faster.<br>
<br>
This command.<br>
<br>
> The problem with *xfsctl()* is, it could make the blocks continuous for<br>
> individual file, but the blocks over files are *not* continuous. For<br>
<div class="im">> example, 10 files, a/b/c/d/e/f... reserved. After I do the real writing to<br>
> these files, it turns out the file 'b' isn't next to file 'a', and some<br>
> file could be far from both previous one and next one, though other files<br>
> may be neighboring with each other, rarely.<br>
><br>
> I use the following code to do the pre-allocation:<br>
><br>
> ftruncate(fd, FILE_SIZE);<br>
><br>
> xfs_flock_t flag = {0};<br>
> flag.l_whence = SEEK_SET;<br>
> flag.l_start = 0;<br>
> flag.l_len = 512*1024*1024;<br>
> xfsctl(fileName, fd, XFS_IOC_RESVSP64, &flag);<br>
><br>
> My question is, how can I guarantee the file system blocks over files<br>
> continuous? Thanks for your time and appreciate your answer.<br>
<br>
</div>You can't, directly. If you have enough contiguous free space in the<br>
AG that you are allocating in, then you will get contiguous files if<br>
the allocation size lines up with the filesystem geometry:<br>
<br>
$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0 512m" foo.$i ; done<br>
$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"<br>
EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS<br>
sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"<br>
0: [0..1048575]: 8096..1056671 0 (8096..1056671) 1048576 10000<br>
0: [0..1048575]: 1056672..2105247 0 (1056672..2105247) 1048576 10000<br>
0: [0..1048575]: 2105248..3153823 0 (2105248..3153823) 1048576 10000<br>
0: [0..1048575]: 3153824..4202399 0 (3153824..4202399) 1048576 10000<br>
0: [0..1048575]: 4202400..5250975 0 (4202400..5250975) 1048576 10000<br>
0: [0..1048575]: 5250976..6299551 0 (5250976..6299551) 1048576 10000<br>
0: [0..1048575]: 6299552..7348127 0 (6299552..7348127) 1048576 10000<br>
0: [0..1048575]: 7348128..8396703 0 (7348128..8396703) 1048576 10000<br>
0: [0..1048575]: 8396704..9445279 0 (8396704..9445279) 1048576 10000<br>
0: [0..1048575]: 9445280..10493855 0 (9445280..10493855) 1048576 10000<br>
<br>
So all those files are contiguous both internally and externally. If<br>
there isn't sufficient contiguous freespace, or there is allocator<br>
contention, this won't happen - it's best effort behaviour....<br>
<br>
Cheers,<br>
<br>
Dave.<br>
<span class="HOEnZb"><font color="#888888">--<br>
Dave Chinner<br>
<a href="mailto:david@fromorbit.com">david@fromorbit.com</a><br>
</font></span></blockquote></div><br></div>