xfs
[Top] [All Lists]

Re: inode64 directory placement determinism

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: inode64 directory placement determinism
From: Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx>
Date: Mon, 18 Aug 2014 11:16:12 -0500
Cc: <xfs@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <20140818070153.GL20518@dastard>
References: <e1eeba7b0fb97c63e41f40ed9ac162d7@localhost> <20140818070153.GL20518@dastard>
User-agent: RoundCube Webmail/0.3.1
On Mon, 18 Aug 2014 17:01:53 +1000, Dave Chinner <david@xxxxxxxxxxxxx>
wrote:
> On Sun, Aug 17, 2014 at 10:29:21PM -0500, Stan Hoeppner wrote:
>> Say I have a single 4TB disk in an md linear device.  The md device has
a
>> filesystem on it formatted with defaults.  It has 4 AGs, 0-3.  I have
>> created 4 directories.  Each should reside in a different AG, the first
>> in
>> AG0.  Now I expand the linear device with an identical 4TB disk and
>> execute
>> xfs_growfs.  I now have 4 more AGs, 4-7.  I create 4 more directories.
>> 
>> Will these 4 new dirs be created sequentially in AGs 4-7, or in the
first
>> 4 AGs?  Is this deterministic, or is there any chance involved?  On the
> 
> Deterministic, assuming single threaded *file-system-wide* directory
> creation. Completely unpredictable under concurrent directory
> creations.  See xfs_ialloc_ag_select/xfs_ialloc_next_ag.
> 
> Note that the rotor used to select the next AG is set to
> zero at mount.
> 
> i.e. single threaded behaviour at agcount = 4:
> 
> dir number    rotor value       destination AG
>  1              0                     0
>  2              1                     1
>  3              2                     2
>  4              3                     3
>  5              0                     0
>  6              1                     1
> ....
> 
> So, if you do what you suggest, and grow *after* the first 4 dirs
> are created, the above is what you'll get because the rotor goes
> back to zero on the fourth directory create. Now, with changing from
> 4 to 8 AGs after the first 4:
> 
> dir number    rotor value       new inode location (AG)
>  1              0                     0
>  2              1                     1
>  3              2                     2
>  4              3                     3
> <grow to 8 AGs>
>  5              0                     0
>  6              1                     1
>  7              2                     2
>  8              3                     3
>  9              4                     4
>  10             5                     5
>  11             6                     6
>  13             7                     7
>  14             0                     0
> 
>> real system these 4TB drives are actually 48TB LUNs.  I'm after
>> deterministic parallel bandwidth to subsequently added RAIDs after each
>> grow operation by simply writing to the proper directory.
> 
> Just create new directories and use the inode number to
> determine their location. If the directory is not in the correct AG,
> remove it and create a new one, until you have directories located
> in the AGs you want.
> 
> Cheers,
> 
> Dave.


Thanks for the info Dave.  Was hoping it would be more straightforward. 
Modifying the app for this is out of the question.  They've spent 3+ years
developing with EXT4 and decided to try XFS at the last minute.  Product is
to ship in October, so optimizations I can suggest are limited.


-- 
Stan

<Prev in Thread] Current Thread [Next in Thread>