xfs
[Top] [All Lists]

Re: bad performance on touch/cp file on XFS system

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: bad performance on touch/cp file on XFS system
From: Greg Freemyer <greg.freemyer@xxxxxxxxx>
Date: Mon, 25 Aug 2014 18:46:31 -0400
Cc: Zhang Qiang <zhangqiang.buaa@xxxxxxxxx>, xfs-oss <xfs@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=M/2sj9S9qg/WfH28NJfx+iVCQWe3NaKlhp2oHDh4+w0=; b=R9/HsT2YAjElaINGw3zFrpuP5o2rwqZ4Z+ee5wPmHWyWoCwtO1NoMHUmfF60vt+MTP 3z8O66FZpG9PiH+GZoa4K/FkOJpNrpqXtKq/Hy4cIbNogynYzRHkfP9FkqHn9xwI13Qk RJhgGXGxaSMFzuft72NbmHa26HhDbYdO/+uPzIlXgMvIzUeQwQbqwus0lKs7s13KdFU8 KtPUfPkehwNBLbeAMn74Uv6PakEQDkPGzieoH9wbFpPl+NCp31dm0EezrXMF1PdZH68D Qegl+VHLwHE6yk27ZvBumQQQRrMxjmJLL/n3DpMUluQTg96Cbdpgd1VVS8OJeuBh6vIb 7ueA==
In-reply-to: <20140825222657.GF20518@dastard>
References: <CAKEtwsWxZseS8M+O7vSR2FRXr4gjVQ0RDO8ok+jMPWq-8jPEeA@xxxxxxxxxxxxxx> <20140825051801.GY26465@dastard> <CAKEtwsXiVKTWAW+YszjNnFnD4_Ld7g2qXEvw48A-SitYSGyXHA@xxxxxxxxxxxxxx> <20140825090843.GE20518@dastard> <CAKEtwsU4gywG7fVVMVU1Y_TG9Pgg_-sFV0=SPg_7Ob5EV6FTew@xxxxxxxxxxxxxx> <20140825222657.GF20518@dastard>
On Mon, Aug 25, 2014 at 6:26 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
> On Mon, Aug 25, 2014 at 06:31:10PM +0800, Zhang Qiang wrote:
>> 2014-08-25 17:08 GMT+08:00 Dave Chinner <david@xxxxxxxxxxxxx>:
>>
>> > On Mon, Aug 25, 2014 at 04:47:39PM +0800, Zhang Qiang wrote:
>> > > I have checked icount and ifree, but I found there are about 11.8 percent
>> > > free, so the free inode should not be too few.
>> > >
>> > > Here's the detail log, any new clue?
>> > >
>> > > # mount /dev/sda4 /data1/
>> > > # xfs_info /data1/
>> > > meta-data=/dev/sda4              isize=256    agcount=4, agsize=142272384
>> >
>> > 4 AGs
>> >
>> Yes.
>>
>> >
>> > > icount = 220619904
>> > > ifree = 26202919
>> >
>> > And 220 million inodes. There's your problem - that's an average
>> > of 55 million inodes per AGI btree assuming you are using inode64.
>> > If you are using inode32, then the inodes will be in 2 btrees, or
>> > maybe even only one.
>> >
>>
>> You are right, all inodes stay on one AG.
>>
>> BTW, why i allocate 4 AGs, and all inodes stay in one AG for inode32?,
>
> Because the top addresses in the 2nd AG go over 32 bits, hence only
> AG 0 can be used for inodes. Changing to inode64 will give you some
> relief, but any time allocation occurs in AG0 is will be slow. i.e.
> you'll be trading always slow for "unpredictably slow".
>
>> > With that many inodes, I'd be considering moving to 32 or 64 AGs to
>> > keep the btree size down to a more manageable size. The free inode
>>
>> btree would also help, but, really, 220M inodes in a 2TB filesystem
>> > is really pushing the boundaries of sanity.....
>> >
>>
>> So the better inodes size in one AG is about 5M,
>
> Not necessarily. But for your storage it's almost certainly going to
> minimise the problem you are seeing.
>
>> is there any documents
>> about these options I can learn more?
>
> http://xfs.org/index.php/XFS_Papers_and_Documentation

Given the apparently huge number of small files would he likely see a
big performance increase if he replaced that 2TB or rust with SSD.

Greg

<Prev in Thread] Current Thread [Next in Thread>