xfs
[Top] [All Lists]

Re: XFS fragmentation on file append

To: Keyur Govande <keyurgovande@xxxxxxxxx>
Subject: Re: XFS fragmentation on file append
From: Stefan Ring <stefanrin@xxxxxxxxx>
Date: Thu, 15 May 2014 22:25:22 +0200
Cc: linux-fsdevel@xxxxxxxxxxxxxxx, Linux fs XFS <xfs@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MOMvTgtdtBdOprehFfxzknCjwcq1pisGTAiAPKmMqko=; b=xmHuOQYH76qHB5DKwb6FEJrtwVmNK1fYHYbnLtuUNIUwCONMb4ZZ7rqepQyZVJS43Y 9HGJaYw8xxz4TbNQOK1tUh2uhJ3B29eIEDH6SMFK7s5SX+K9pFJkYu/TEJcgzqvdaa4C 4QJIW8xiR5YajZJnFwMV2R9DemtJhRF6grB5367wJEK07qOSBLmN8WHzooY84cwMncDI jpoeJJMldISofDMLW+FPFpobOJkl9UnRsyX1peWA3meWZ5RAzxqm2smbEPeiMT9nYSG2 RU8dBpSpNGXAHbOmGJBhinKTmwd90OfRJWEBaUKk6zpuDTjThGnoBPONqgdI6pC9rb8p YIHw==
In-reply-to: <CAJhmKHmxYbzX=8u7=HcV9xgSq4Xs8HMFcdLhnL01CwAQHpjEkw@xxxxxxxxxxxxxx>
References: <CAJhmKHmwvCBx=arJ6m2ZhuQVq=Jj-XbBGvEPjTPY5a1QtWRTCQ@xxxxxxxxxxxxxx> <20140408015012.GB22917@dastard> <CAJhmKHnap0vu8m4AXBxJr9L0RKoquFrm3NjKQdDBJwpRTYmfpw@xxxxxxxxxxxxxx> <20140408053117.GJ27017@dastard> <CAJhmKHk5fWVaDTEfL_oESULspavk83eBEsyw7=qEp8bC7f2S2g@xxxxxxxxxxxxxx> <CAAxjCEzdYRug8CUjhmB7AHJCNAJs4ERRTLXGAHXqi1sw-vOriw@xxxxxxxxxxxxxx> <CAJhmKHmxYbzX=8u7=HcV9xgSq4Xs8HMFcdLhnL01CwAQHpjEkw@xxxxxxxxxxxxxx>
On Thu, Apr 24, 2014 at 11:49 PM, Keyur Govande <keyurgovande@xxxxxxxxx> wrote:
> On Thu, Apr 24, 2014 at 2:54 AM, Stefan Ring <stefanrin@xxxxxxxxx> wrote:
>> I've become interested in this topic, as I'm also running MySQL with
>> O_DIRECT and innodb_file_per_table. Out of curiosity, I immediately
>> ran xfs_bmap on a moderately sized table space (34GB). It listed
>> around 30000 fragments, on average one for every MB.
>>
>> I want to report what happened then: A flurry of activity started on
>> both disks (root/swap lives on one of them, the data volume containing
>> the MySQL files on another) and lasted for about two minutes.
>> Afterwards, all memory previously allocated to the file cache has
>> become free, and also everything XFS seems to keep buffered internally
>> (I think it's called SReclaimable) was released. Swap usage increased
>> only slightly. dmesg was silent during that time.
>>
>> This is a 2.6.32-358.2.1.el6.x86_64 kernel with xfsprogs 3.1.1 (CentOS
>> 6.4). The machine has 64GB of RAM (2 NUMA nodes) and 24 (virtual)
>> cores. Is this known behavior of xfs_bmap?
>
> Interesting...it looks like your box flushed all of the OS buffer
> cache. I am unable to reproduce this behavior on my test box with the
> 3.10.37 kernel. I also tried with 2.6.32-358.18.1.el6.x86_64 and
> didn't hit the issue, but obviously our access patterns differ wildly.

I tried it again, logging a few files in /proc periodically:
https://dl.dropboxusercontent.com/u/5338701/dev/xfs/memdump.tar.xz

Inside the archive, "memdump" is the simplistic script used to create
the other files. A few seconds in, I invoked xfs_bmap on the same file
again (this time weighing in at 81GB), and it spit out 36000
fragments. It took only a few seconds to completely drain 40 GB of
buffer memory.

cciss/c0d0 is the device where the XFS filesystem lives, while sda
contains root and swap.

If somebody could gain some insight from this, I'd be happy.

<Prev in Thread] Current Thread [Next in Thread>
  • Re: XFS fragmentation on file append, Stefan Ring <=