[Top] [All Lists]

Re: XFS peculiar behavior

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: XFS peculiar behavior
From: Yannis Klonatos <klonatos@xxxxxxxxxxxx>
Date: Thu, 24 Jun 2010 17:11:29 +0300
Cc: xfs@xxxxxxxxxxx, sandeen@xxxxxxxxxxx, andi@xxxxxxxxxxxxxx
In-reply-to: <20100623231700.GP6590@dastard>
References: <4C21B9AF.9010307@xxxxxxxxxxxx> <20100623231700.GP6590@dastard>
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: Gecko/20100512 Thunderbird/3.0.5
Hello again,

First of all, thank you all for your quick replies. I attach all the information you requested in your responses.

1) The output of xfs_info is the following:

meta-data=/dev/sdf             isize=256    agcount=32, agsize=45776328 blks
               =                         sectsz=512   attr=0
data = bsize=4096 blocks=1464842496, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1
naming    =version 2            bsize=4096
log          =internal               bsize=4096   blocks=32768, version=1
= ectsz=512 sunit=0 blks, lazy-count=0
realtime  =none                   extsz=4096   blocks=0, rtextents=0

2) The output of xfs_bmap in the lineitem.MYI table of the TPC-H workload is at one run:

EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..6344271]: 11352529416..11358873687 31 (72..6344343) 6344272 1: [6344272..10901343]: 1464842608..1469399679 4 (112..4557183) 4557072 2: [10901344..18439199]: 1831053200..1838591055 5 (80..7537935) 7537856 3: [18439200..25311519]: 2197263840..2204136159 6 (96..6872415) 6872320 4: [25311520..26660095]: 2563474464..2564823039 7 (96..1348671) 1348576

Given that all disk blocks are in units of 512-byte blocks, if I interpret the output correctly the first file is at block 1465352792 = 698.4GByte offset and the last block is at 5421.1GByte offset, meaning that this specific table is split over a 4,7TByte distance.

However, in another run (with a clean file system again)

EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..26660095]: 11352529416..11379189511 31 (72..26660167) 26660096

3) For the copy, as i mentioned in my previous mail, i copied the database over nfs using the cp -R linux program. Thus, i believe all the files are copied sequentially, the one after the other, with no other concurrent write operations running at the background. The file-system was pristine before the cp with no files, and just the mount directory was created (all the other necessary files and directories are created from the cp program).

4) The version of xfsprogs is 2.9.4 (acquired with xfs_info -v) and the version of the kernel is 2.6.18-164.11.1.el5.

If you require any further information let me know. Let me state that i can also provide you with the complete
data-set if you feel it necessary trying to reproduce the issue.

Yannis Klonatos
Hi all!

         I have come across the following peculiar behavior in XFS
and i would appreciate any information anyone
could provide.
         In our lab we have a system that has twelve 500GByte hard
disks (total capacity 6TByte), connected to an
Areca (ARC-1680D-IX-12) SAS storage controller. The disks are
configured as a RAID-0 device. Then I create
a clean XFS filesystem on top of the raid volume, using the whole
capacity. We use this test-setup to measure
performance improvement for a TPC-H experiment. We copy the database
over the clean XFS filesystem using the
cp utility. The database used in our experiments is 56GBytes in size
(data + indices).
         The problem is that i have noticed that XFS may - not all
times - split a table over a large disk distance. For
example in one run i have noticed that a file of 13GByte is split
over a 4,7TByte distance (I calculate this distance
by subtracting the final block used for the file with the first one.
The two disk blocks values are acquired using the
FIBMAP ioctl).
         Is there some reasoning behind this (peculiar) behavior? I
would expect that since the underlying storage is so
large, and the dataset is so small, XFS would try to minimize disk
seeks and thus place the file sequentially in disk.
Furthermore, I understand that there may be some blocks left unused
by XFS between subsequent file blocks used
in order to handle any write appends that may come afterward. But i
wouldn't expect such a large splitting of a single
         Any help?

<Prev in Thread] Current Thread [Next in Thread>