09.06.2010 03:18, Dave Chinner wrote:
On Wed, Jun 09, 2010 at 12:34:00AM +0400, Michael Tokarev wrote:
Simple test doing random reads or writes of 4k blocks in a 1Gb
file located on an xfs filesystem, Mb/sec:
read write write
2.6.27 xfs 1.17 3.69 3.80
2.6.32 xfs 1.26 0.52 5.10
2.6.32 ext3 1.19 4.91 5.02
Note the 10 times difference between O_SYNC and O_DIRECT writes
in 2.6.32. This is, well, huge difference, and this is where
the original slowdown comes from, apparently.
Are you running on the raw block device, or on top of LVM/DM/MD to
split up the space on the RAID drive? DM+MD have grown barrier
support since 2.6.27, so it may be that barriers are now being
passed down to the raid hardware on 2.6.32 and they never were on
2.6.27. Can you paste the output of dmesg when the XFS filesystem in
That's why I asked how to tell if barriers are actually hitting the
device in question.
No, this is the only machine where DM/MD is _not_ used. On all other
machines we use MD software raid, this machine comes with an onboard
raid controller that does not work in JBOD mode so I weren't able to
use linux software raid. This is XFS on top of Adaptec RAID card,
Also, as I mentioned in the previous email, remounting with nobarrier
makes no difference whatsoever.
(Another side note here - I discovered that unknown options are
silently ignored in "remount mode" while correctly rejected in
"plain mount" mode, -- it looks like a kernel bug actually, but
it's entirely different issue).
question is mounted on both 2.6.27 and 2.6.32 so we can see if
there is a difference in the use of barriers?
Also, remember that O_DIRECT does not imply O_SYNC. O_DIRECT writes
only write data, while O_SYNC will also write metadata and/or the
I know this. I also found osyncisosync and osyncisdsync mount
options, and when I try to use the latter, kernel tells it's the
default and hence deprecated. I don't need metadata updates, but
it _looks_ like the system is doing such updates (with barriers
or flushes?) anyway even when mounted with -o osyncisdsync it behaves
the same: very slow.
I also experimented with both O_SYNC|O_DIRECT: it is as slow as
without O_DIRECT, i.e. O_SYNC makes whole thing slow regardless
of other options.
I looked at the dmesg outputs, and there's no relevant differences
related to block devices or usage of barriers. For XFS it always
mounts like this:
SGI XFS with ACLs, security attributes, large block/inode numbers, no debug
SGI XFS Quota Management subsystem
XFS mounting filesystem sda6
and for the device in question, it is always like
Adaptec aacraid driver 1.1-5-ms
aacraid 0000:03:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
AAC0: kernel 5.1-0 Feb 1 2006
AAC0: monitor 5.1-0
AAC0: bios 5.1-0
AAC0: serial 267BE0
AAC0: Non-DASD support enabled.
AAC0: 64bit support enabled.
AAC0: 64 Bit DAC enabled
scsi0 : aacraid
scsi 0:0:0:0: Direct-Access Adaptec f0500 V1.0 PQ: 0 ANSI: 2
sd 0:0:0:0: [sda] 286715904 512-byte hardware sectors (146799 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 06 00 10 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and
sda: sda1 sda2 sda3 < sda5 sda6 >
There are tons of other differences, but that is to be expected (like
format of CPU topology printing which is changed between .27 and .32).