On Fri, Aug 20, 2004 at 01:42:53PM -0700, Jason Howard wrote:
> Hello All,
hi there,
> I am wondering if anyone can offer any insight into a problem I have been
> seeing with XFS included in the 2.4.27 kernel. I am seeing a huge difference
> between a raw device (/dev/md0) read and a filesystem read when reading from
> an array that is capable of >500 MBytes/sec. The reads and writes are
> sequential, both on the raw disk device and XFS filesystem (using sequential
> files).
Buffered or direct reads? (could you try both?)
Is this MD RAID5? (if so, could you send xfs_info output for this
filesystem as well?)
> Here is an example of what we normally see on a fairly slow array:
> RAW PERFORMANCE
> Read Average : 212.674 MB/SEC over 1000 frames
> Write Average: 234.769 MB/SEC over 1000 frames
> FS PERFORMANCE
> Read Average : 148.125 MB/SEC over 1000 frames
> Write Average: 286.031 MB/SEC over 1000 frames
>
> Now here is the problem I am seeing on a fast array:
> RAW PERFORMANCE
> Read Average : 594.965 MB/SEC over 1000 frames
> Write Average: 350.604 MB/SEC over 1000 frames
> FS PERFORMANCE
> Read Average : 67.280 MB/SEC over 1000 frames
> Write Average: 305.363 MB/SEC over 1000 frames
>
> I have seen similar results on two separate arrays, one a software raid of
> dual U320 arrays, the other is a software raid of dual 2G fibre w/ 16 disks.
> I don't suspect it is a problem with the physical interfaces, as I am seeing
> the same issue on multiple technologies. Also, I have pretty much ruled out
> a problem with Linux software raid as we are getting expected performance
> from the raw /dev/md0 device. That is not to say that it couldn't very well
> be a nasty interaction between software raid and XFS.
If it is a RAID5 device, make sure your filesystem sector and
block sizes are the same (both are mkfs.xfs options).
Otherwise, not sure... could be a readahead oddity if this is
buffered IO. As another data point, what do the ext2 numbers
look like? (this will point to an XFS-specific problem, or a
more generic - eg. readahead - type of problem).
cheers.
--
Nathan
|