XFS buffer IO performance is very poor

[yy]

In functionxfs_file_aio_read, will requestXFS_IOLOCK_SHARED lock for both direct IO and buffered IO:
STATIC ssize_t
xfs_file_aio_read(
struct kiocb *iocb,
const struct iovec *iovp,
unsigned long nr_segs,
loff_t pos)
{
...
xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);


https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/fs/xfs/linux-2.6/xfs_file.c?at=master#cl-281


so write will prevent read in XFS.


However, in function generic_file_aio_read for ext3, will not lockinode-i_mutex, so write will not prevent read in ext3.


I think this maybe the reason of poor performance for XFS. I do not know if this is a bug, or design flaws of XFS.




Best regards,
yy


原始邮件
发件人:yyyy at xspring.net
收件人:xfsxfs at oss.sgi.com; Eric Sandeensandeen at sandeen.net; bfoster at redhat.com
发送时间:2015年2月12日(周四) 13:30
主题:Re: XFS buffer IO performance is very poor


Brian and Eric,


Thanks very much for your replay.


I changed partition start position with 256K, but the performance is still poor, no change.
# fdisk -ul /dev/sdb          
 Device Boot   Start     End   Blocks  Id System
/dev/sdb1       512 2929356359 1464677924  83 Linux


I checked the XFS’s code, I agree thatXFS_IOLOCK_EXCL lock maybe the reason:
https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/fs/xfs/linux-2.6/xfs_file.c?at=master#cl-714
STATIC ssize_t
xfs_file_buffered_aio_write(
struct kiocb *iocb,
const struct iovec *iovp,
unsigned long nr_segs,
loff_t pos,
size_t ocount)
{
struct file *file = iocb-ki_filp;
struct address_space *mapping = file-f_mapping;
struct inode *inode = mapping-host;
struct xfs_inode *ip = XFS_I(inode);
ssize_t ret;
int enospc = 0;
int iolock = XFS_IOLOCK_EXCL;
size_t count = ocount;


xfs_rw_ilock(ip, iolock);


ret = xfs_file_aio_write_checks(file, pos, count, iolock);
if (ret)




However，I found that EXT3 also have mutex when with buffered IO:
https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/mm/filemap.c?at=master#cl-2642
ssize_t generic_file_aio_write(struct kiocb *iocb, const struct iovec *iov,
unsigned long nr_segs, loff_t pos)
{
struct file *file = iocb-ki_filp;
struct inode *inode = file-f_mapping-host;
ssize_t ret;


BUG_ON(iocb-ki_pos != pos);


sb_start_write(inode-i_sb);
mutex_lock(inode-i_mutex);
ret = __generic_file_aio_write(iocb, iov, nr_segs, iocb-ki_pos);
mutex_unlock(inode-i_mutex);


I still don’t understand why ext3 does not have this problem with buffered IO.


Best regards,
yy


原始邮件
发件人:Eric Sandeensandeen at sandeen.net
收件人:yyyy at xspring.net; xfsxfs at oss.sgi.com
发送时间:2015年2月12日(周四) 00:08
主题:Re: XFS buffer IO performance is very poor


On 2/11/15 1:39 AM, yy wrote: snip (In addition to Brian's questions):  XFS format parametes:   #mkfs.xfs -d su=256k,sw=5 /dev/sdb1   #cat /proc/mounts   /dev/sdb1 /data1 xfs rw,noatime,attr2,delaylog,nobarrier,logbsize=256k,sunit=512,swidth=2560,noquota 0 0   #fdisk -ul  Device Boot Start End Blocks Id System  /dev/sdb1 128 2929356359 1464678116 83 Linux so 128*512 = 64k; your partition doesn't start on a 256k stripe unit boundary, right? Shouldn't it do so? -Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20150212/4d1b6212/attachment.html>