Re: XFS buffer IO performance is very poor

["yy" <yy@xxxxxxxxxxx>]

In function xfs_file_aio_read, will request XFS_IOLOCK_SHARED lock for both direct IO and buffered IO:

STATIC ssize_t

xfs_file_aio_read(

struct kiocb *iocb,

const struct iovec *iovp,

unsigned long nr_segs,

loff_t pos)

{

...

xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);

https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/fs/xfs/linux-2.6/xfs_file.c?at=master#cl-281

so write will prevent read in XFS.

However, in function generic_file_aio_read for ext3, will not lock inode->i_mutex, so write will not prevent read in ext3.

I think this maybe the reason of poor performance for XFS. I do not know if this is a bug, or design flaws of XFS.

Best regards,

yy

 ååéä 

åää: yy<yy@xxxxxxxxxxx>

æää: xfs<xfs@xxxxxxxxxxx>; Eric Sandeen<sandeen@xxxxxxxxxxx>; bfoster@xxxxxxxxxx

åéæé: 2015å2æ12æ(åå)â13:30

äé: Re: XFS buffer IO performance is very poor

Brian and Eric,

Thanks very much for your replay.

I changed partition start position with 256K, but the performance is still poor, no change.

# fdisk -ul /dev/sdb                    

   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1             512  2929356359  1464677924   83  Linux

I checked the XFSâs code, I agree that XFS_IOLOCK_EXCL lock maybe the reason:

https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/fs/xfs/linux-2.6/xfs_file.c?at=master#cl-714

STATIC ssize_t

xfs_file_buffered_aio_write(

struct kiocb *iocb,

const struct iovec *iovp,

unsigned long nr_segs,

loff_t pos,

size_t ocount)

{

struct file *file = iocb->ki_filp;

struct address_space *mapping = file->f_mapping;

struct inode *inode = mapping->host;

struct xfs_inode *ip = XFS_I(inode);

ssize_t ret;

int enospc = 0;

int iolock = XFS_IOLOCK_EXCL;

size_t count = ocount;

xfs_rw_ilock(ip, iolock);

ret = xfs_file_aio_write_checks(file, &pos, &count, &iolock);

if (ret)

HoweverïI found that EXT3 also have mutex when with buffered IO:

https://bitbucket.org/hustcat/kernel-2.6.32/src/786d720807052737bb17bc44da9da20554400039/mm/filemap.c?at=master#cl-2642

ssize_t generic_file_aio_write(struct kiocb *iocb, const struct iovec *iov,

unsigned long nr_segs, loff_t pos)

{

struct file *file = iocb->ki_filp;

struct inode *inode = file->f_mapping->host;

ssize_t ret;

BUG_ON(iocb->ki_pos != pos);

sb_start_write(inode->i_sb);

mutex_lock(&inode->i_mutex);

ret = __generic_file_aio_write(iocb, iov, nr_segs, &iocb->ki_pos);

mutex_unlock(&inode->i_mutex);

I still donât understand why ext3 does not have this problem with buffered IO. 

Best regards,

yy

 ååéä 

åää: Eric Sandeen<sandeen@xxxxxxxxxxx>

æää: yy<yy@xxxxxxxxxxx>; xfs<xfs@xxxxxxxxxxx>

åéæé: 2015å2æ12æ(åå)â00:08

äé: Re: XFS buffer IO performance is very poor

On 2/11/15 1:39 AM, yy wrote:

<snip>

(In addition to Brian's questions):

> XFS format parametes:
> 
> #mkfs.xfs -d su=256k,sw=5 /dev/sdb1
> 
> #cat /proc/mounts
> 
> /dev/sdb1 /data1 xfs rw,noatime,attr2,delaylog,nobarrier,logbsize=256k,sunit=512,swidth=2560,noquota 0 0
> 
> #fdisk -ul
> Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1             128  2929356359  1464678116   83  Linux

so 128*512 = 64k; your partition doesn't start on a 256k stripe unit boundary,
right?  Shouldn't it do so?

-Eric