ååï XFS direct IO problem

["YeYin" <eyniy@xxxxxx>]

 Dave,

Thank you for your explanation. I got the reason, and I write some code to  simulate the MySQL.It will reproduce the progress:â

open file without direct flag

read file   //cause kernel readahead 4 pages, and inode->i_mapping->nrpages > 0

close file

open file with direct flag

lseek 4*4096 // skip 4 readahead pages

read  file //cause xfs_flushinval_pages to do nothing

...

I'd like to ask XFS how to resovle this problem?

Attach codeïâ

/* gcc -o test_read test_read.c

 * dd if=/dev/zero of=/data1/fo.dat bs=4096 count=10

 * ./test_read /data1/fo.dat 2 direct

 * */

#define _GNU_SOURCE

#include <stdio.h>

#include <unistd.h>

#include <sys/types.h>

#include <sys/stat.h>

#include <fcntl.h>

#include <errno.h>

#include <string.h>

#include <stdlib.h>

#define BUFSIZE 4096

int read_count = 2;

int main(int argc, char *argv[]){

if(argc < 3){

fprintf(stderr, "usage: %s <file> <count> [buffer|direct]\n", argv[0]);

exit(1);

}

char *buf = memalign(BUFSIZE - 1, BUFSIZE);

char *file = argv[1];

read_count = atoi(argv[2]);

int ret = 0,sum = 0, i = 0, fd = -1;

if(argc == 4 && strncmp(argv[3], "direct",6) == 0){

//fd = open(file, O_RDONLY|O_DIRECT);

fd = open(file, O_RDONLY);

if(fd < 0){

fprintf(stderr, "open read only file failed\n");

exit(1);

}

ret = read(fd, buf, BUFSIZE);

if(ret < 0){

fprintf(stderr, "buffer read error\n");

}

close(fd);

fd = open(file, O_RDWR);

if(fd < 0){

fprintf(stderr, "open read only file failed\n");

exit(1);

}

if (fcntl(fd, F_SETFL, O_RDONLY|O_DIRECT) == -1) {

fprintf(stderr, "set direct error\n");

exit(1);

}

}else{

fd = open(file, O_RDONLY);

if(fd < 0){

fprintf(stderr, "open buf file failed\n");

exit(1);

}

}

while(i++ < read_count){

//memset(buf, 0, BUFSIZE);

if(buf == NULL){

fprintf(stderr, "memory allocate failed\n");

exit(1);

}

if(lseek(fd, 4*4096, SEEK_SET) < 0){

fprintf(stderr, "seek error!\n");

break;

}

ret = read(fd, buf, BUFSIZE);

if(ret > 0){

sum += ret;

}else if(ret == 0){

printf("read end\n");

break;

}

else{

printf("error:%d\n", errno);

break;

}

sleep(1);

}

printf("read sum: %d\n", sum);

close(fd);

free(buf);

return 0;

}â

------------------ ååéä ------------------

åää: "Dave Chinner";<david@xxxxxxxxxxxxx>;

åéæé: 2015å4æ8æ(ææä) äå12:49

æää: "YeYin"<eyniy@xxxxxx>;

æé: "xfs"<xfs@xxxxxxxxxxx>;

äé: Re: XFS direct IO problem

On Wed, Apr 08, 2015 at 12:21:45PM +0800, YeYin wrote:
> Hi, About 2 months ago, I asked one problem in XFS, see
> here(http://oss.sgi.com/archives/xfs/2015-02/msg00197.html).
>
>
> After that, I use direct IO in MySQL, see
> here(https://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_flush_method).â
>
>
> However, I found that MySQL performance is still poor sometimes. I
> use some tools(https://github.com/brendangregg/perf-toolsâ) to
> trace the kernel, I found some problems:

<snip>

> This will cause bad performance, even direct IO. I still don't
> understand why not truncate_inode_page called?

Because the cached page must be outside the range of the direct IO
that is in progress - direct IO only tries to flush pages over the
range it is doing the IO over.

> Every time, after I run this: echo 1 > /proc/sys/vm/drop_caches
>
> Immediately enhance performance.

Because that flushes whatever page is in the cache. Can you identify
what offset that cached page is at? Tracing the xfs events will tell
you what pages that operation invalidates on each inode, and knowing
the offset may tell us why that page is not getting flushed.

Alternatively, write a simple C program that deomnstrates the same
problem so we can reproduce it easily, fix the problem and turn it
into a regression test....

Cheers,

Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx