xfs
[Top] [All Lists]

Re: Questions about XFS

To: xfs@xxxxxxxxxxx
Subject: Re: Questions about XFS
From: Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx>
Date: Wed, 12 Jun 2013 07:12:34 -0500
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <1371025611.7096.10.camel@acme>
References: <loom.20130611T112155-970@xxxxxxxxxxxxxx> <51B72D3D.5010206@xxxxxxxxxx> <CAO9HMNGjdikgX+_434aGVJ2NAJ0hxDNLo+Vsa46GH3psXr4sKQ@xxxxxxxxxxxxxx> <1371025611.7096.10.camel@acme>
Reply-to: stan@xxxxxxxxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6
On 6/12/2013 3:26 AM, Roger Oberholtzer wrote:
...
> I have an application that is streaming data to an XFS disk at a
> sustained 25 MB/sec. This is well below what the hardware supports. The
> application does fopen/fwrite/fclose (no active flushing or syncing).

Buffered IO.

> I see that as my application writes data (the only process writing the
> only open file on the disk), the system cache grows and grows. Here is
> the unusual part: periodically, writes take some number of seconds to
> complete, rather than the typical <50 msecs). The increased time seems
> to correspond to the increasing size of the page cache.

Standard Linux buffered IO behavior.  Note this is not XFS specific.

> If I do:
> 
> echo 1 > /proc/sys/vm/drop_caches

Dumps the page cache forcing your buffered writes to disk.

> while the application is runnung, then the writes do not occasionally
> take longer. Until the cache grows again, and I do the echo again.

Which seems a bit laborious.

> I am sure I must be misinterpreting what I see.

Nope.  The Linux virtual memory system has behaved this way for quite
some time.  You can teak how long IOs stay in cache.  See dirty_* at
https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Given the streaming nature you describe, have you looked at possibly
using O_DIRECT?

-- 
Stan

<Prev in Thread] Current Thread [Next in Thread>