On 6/12/2013 3:26 AM, Roger Oberholtzer wrote:
...
> I have an application that is streaming data to an XFS disk at a
> sustained 25 MB/sec. This is well below what the hardware supports. The
> application does fopen/fwrite/fclose (no active flushing or syncing).
Buffered IO.
> I see that as my application writes data (the only process writing the
> only open file on the disk), the system cache grows and grows. Here is
> the unusual part: periodically, writes take some number of seconds to
> complete, rather than the typical <50 msecs). The increased time seems
> to correspond to the increasing size of the page cache.
Standard Linux buffered IO behavior. Note this is not XFS specific.
> If I do:
>
> echo 1 > /proc/sys/vm/drop_caches
Dumps the page cache forcing your buffered writes to disk.
> while the application is runnung, then the writes do not occasionally
> take longer. Until the cache grows again, and I do the echo again.
Which seems a bit laborious.
> I am sure I must be misinterpreting what I see.
Nope. The Linux virtual memory system has behaved this way for quite
some time. You can teak how long IOs stay in cache. See dirty_* at
https://www.kernel.org/doc/Documentation/sysctl/vm.txt
Given the streaming nature you describe, have you looked at possibly
using O_DIRECT?
--
Stan
|