On Mon, Jan 31, 2005 at 05:32:31PM -0800, Bryan Henderson wrote:
> Thanks for the numbers, though there are enough variables here that it's
> hard to make any hard conclusions.
I agree, I just throught the data was interesting.
> When I've seen these comparisons in the past, it turned out to be one of
> two things:
>
> 1) The system with the smaller I/Os (I/O = unit seen by the device) had
> more CPU time per megabyte in the code path to start I/O, so that it
> started less I/O. The small I/Os are a consequence of the lower
> throughput, not a cause. You can often rule this out just by looking at
> CPU utilization.
Actually in this case, I was running a single threaded test on an
8-way machine. So, the flushing should have been able to happen on a
different (mostly dedicated) CPU from the application that was writing.
In this case, the differences in filesystems was probably the cause.
What might be a better test is to take JFS, and "hobble" it so that it
won't submit multi-page BIOs (i.e. kill it's writepages aop)
> 2) The system with the smaller I/Os had a window tuning problem in which
> it was waiting for previous I/O to complete before starting more, with
> queues not full, and thus starting less I/O. Some devices, with good
> intentions, suck the Linux queue dry, one tiny I/O at a time, and then
> perform miserably processing those tiny I/Os. Properly tuned, the device
> would buffer fewer I/Os and thus let the queues build inside Linux and
> thus cause Linux to send larger I/Os.
>
> People have done ugly queue plugging algorithms to try to defeat this
> queue sucking by withholding I/O from a device willing to take it. Others
> defeat it by withholding I/O from a willing Linux block layer, instead
> saving up I/O and submitting it in large bios.
Sure, it seems to work well in practice. We've played a little bit
with adjusting the plugging/unplugging thresholds, and in general,
playing with them either made things worse or simply had no effect.
Sonny
--
IBM LTC Kernel Perf.
|