observed significant performance improvement using "delaylog" in a real-world application

[Peter Niemayer]

On 08/11/2010 10:03 AM, Michael Monnerie wrote:
> On Dienstag, 10. August 2010 Peter Niemayer wrote:
>> 17% performance increase for our benchmark scenario
>
> Can you say what kind of data accesses you have?

The application basically writes a large data set to ~ 100,000 files,
which consists of many small messages. Each message consists of a 
primary key (an integer in the range of 0 to approx. 1,000,000) and
an (almost random) number of data bytes (length in the range from 10 to
~1000 byte).

For each message, the application opens the file that is determined by
the primary key with O_APPEND, write()s the data bytes to the file.
Then it closes the file. (There are usually a few to many messages per
primary key / file).
The application runs 4 threads in parallel to spread the above action
over 4 CPU cores, each thread processes a quarter of the primary keys
(primary_key & 0x03).

This description is, of course, somewhat simplified, but it should
get you an idea on the kind of I/O-operations.

Regards,

Peter Niemayer