[Top] [All Lists]

Re: observed significant performance improvement using "delaylog" in a r

To: linux-xfs@xxxxxxxxxxx
Subject: Re: observed significant performance improvement using "delaylog" in a real-world application
From: Peter Niemayer <niemayer@xxxxxx>
Date: Wed, 11 Aug 2010 11:45:41 +0200
In-reply-to: <201008111003.36890@xxxxxx>
References: <i3rt4t$46c$1@xxxxxxxxxxxxxxx> <201008111003.36890@xxxxxx>
User-agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv: Gecko/20100713 Thunderbird/3.1.1
On 08/11/2010 10:03 AM, Michael Monnerie wrote:
On Dienstag, 10. August 2010 Peter Niemayer wrote:
17% performance increase for our benchmark scenario

Can you say what kind of data accesses you have?

The application basically writes a large data set to ~ 100,000 files,
which consists of many small messages. Each message consists of a primary key (an integer in the range of 0 to approx. 1,000,000) and
an (almost random) number of data bytes (length in the range from 10 to
~1000 byte).

For each message, the application opens the file that is determined by
the primary key with O_APPEND, write()s the data bytes to the file.
Then it closes the file. (There are usually a few to many messages per
primary key / file).
The application runs 4 threads in parallel to spread the above action
over 4 CPU cores, each thread processes a quarter of the primary keys
(primary_key & 0x03).

This description is, of course, somewhat simplified, but it should
get you an idea on the kind of I/O-operations.


Peter Niemayer

<Prev in Thread] Current Thread [Next in Thread>