[Top] [All Lists]

Re: How to track down abysmal performance ata - raid1 - crypto - vg/lv -

To: Michael Monnerie <michael.monnerie@xxxxxxxxxxxxxxxxxxx>, Christoph Hellwig <hch@xxxxxxxxxxxxx>
Subject: Re: How to track down abysmal performance ata - raid1 - crypto - vg/lv - xfs
From: Dominik Brodowski <linux@xxxxxxxxxxxxxxxxxxxx>
Date: Wed, 4 Aug 2010 12:25:26 +0200
Cc: linux-raid@xxxxxxxxxxxxxxx, xfs@xxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dm-devel@xxxxxxxxxx
In-reply-to: <20100804092122.GA2998@xxxxxxxxxxxxx> <201008041116.09822@xxxxxx>
Mail-followup-to: Dominik Brodowski <linux@xxxxxxxxxxxxxxxxxxxx>, Michael Monnerie <michael.monnerie@xxxxxxxxxxxxxxxxxxx>, Christoph Hellwig <hch@xxxxxxxxxxxxx>, linux-raid@xxxxxxxxxxxxxxx, xfs@xxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dm-devel@xxxxxxxxxx
References: <20100804073546.GA7494@xxxxxxxxxxxxxxxxxxxxxxxxxx> <20100804085039.GA11671@xxxxxxxxxxxxx> <20100804091317.GA27779@xxxxxxxxxxxxxxxxxx> <20100804092122.GA2998@xxxxxxxxxxxxx> <20100804073546.GA7494@xxxxxxxxxxxxxxxxxxxxxxxxxx> <201008041116.09822@xxxxxx>
User-agent: Mutt/1.5.18 (2008-05-17)

many thanks for your feedback. It seems the crypto step is the culprit:

Reading 1.1 GB with dd, iflag=direct, bs=8k:

/dev/sd*                35.3 MB/s       ( 90 %)
/dev/md*                39.1 MB/s       (100 %)
/dev/mapper/md*_crypt    3.9 MB/s       ( 10 %)
/dev/mapper/vg1-*        3.9 MB/s       ( 10 %)

The "good" news: it also happens on my notebook, even though it has a
different setup (no raid, disk -> lv/vg -> crypt). On my notebook, I'm
more than happy to test out different kernel versions, patches etc.

/dev/sd*                17.7 MB/s       (100 %)
/dev/mapper/vg1-*       16.2 MB/s       ( 92 %)
/dev/mapper/*_crypt      3.1 MB/s       ( 18 %)

On a different system, a friend of mine reported (with 2.6.33):

/dev/sd*                51.9 MB/s       (100 %)
dm-crypt                32.9 MB/s       ( 64 %)

This shows that the speed drop when using dmcrypt is not always a factor of
5 to 10... Btw, it occurs both with aes-lrw-benbi and aes-cbc-essiv:sha256 ,
and (on my notebook) the CPU is mostly idling or waiting. 


PS: Bonnie output:

Writing intelligently...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.03d       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
ABCABCABCABCABC 16G           60186   4 24796   4           53386   5 281.1   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  3176  16 +++++ +++  4641  28  5501  20 +++++ +++  2286   9

<Prev in Thread] Current Thread [Next in Thread>