xfs
[Top] [All Lists]

Re: [PATCH 09/10] xfstest: add defragmentation stress tests for ext4

To: Rich Johnston <rjohnston@xxxxxxx>
Subject: Re: [PATCH 09/10] xfstest: add defragmentation stress tests for ext4
From: Dmitry Monakhov <dmonakhov@xxxxxxxxxx>
Date: Sat, 02 Mar 2013 05:30:43 +0400
Cc: xfs@xxxxxxxxxxx, linux-fsdevel@xxxxxxxxxxxxxxx, linux-ext4@xxxxxxxxxxxxxxx, dchinner@xxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:from:to:cc:subject:in-reply-to:references :user-agent:date:message-id:mime-version:content-type; bh=rAnlwywUBdaidJZfgdc380R8YIPKVeddCqbqzXy7LpU=; b=iQ6UCxH4q3YUk3I2kvlOv6lM+mvRtxr5v136dm2LcA571dHIKCJj4zOLda0VxlMvKd wqtzgg8gUnPzLB0y7Revo+6WDqIgrD3F4ITw3R1+MezQXhcS5m/2n7gn0ARAazJ4l4r4 mrSqSiwLc1VM42o/nY7QU7tG5QtsIXSYiWPoVujqEtZ8ZtEU9cUNev+KA9YdbxLYyqko gP5Gh/fap7xy98hGBNoIyOeHrWD84pGuawDbpd9yGnb+W2RXJ8Pn9MofiTKEFupYaHG0 ERHK//xGgNVD5U6D9E1dJRcusz2iM9WyqfjiMrMX/ZxwTT8GyMrucaZ5wSOuXjY7tBdT C7zw==
In-reply-to: <51310046.5090306@xxxxxxx>
References: <1361356935-29153-1-git-send-email-dmonakhov@xxxxxxxxxx> <1361356935-29153-10-git-send-email-dmonakhov@xxxxxxxxxx> <51310046.5090306@xxxxxxx>
Sender: Dmitry Monakhov <rjevskiy@xxxxxxxxx>
User-agent: Notmuch/0.6.1 (http://notmuchmail.org) Emacs/23.3.1 (x86_64-redhat-linux-gnu)
On Fri, 1 Mar 2013 13:23:50 -0600, Rich Johnston <rjohnston@xxxxxxx> wrote:
> On 02/20/2013 04:42 AM, Dmitry Monakhov wrote:
> > Perform various regression tests for ext4defrag  subsystem
> >
> >   299'th Test1: Defragment file while other task does direct AIO
> >   300'th Test2: Perform defragmentation on file under buffered AIO
> >     while third task does direct AIO to donor file
> >   301'th Test3: Two defrag tasks use common donor file.
> >   302'th Test4: Stress defragmentation. Several threads pefrorm
> typo                                                    perform
> 
> >     fragmentation at random position use inplace=1 will
> >     allocate and free blocks inside defrag event improve
> >     load pressure.
> >
> > This tests are considered dengerous because 300'th and 301'th are known
> typo                        dangerous
> 
> > to trigger OOPS on recent kernels 
> > see:https://gist.github.com/dmonakhov/4770294
> >
> > Signed-off-by: Dmitry Monakhov <dmonakhov@xxxxxxxxxx>
> > ---
> >   299           |  126 +++++++++++++++++++++++++++++++++++++++++++++++
> >   299.out       |    4 ++
> >   300           |  143 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >   300.out       |    4 ++
> >   301           |  153 
> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >   301.out       |    4 ++
> >   302           |  130 ++++++++++++++++++++++++++++++++++++++++++++++++
> >   302.out       |    4 ++
> >   common.defrag |    4 +-
> >   group         |    6 ++-
> >   10 files changed, 575 insertions(+), 3 deletions(-)
> >   create mode 100755 299
> >   create mode 100644 299.out
> >   create mode 100755 300
> >   create mode 100644 300.out
> >   create mode 100755 301
> >   create mode 100644 301.out
> >   create mode 100755 302
> >   create mode 100644 302.out
> >
> > diff --git a/299 b/299
> > new file mode 100755
> > index 0000000..4a7a98c
> > --- /dev/null
> > +++ b/299
> > @@ -0,0 +1,126 @@
> > +#! /bin/bash
> > +# FSQA Test No. 299
> > +#
> > +# Ext4 defragmentation stress test
> > +# Defragment file while other task does direct io
> > +#-----------------------------------------------------------------------
> > +# (c) 2013 Dmitry Monakhov
> > +#
> > +# This program is free software; you can redistribute it and/or
> > +# modify it under the terms of the GNU General Public License as
> > +# published by the Free Software Foundation.
> > +#
> > +# This program is distributed in the hope that it would be useful,
> > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +# GNU General Public License for more details.
> > +#
> > +# You should have received a copy of the GNU General Public License
> > +# along with this program; if not, write the Free Software Foundation,
> > +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> > +#
> > +#-----------------------------------------------------------------------
> > +#
> > +# creator
> > +owner=dmonakhov@xxxxxxxxxx
> > +
> > +seq=`basename $0`
> > +echo "QA output created by $seq"
> > +
> > +here=`pwd`
> > +tmp=/tmp/$$
> > +status=1   # failure is the default!
> > +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15
> > +
> > +# get standard environment, filters and checks
> > +. ./common.rc
> > +. ./common.filter
> > +. ./common.defrag
> > +
> > +# real QA test starts here
> > +_supported_os Linux
> > +_supported_fs ext4
> > +_need_to_be_root
> > +_require_scratch
> > +_require_defrag
> > +
> I think you meant to set NUM_JOBS here like you did in [Patch 08/10], 
> something like:
> 
> +NUM_JOBS=$((4*LOAD_FACTOR))
Indeed. I've forget to update patch.
> 
> otherwise you get
> 
> ###    [not run] /usr/local/bin/fio too old
> 
> > +BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
> > +# We need space for 2 files (test file, and donor one)
> > +# reserve 30% in order to avoid ENOSPC
> > +FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1))))
> > +
> > +cat >$tmp-$seq.fio <<EOF
> > +# Common e4defrag regression tests
> > +[global]
> > +ioengine=ioe_e4defrag
> > +iodepth=1
> > +directory=${SCRATCH_MNT}
> > +filesize=${FILE_SIZE}
> > +size=999G
> > +buffered=0
> > +fadvise_hint=0
> > +group_reporting
> > +
> > +#################################
> > +# Test1
> > +# Defragment file while other task does direct io
> > +
> > +# Continious sequential defrag activity
> > +[defrag-4k]
> > +ioengine=e4defrag
> > +iodepth=1
> > +bs=128k
> > +donorname=test1.def
> > +filename=test1
> > +inplace=0
> > +rw=write
> > +numjobs=${NUM_JOBS}
> > +runtime=30*${TIME_FACTOR}
> > +time_based
> > +
> > +# Verifier
> > +[aio-dio-verifier]
> > +ioengine=libaio
> > +iodepth=128*${LOAD_FACTOR}
> > +numjobs=1
> > +verify=crc32c-intel
> > +verify_fatal=1
> > +verify_dump=1
> > +verify_backlog=1024
> > +verify_async=1
> > +verifysort=1
> > +direct=1
> > +bs=64k
> > +rw=randwrite
> > +filename=test1
> > +runtime=30*${TIME_FACTOR}
> > +time_based
> > +EOF
> > +
> > +_workout()
> > +{
> > +   echo ""
> > +   echo " Start defragment activity "
> git does not like trailing whitespace   ^
> (same for the other 3 tests)
> > +   echo ""
> > +   cat $tmp-$seq.fio >>  $seq.full
> > +   run_check $FIO_PROG $tmp-$seq.fio
> > +}
> > +
> > +_require_fio $tmp-$seq.fio
> > +
> > +_scratch_mkfs  >> $seq.full 2>&1
> > +_scratch_mount
> > +
> > +if ! _workout; then
> > +   umount $SCRATCH_DEV 2>/dev/null
> > +   exit
> > +fi
> > +
> > +if ! _scratch_unmount; then
> > +   echo "failed to umount"
> > +   status=1
> > +   exit
> > +fi
> > +_check_scratch_fs
> > +status=$?
> > +exit diff --git a/299.out b/299.out new file mode 100644 index 
> > 0000000..b215a3f
> > diff --git a/299.out b/299.out
> > new file mode 100644
> > index 0000000..c2b0b4c
> > --- /dev/null
> > +++ b/299.out
> > @@ -0,0 +1,4 @@
> > +QA output created by 299
> > +
> > + Start defragment activity
> remove trailing whitespace   ^
> (same for the other 3 tests)
> 
> Let me know if you agree with these changes and I will make them at 
> commit time.
Yes, please do.
> 
> Thanks
> --Rich
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@xxxxxxxxxxxxxxx
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

<Prev in Thread] Current Thread [Next in Thread>