xfs
[Top] [All Lists]

Re: [PATCH 00/12] run more generic tests on TEST_DIR

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: [PATCH 00/12] run more generic tests on TEST_DIR
From: Christoph Hellwig <hch@xxxxxxxxxxxxx>
Date: Fri, 13 Dec 2013 03:10:39 -0800
Cc: Christoph Hellwig <hch@xxxxxxxxxxxxx>, xfs@xxxxxxxxxxx, Stanislav Kholmanskikh <stanislav.kholmanskikh@xxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <20131213010934.GN10988@dastard>
References: <20131211073445.353655850@xxxxxxxxxxxxxxxxxxxxxx> <20131211225012.GL10988@dastard> <20131212180312.GC19422@xxxxxxxxxxxxx> <20131213010934.GN10988@dastard>
User-agent: Mutt/1.5.21 (2010-09-15)
On Fri, Dec 13, 2013 at 12:09:34PM +1100, Dave Chinner wrote:
> > You can have different test devices, or simply not bother with aging
> > it for every run.  You're missing the coverage of all the test dir
> > using tests, which are a lot with the above version anyway.
> 
> IOWs, you're saying that you don't consider MKFS_OPTIONS as a first
> class citizen. I've been using it for 7 or 8 years for exactly this
> purpose - iterating testing of a change quickly across multiple
> configurations without perturbing the long term aging of the test
> device.

But you're limiting yourself to the tests only using the scratch
device for that testing, leaving out all the ones using the TEST
directory.

> I'm not opposed to making the change, just pointing out that
> reducing the usage of the scratch device has a fairly significant
> impact on test coverage for anyone who uses MKFS_OPTIONS in their
> workflow...

It does have an impact for that particular workload, but I think that
workload is broken as you only test your specific config for those
tests using the scratch device, and do not get the coverage for the
tests using the test device.

git-grep -l TEST_DIR tests/generic/ | grep -v out | wc -l
65
git-grep -l TEST_DIR tests/xfs/ | grep -v out | wc -l 
23


hch@brick:~/work/xfstests$ git-grep -l _require_scratch tests/generic/ | wc -l
58
hch@brick:~/work/xfstests$ git-grep -l _require_scratch tests/xfs/ | wc -l
128

So you're missing close to 2/3s of the tests already.

<Prev in Thread] Current Thread [Next in Thread>