On Wed, 2012-01-04 at 10:44 +1100, Dave Chinner wrote:
> Given that more people are using xfstests and developing tests, we
> need to consider how to make it friendlier to hack on. The current
> structure of the tree is difficult to work with, the way tests are
> organised and numbered make it difficult to co-ordinate new tests
> and results in patch conflicts, etc.
Coordination of numbers is not a big deal, the test names/numbers
can be easily fixed up at commit time. I also thought that the
numbers--though meaningless on their own--also avoided having to
decide where a particular test belongs. I.e., a test that exercises
several categories of things (maybe preallocation, quota, and ENOSPC)
won't be hidden in any sort of "enospc" test directory.
I do think the growing number of tests is making it a bit unwieldy
though, so I think some sort of reorganization is a good plan.
> We also see problems arising from people not really understanding how
> the xfstests harness is designed and how it really is supposed to
> work, so an overview of the underlying principles of operation would
> probably be helpful to a lot of people. It will also save
> review and rework time if we can avoid having people make the same
> mistakes the first time they submit tests....
This is very important. And the gist of it ought to be
captured somewhere if it is not already.
> I'd also like to discuss some potential infrastructure changes to
> make it easier to add new tests without conflicts with others
> developing new tests. Some of the ideas Christoph and I have
> previously tossed around include:
> - break tests up into groups in their own subdirectories.
> e.g. generic tests, xfs/ext4/btrfs specific tests, stress
> tests, performance tests, large FS tests, etc
> - change the way we define groups of tests so we don't have
> a single registry of tests and their groups
> - allow different naming of tests, such as desciptive text
> names rather than just plain numbers
> - allow duplicate test names in different groups
Despite what I said above, I don't disagree with any of this.
Perhaps the tests can be buried in one or more subdirectories,
but each FSTYP defines its own groups file to drive testing.
> I'm sure that other users of xfstests will have some ideas on how to
> improve it for the way they run it, so I'd like to gather and
> incorporate these ideas into any structural change we make to
Should be a good discussion. It might be useful to have a
proposal or two to work with as a starting point, or maybe
an outline of the types of changes (naming, directory
structure, etc.), to help keep things focused.