xfs
[Top] [All Lists]

[RFC] [PATCH 0/18] xfstests: move tests out of top level

To: xfs@xxxxxxxxxxx
Subject: [RFC] [PATCH 0/18] xfstests: move tests out of top level
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Thu, 26 Jul 2012 19:27:54 +1000
Alt-Subject: Games with Sed, Grep and Awk.

This series is based on top of the large filesystem test series.

This moves all the tests into a ./tests subdirectory, and sorts them into
classes of related tests. Those are:

        tests/generic:  valid for all filesystems
        tests/shared:   valid for a limited number of filesystems
        tests/xfs:      xfs specific tests
        tests/btrfs     btrfs specific tests
        tests/ext4      ext4 specific tests
        tests/udf       udf specific tests

Each directory has it's own group file to determine what groups the
tests are associated with. Tests are run in exactly the same was as
before, but when trying to run individual tests you need to specify
the class as well. e.g. the old way:

# ./check 001

The new way:

# ./check generic/001

The output also indicates what class the test came from:

$ sudo ./check -g auto
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/x86_64 test-1 3.5.0-rc5-dgc+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/vdb
MOUNT_OPTIONS -- /dev/vdb /mnt/scratch

generic/001      3s
generic/002      0s
generic/005      1s
generic/006      10s
generic/007      2s
generic/010      [not run] dbtest was not built for this platform
generic/011      15s
generic/013      43s
generic/014      2s
generic/015      0s
generic/020      3s
generic/053      0s
....

[I'd post more example output, but a lightning strike took out power
a little while ago and so all the output in my scrollback buffers
went bye-bye...]

The test classes that are run are generic, shared and $FSTYP, hence
avoiding most "notrun, wrong filesystem" cases.

Further, the test result/status files (e.g. 001.full) are also moved
out of the top level directory into a new results directory. This
defaults to ./results ($RESULT_BASE) and duplicates the heirarchy of
the tests/ directory. It is created on demand. Each test is passed
$RESULT_DIR which points to the directory it should dump it's output
files in. i.e. "echo foo > $RESULT_DIR/$seq.full".

There's a bunch of cleanup at the start of the series, removing
stuff that I don't think has been used for years. e.g. i didn't even
know the remake script existed, but it's usefulness is minimal are
we rarely, if ever, regenerate every single .out file in the test
suite. Hence stuff is removed to make it easy to convert the
important stuff to use the new structure....

Feel free to debate things like the renaming of variables - I just
used sed scripts to do most of the conversion and most of them are
in the commit messages so I can easily re-run them to do global
search/replace if you've got better ideas for naming stuff ($seqres
is pretty 'orrible)....

Overall, this series shows the direction I want to take xfstests in.
The next steps are:

        - remove remaining limitations on test naming (i.e.  must be
          numbered) so that we can have descriptive names
        - move all of the output into the results directory and
          enable it to be hosted externally so it can be archived
          and data mined easily
        - move all the common* files to a subdirectory
        - allow running of test classes, not just groups
        - re-introduce the expunged file functionailty (which I
          didn't know existed) because I can see how useful that is
          for running regular QA with a current xfstests on an older
          distro (e.g. RHEL5) to avoid running tests that are known
          to fail or test features that aren't in old kernels...

I sent the patches in git format for all the renames - you don't
need to see a patchset that is this size:

1280 files changed, 102397 insertions(+), 104307 deletions(-)

When turning on rename detection makes it this size:

686 files changed, 1722 insertions(+), 3632 deletions(-)

Which is much more manageable to review....

I'm certain there are problems still in there - I haven't done a lot
of weird command line testing and really only just enough testing to
make sure a typical auto group test run mostly passes.....

Comments, additional ideas, new functionality, modifications, etc 
are all welcome.

<Prev in Thread] Current Thread [Next in Thread>