xfs
[Top] [All Lists]

Re: [PATCH] xfstests: 253: test the metadump functionality of xfs_db

To: xfs@xxxxxxxxxxx
Subject: Re: [PATCH] xfstests: 253: test the metadump functionality of xfs_db
From: Alex Elder <aelder@xxxxxxx>
Date: Tue, 22 Mar 2011 12:35:46 -0500
In-reply-to: <1298412790.1960.80.camel@doink>
References: <201102182026.p1IKQWRP026319@xxxxxxxxxxxxxxxxxxxxxx> <1298412790.1960.80.camel@doink>
Reply-to: aelder@xxxxxxx
On Tue, 2011-02-22 at 16:13 -0600, Alex Elder wrote:
> (I'm not sure why, but this ended up as a strange attachment the
> first time, so I'm re-sending it.)

Ping?

> 
> This patch creates a test that exercises xfs_metadump, with a focus
> on its obfuscation of names.  It was created to verify fixes that
> avoided a hang condition when running "xfs_metadump" on a directory
> containing files having particular bit patterns in their name.
> Arkadiusz Miśkiewicz first reported seeing this while attempting
> to create a metadump for a filesystem containing a file named
> "R\323\257NE".
> 
> For now this scrikpt checks the following (using only filenames, not
> attributes):
> - that short names (4 characters or less) aren't obfuscated
> - that long names get obfuscated
> - that (long) directory names get obfuscated
> - that names that are known to produce bit patterns that lead
>   to invalid path components still generate obfuscated names
>   (this could previously lead to a hang)
> - that many names of the same length can still generate new
>   obfuscated names (this could previously lead to a hang)
> - that nether " neither "lost+found" nor orphaned files stored in it
>   get obfuscated
> 
> Right now there are two sets of "ls" commands executed (one before
> and one after obfuscation).  This produces repeatable results for
> me on one filesystem, but on a different filesystem I expect the
> inode numbers to change (and random number generation might change
> the output too).  I'm interested in suggestions on how to filter
> the output so the results can be verified.  If nothing else, the
> test serves its purpose if I simply comment out those commands,
> and will do that if there's not a better suggstion.
> 
> Signed-off-by: Alex Elder <aelder@xxxxxxx>
> 
> ---
>  253     |  192
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  253.out |  145 ++++++++++++++++++++++++++++++++++++++++++++++++
>  group   |    1 
>  3 files changed, 338 insertions(+)
> 
> Index: b/253
> ===================================================================
> --- /dev/null
> +++ b/253
> @@ -0,0 +1,192 @@
> +#! /bin/bash
> +# FS QA Test No. 253
> +#
> +# Test xfs_db metadump functionality.
> +#
> +# This test was created to verify fixes for problems where metadump
> +# would never complete due to an inability to find a suitable
> +# obfuscated name to use.  It also verifies a few other things,
> +# including ensuring the "lost+found" directory and orphaned files
> +# in it do not get obfuscated.
> +#
> +# This test also creates a number of files that are effectively
> +# duplicates of existing files; this can happen in certain rare
> +# instances where the obfuscation process has produced a filename
> +# that is already in use (and no other name is available to use).
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2011 SGI.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=aelder@xxxxxxx
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1     # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +    cd /
> +    rm -f $tmp.*
> +    rm -rf "${OUTPUT_DIR}"
> +    rm -f "${METADUMP_FILE}"
> +}
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +_require_scratch
> +
> +# real QA test starts here
> +
> +OUTPUT_DIR="${SCRATCH_MNT}/test_${seq}"
> +METADUMP_FILE="${TEST_MNT}/${seq}_metadump"
> +ORPHANAGE="lost+found"
> +
> +_supported_fs xfs
> +_supported_os Linux
> +
> +function create_file() {
> +     [ $# -eq 1 ] ||         return 1
> +     touch $(printf "$@")
> +}
> +
> +_scratch_mkfs >/dev/null 2>&1
> +_scratch_mount
> +
> +# Initialize and mount the scratch filesystem, then create a bunch
> +# of files that exercise the original problem.
> +#
> +# The problem arose when a file name produced a hash that contained
> +# either 0x00 (string terminator) or 0x27 ('/' character) in a
> +# spot used to determine a character in an obfuscated name.  This
> +# occurred in one of 5 spots at the end of the name, at position
> +# (last-4), (last-3), (last-2), (last-1), or (last).
> +
> +rm -f "${METADUMP_FILE}"
> +
> +mkdir -p "${OUTPUT_DIR}"
> +
> +cd "${OUTPUT_DIR}"
> +# Start out with some basic test files
> +create_file 'abcde'          # hash 0x1c58f263 ("normal" name)
> +
> +create_file 'f'                      # hash 0x00000066 (1-byte name)
> +create_file 'gh'             # hash 0x000033e8 (2-byte name)
> +create_file 'ijk'            # hash 0x001a756b (3-byte name)
> +create_file 'lmno'           # hash 0x0d9b776f (4-byte name)
> +create_file 'pqrstu'         # hash 0x1e5cf9f2 (6-byte name)
> +create_file 'abcdefghijklmnopqrstuvwxyz' # a most remarkable word
> (0x55004ae3)
> +
> +# Create a short directory name; it won't be obfuscated.  Populate
> +# it with some longer named-files.  The first part of the obfuscated
> +# filenames should use printable characters.
> +mkdir foo
> +create_file 'foo/longer_file_name_1' # hash 0xe83634ec
> +create_file 'foo/longer_file_name_2' # hash 0xe83634ef
> +create_file 'foo/longer_file_name_3' # hash 0xe83634ee
> +
> +# Now create a longer directory name
> +mkdir longer_directory_name
> +create_file 'longer_directory_name/f1'       # directory hash 0x9c7accdd
> +create_file 'longer_directory_name/f2'       # filenames are short, no hash
> +create_file 'longer_directory_name/f3'
> +
> +# The problematic name originally reported by Arkadiusz Miśkiewicz
> +
> +create_file 'R\323\257NE'    # hash 0x3a4be740, forces  (last-3) = 0x2f
> +
> +# Other name that force a 0x00 byte
> +create_file 'Pbcde'          # hash 0x0c58f260, forces  (last-4) = 0x00
> +create_file 'a\001\203de'    # hash 0x1000f263, forces  (last-3) = 0x00
> +create_file 'ab\001\344e'    # hash 0x1c403263, forces  (last-2) = 0x00
> +create_file 'abc\200e'               # hash 0x1c588063, forces  (last-1) = 
> 0x00
> +create_file 'abcd\006'               # hash 0x1c58f200, forces    (last) = 
> 0x00
> +
> +# Names that force a 0x2f byte; note no name will ever force (last-4) =
> 0x2f
> +create_file 'a.\343de'               # hash 0x15f8f263 forces   (last-3) = 
> 0x00
> +create_file 'ac\257de'               # hash 0x1c4bf263, forces  (last-2) = 
> 0x2f
> +create_file 'abe\257e'               # hash 0x1c5917e3, forces  (last-1) = 
> 0x2f
> +create_file 'abcd)'          # hash 0x1c58f22f, forces    (last) = 0x2f
> +
> +# The following names are possible results of obfuscating the name
> +# "abcde".  Previously, xfs_metadump could get hung up trying to
> +# obfuscate names when too many of the same length had the same hash
> +# value.
> +create_file '!bcda'          # essentially a dup of 'abcde'
> +create_file 'Abcdg'          # essentially a dup of 'abcde'
> +create_file 'qbcdd'          # essentially a dup of 'abcde'
> +create_file '1bcd`'          # essentially a dup of 'abcde'
> +create_file 'Qbcdf'          # essentially a dup of 'abcde'
> +create_file '\001bcdc'               # essentially a dup of 'abcde'
> +create_file 'Qbce\346'               # essentially a dup of 'abcde'
> +create_file 'abb\344e'               # essentially a dup of 'abcde'
> +
> +# The orphanage directory (lost+found) should not be obfuscated.
> +# Files thereunder can be, but not if their name is the same as
> +# their inode number.  Test this.
> +
> +cd "${SCRATCH_MNT}"
> +mkdir -p "${ORPHANAGE}"
> +
> +TEMP_ORPHAN="${ORPHANAGE}/__orphan__"
> +NON_ORPHAN="${ORPHANAGE}/__should_be_obfuscated__"
> +
> +# Create an orphan, whose name is the same as its inode number
> +touch "${TEMP_ORPHAN}"
> +INUM=$(ls -i "${TEMP_ORPHAN}" | awk '{ print $1; }')
> +ORPHAN="${SCRATCH_MNT}/lost+found/${INUM}"
> +mv "${TEMP_ORPHAN}" "${ORPHAN}"
> +
> +# Create non-orphan, which *should* be obfuscated
> +touch "${NON_ORPHAN}"
> +
> +# Get a listing of all the files before obfuscation
> +ls -R
> +ls -R | od -c
> +
> +# Now unmount the filesystem and create a metadump file
> +cd /; sync; sync             # Old school
> +
> +_scratch_unmount
> +
> +xfs_metadump -f "${SCRATCH_DEV}" "${METADUMP_FILE}"
> +
> +# Now restore the obfuscated one back and take a look around
> +xfs_mdrestore "${METADUMP_FILE}" "${SCRATCH_DEV}"
> +
> +_scratch_mount
> +
> +# Get a listing of all the files after obfuscation
> +cd "${SCRATCH_MNT}"
> +ls -R
> +ls -R | od -c
> +
> +# Finally, re-make the filesystem since to ensure we don't
> +# leave a directory with duplicate entries lying around.
> +cd /
> +_scratch_unmount
> +_scratch_mkfs >/dev/null 2>&1
> +
> +# all done
> +status=0
> +exit
> Index: b/253.out
> ===================================================================
> --- /dev/null
> +++ b/253.out
> @@ -0,0 +1,145 @@
> +QA output created by 253
> +.:
> +lost+found
> +test_253
> +
> +./lost+found:
> +201326721
> +__should_be_obfuscated__
> +
> +./test_253:
> +bcdc
> +!bcda
> +1bcd`
> +Abcdg
> +Pbcde
> +Qbcdf
> +Qbceæ
> +RÓ¯NE
> +aƒde
> +a.ãde
> +abäe
> +abbäe
> +abcd
> +abcd)
> +abcde
> +abcdefghijklmnopqrstuvwxyz
> +abc€e
> +abe¯e
> +ac¯de
> +f
> +foo
> +gh
> +ijk
> +lmno
> +longer_directory_name
> +pqrstu
> +qbcdd
> +
> +./test_253/foo:
> +longer_file_name_1
> +longer_file_name_2
> +longer_file_name_3
> +
> +./test_253/longer_directory_name:
> +f1
> +f2
> +f3
> +0000000   .   :  \n   l   o   s   t   +   f   o   u   n   d  \n   t   e
> +0000020   s   t   _   2   5   3  \n  \n   .   /   l   o   s   t   +   f
> +0000040   o   u   n   d   :  \n   2   0   1   3   2   6   7   2   1  \n
> +0000060   _   _   s   h   o   u   l   d   _   b   e   _   o   b   f   u
> +0000100   s   c   a   t   e   d   _   _  \n  \n   .   /   t   e   s   t
> +0000120   _   2   5   3   :  \n 001   b   c   d   c  \n   !   b   c   d
> +0000140   a  \n   1   b   c   d   `  \n   A   b   c   d   g  \n   P   b
> +0000160   c   d   e  \n   Q   b   c   d   f  \n   Q   b   c   e 346  \n
> +0000200   R 323 257   N   E  \n   a 001 203   d   e  \n   a   . 343   d
> +0000220   e  \n   a   b 001 344   e  \n   a   b   b 344   e  \n   a   b
> +0000240   c   d 006  \n   a   b   c   d   )  \n   a   b   c   d   e  \n
> +0000260   a   b   c   d   e   f   g   h   i   j   k   l   m   n   o   p
> +0000300   q   r   s   t   u   v   w   x   y   z  \n   a   b   c 200   e
> +0000320  \n   a   b   e 257   e  \n   a   c 257   d   e  \n   f  \n   f
> +0000340   o   o  \n   g   h  \n   i   j   k  \n   l   m   n   o  \n   l
> +0000360   o   n   g   e   r   _   d   i   r   e   c   t   o   r   y   _
> +0000400   n   a   m   e  \n   p   q   r   s   t   u  \n   q   b   c   d
> +0000420   d  \n  \n   .   /   t   e   s   t   _   2   5   3   /   f   o
> +0000440   o   :  \n   l   o   n   g   e   r   _   f   i   l   e   _   n
> +0000460   a   m   e   _   1  \n   l   o   n   g   e   r   _   f   i   l
> +0000500   e   _   n   a   m   e   _   2  \n   l   o   n   g   e   r   _
> +0000520   f   i   l   e   _   n   a   m   e   _   3  \n  \n   .   /   t
> +0000540   e   s   t   _   2   5   3   /   l   o   n   g   e   r   _   d
> +0000560   i   r   e   c   t   o   r   y   _   n   a   m   e   :  \n   f
> +0000600   1  \n   f   2  \n   f   3  \n
> +0000610
> +.:
> +lost+found
> +nGp^`%,
> +
> +./lost+found:
> +201326721
> +P4cqsw77yv8UsYbcCa-!i7
> +
> +./nGp^`%,:
> +ƒdc
> +.ãdc
> +bäc
> +b.äc
> +bbã
> +bcdc
> +bceã
> +bd.ã
> +âcd`
> +R.Î@
> +bcd
> +bcd.
> +!bcda
> +Abcdg
> +Abceç
> +R_KspN6ry7jG8CU4bonN2ovd
> +aujzfJamyN3xYjal:X|
> +f
> +foo
> +gh
> +ijk
> +lmno
> +zrst"
> +bcdk
> +bceë
> +Ábcdo
> +Ábceï
> +
> +./nGp^`%,/aujzfJamyN3xYjal:X|:
> +f1
> +f2
> +f3
> +
> +./nGp^`%,/foo:
> +UROC0Bh9cHwp-qI{
> +dFXYpeUryNGb0        a7
> +hBh8n-B-XqcrW
{-KY
> +0000000   .   :  \n   l   o   s   t   +   f   o   u   n   d  \n   n   G
> +0000020   p 004   ^   `   %   ,  \n  \n   .   /   l   o   s   t   +   f
> +0000040   o   u   n   d   :  \n   2   0   1   3   2   6   7   2   1  \n
> +0000060   P   4   c   q   s   w   7   7   y   v   8   U   s   Y   b   c
> +0000100   C   a   -  \v 031   !   i   7  \n  \n   .   /   n   G   p 004
> +0000120   ^   `   %   ,   :  \n 001 001 203   d   c  \n 001   . 343   d
> +0000140   c  \n 001   b 001 344   c  \n 001   b   . 344   c  \n 001   b
> +0000160   b 001 343  \n 001   b   c   d   c  \n 001   b   c   e 343  \n
> +0000200 001   b   d   . 343  \n 001 342   c   d   `  \n 003   R   . 316
> +0000220   @  \n 021   b   c   d 001  \n 021   b   c   d   .  \n   !   b
> +0000240   c   d   a  \n   A   b   c   d   g  \n   A   b   c   e 347  \n
> +0000260   R   _   K   s   p   N   6   r   y   7   j   G   8   C   U   4
> +0000300   b   o   n   N   2  \b   o   v   d 024  \n   a   u   j   z   f
> +0000320   J   a   m   y   N   3   x   Y   j   a   l 001   :   X 030   |
> +0000340  \n   f  \n   f   o   o  \n   g   h  \n   i   j   k  \n   l   m
> +0000360   n   o  \n   z 001   r   s   t   "  \n 201   b   c   d   k  \n
> +0000400 201   b   c   e 353  \n 301   b   c   d   o  \n 301   b   c   e
> +0000420 357  \n  \n   .   /   n   G   p 004   ^   `   %   ,   /   a   u
> +0000440   j   z   f   J   a   m   y   N   3   x   Y   j   a   l 001   :
> +0000460   X 030   |   :  \n   f   1  \n   f   2  \n   f   3  \n  \n   .
> +0000500   /   n   G   p 004   ^   `   %   ,   /   f   o   o   :  \n   U
> +0000520   R   O   C   0   B   h   9   c   H   w   p   -  \b   q 031   I
> +0000540   {  \n   d   F   X   Y   p   e   U   r   y   N   G   b   0  \t
> +0000560 025 026   a   7  \n   h   B   h   8   n   -   B   -   X   q   c
> +0000600   r   W  \r   {   -   K   Y  \n
> +0000610
> Index: b/group
> ===================================================================
> --- a/group
> +++ b/group
> @@ -366,3 +366,4 @@ deprecated
>  250 auto quick rw prealloc metadata
>  251 ioctl trim
>  252 auto quick prealloc
> +253 auto quick group
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs



<Prev in Thread] Current Thread [Next in Thread>
  • Re: [PATCH] xfstests: 253: test the metadump functionality of xfs_db, Alex Elder <=