Re: xfsrestore report SUCCESS but not restore all files in kernel 3.17

[Tommy Wu <wu.tommy@xxxxxxxxx>]

It' look like no help after rebuild xfsdump/xfsprogs, it still got the same error for some case.

2014-10-30 16:39 GMT+08:00 Tommy Wu <wu.tommy@xxxxxxxxx>:

I'm confusing now...

in the test VM and one of my machine, rebuild xfsdump/xfsprogs will make the test work.

In one of my machine, still got error.

fw1:/vol/backup/x# ldd /sbin/xfsdump
ÂÂÂÂÂÂÂ linux-vdso.so.1 (0x00007fff3f7fe000)
ÂÂÂÂÂÂÂ libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f8c2bf9d000)
ÂÂÂÂÂÂÂ libhandle.so.1 => /lib/libhandle.so.1 (0x00007f8c2bd99000)
ÂÂÂÂÂÂÂ libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f8c2bb93000)
ÂÂÂÂÂÂÂ libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8c2b976000)
ÂÂÂÂÂÂÂ libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8c2b5cd000)
ÂÂÂÂÂÂÂ /lib64/ld-linux-x86-64.so.2 (0x00007f8c2c1b5000)
fw1:/vol/backup/x# ldd /sbin/xfsrestore
ÂÂÂÂÂÂÂ linux-vdso.so.1 (0x00007fffcee8d000)
ÂÂÂÂÂÂÂ libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007ff53d507000)
ÂÂÂÂÂÂÂ libhandle.so.1 => /lib/libhandle.so.1 (0x00007ff53d303000)
ÂÂÂÂÂÂÂ libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007ff53d0fd000)
ÂÂÂÂÂÂÂ libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff53cee0000)
ÂÂÂÂÂÂÂ libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff53cb37000)
ÂÂÂÂÂÂÂ /lib64/ld-linux-x86-64.so.2 (0x00007ff53d71f000)

the xfsdump command I use:
xfsdump -v debug -l 0 -o -p 300 -J -F -M test -L test -f test.xfsdump /mnt/backup > xfsdump.log

the log: https://mega.co.nz/#!xIhTjD6Y!Gwp_ilW5UyRiqn2DSxPTHM0TH5qBghMlKXAmHoWk2iQ

the xfsrestore command I use:
xfsrestore -v debug,tree=nitty -p 300 -J -f test.xfsdump /vol/backup/x/ > xfsrestore.log

the log: https://mega.co.nz/#!oZIFWQST!VUY90EZ8XvZDYy9GwF5mqtQVtWNW90Sar5MZhNzGbYI

2014-10-30 13:03 GMT+08:00 Dave Chinner <david@xxxxxxxxxxxxx>:

On Thu, Oct 30, 2014 at 11:07:21AM +0800, Tommy Wu wrote:
> I just create a x86-64 VM (using VirtualBox 4.3.18, under windows 8).
> Install Debian jessie on it. Create a test partition in LVM for testing.
> (so I think it's not hardware issue here)
>
> test procedure:
> 1. mkfs.xfs the 1st partition in LVM
> 2. mount the 1st partition
> 3. extract linux-3.17.tar.xz to the partition
> 4. xfsdump the 1st partition
> 5. mkfs.xfs the 2nd partition in LVM
> 6. mount the 2nd partition
> 7. xfsrestore the dump file to 2nd partition
> 8. compare the file/directory in 2 partitions.

Partition sizes are roughly a 5GB source and 10GB destination.

So there's this many files in the fs:

> xfsrestore: 3054 directories and 50556 entries processed

OK, so on 3.18-rc2:

> root@debian:/mnt/xfsdump# mount /dev/debian/xfsrestore /mnt/xfsrestore
> root@debian:/mnt/xfsdump# cd /mnt/xfsrestore/
> root@debian:/mnt/xfsrestore# xfsdump -l 0 -o -p 300 -J -F -M test -L test -
> /mnt/xfsdump | gzip -qv > /mnt/xfsrestore/xfsdump.gz
....
> xfsdump: media file size 575648352 bytes
> xfsdump: dump size (non-dir files) : 560258760 bytes

These are a handful of bytes different.

> xfsdump: dump complete: 21 seconds elapsed
> xfsdump: Dump Status: SUCCESS
>Â 78.5%
> root@debian:/mnt/xfsrestore# ls -la
> total 120680
> drwxr-xr-x 2 root root    23 Oct 30 10:48 .
> drwxr-xr-x 4 root root   4096 Oct 30 10:37 ..
> -rw-r--r-- 1 root root 123569739 Oct 30 10:48 xfsdump.gz
> root@debian:/mnt/xfsrestore# cat /mnt/xfsrestore/xfsdump.gz | gzip -dqv |
> xfsrestore -p 300 -J -t - | grep xfsrestore

and the test shows:

> xfsrestore: reading directories
> xfsrestore: 2035 directories and 33045 entries processed

A thousand less directories and 20000 less files, which doesn't make
much sense for a dump that has a few bytes difference in size.

So, I've done almost exactly the same test locally (I used the
3.18-rc2 tarball) and on both the upstream 3.18-rc2 kernel and my
current xfs-fixes-for-3.18-rc3 branch I can't reproduce your
problem:

$Â find /mnt/test -type d |wc -l
3080
$ find /mnt/test |wc -l
51068
$ zcat dump.gz | sudo xfsrestore -p 300 -J - . |grep xfsrestore
.....
xfsrestore: reading directories
xfsrestore: 3080 directories and 51068 entries processed
xfsrestore: directory post-processing
xfsrestore: restoring non-directory files
xfsrestore: restore complete: 7 seconds elapsed
xfsrestore: Restore Status: SUCCESS
$ find /mnt/scratch -type d |wc -l
3080
$ find /mnt/scratch |wc -l
51069
$ ls /mnt/scratch
dump.gz linux-3.18-rc2
$

Which shows that it's working just fine. I've tried cold cache
dumps. I've tried crc enabled filesystems, I've tried a bunch of
random permutations, but I cannot get dump/restore to do what you
are seeing it do. Whatever problem you are seeing is specific to
your system.

Hmmm - can you run 'ldd xfsdump' and 'ldd xfsrestore' and paste the
output for me?

Can you also please run a full test again on a kernel built from the
current xfs-fixes-for-3.18-rc3 branch, with full dump/restore debug
enabled and send me the output? i.e.

$ xfsdump -v debug ...

and

$ xfsrestore -v debug,tree=nitty ...

They ar going to produce a lot of output, so please capture it to
files rather than trying to cut/paste output to an email.

Cheers,

Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx

--

Tommy Wu

--

Tommy Wu