Hi!
I'm currently using LVM snapshots to create full system backups
of a bunch of Xen-based virtual machines (so-called domUs).
Those domUs all run Xen kernel 2.6.18 from the Xen 3.2.0 release
(32bit domU on 32bit dom0, I can post the .config if needed).
All domUs are using XFS on their LVM logical volumes.
The backup of all mounted snapshot volumes is made using
rsnapshot/rsync. This has been running smoothly for some
weeks now on 5 domUs.
Yesterday this happened during the backup on 1 domU:
--8<--
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x604d68
("xfs_trans_read_buf") error 5 buf count 4096
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x66c5a0
("xfs_trans_read_buf") error 5 buf count 4096
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x202f70
("xfs_trans_read_buf") error 5 buf count 4096
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x2701f8
("xfs_trans_read_buf") error 5 buf count 4096
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x6a78
("xfs_trans_read_buf") error 5 buf count 4096
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x600500
("xfs_trans_read_buf") error 5 buf count 8192
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x600520
("xfs_trans_read_buf") error 5 buf count 8192
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x600520
("xfs_trans_read_buf") error 5 buf count 8192
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0xdd0
("xfs_trans_read_buf") error 5 buf count 8192
kernel: I/O error in filesystem ("dm-21") meta-data dev dm-21 block 0x4055d0
("xfs_trans_read_buf") error 5 buf count 8192
[...many more of such messages...]
kernel: xfs_force_shutdown(dm-21,0x1) called from line 424 of file
fs/xfs/xfs_rw.c. Return address = 0xc02b1cbb
kernel: Filesystem "dm-21": I/O Error Detected. Shutting down filesystem: dm-21
kernel: Please umount the filesystem, and rectify the problem(s)
kernel: xfs_force_shutdown(dm-21,0x1) called from line 424 of file
fs/xfs/xfs_rw.c. Return address = 0xc02b1cbb
--8<--
The rsync process was then terminated with SIGBUS (exit code 135 -> 128+7).
The device dm-21 was the snapshot of the /var filesystem and
was mounted using nouuid,norecovery.
Is it possible that the LVM snapshot (that should be using
xfs_freeze/xfs_unfreeze) has created an inconsistent/damaged
snapshot that was kept from being repaired through norecovery?
Any other ideas?
--
Regards,
Wolfram Schlich <wschlich@xxxxxxxxxx>
Gentoo Linux * http://dev.gentoo.org/~wschlich/
|