Bugzilla – Bug 400
lvm2 snapshots?
Last modified: 2009-01-05 11:14:57 CST
We run linux 2.6.11.6 with LVM2 and XFS. We try to use snapshots to do nightly backups. The error message is: Mar 29 14:04:26 fs kernel: Access to block zero: fs: <dm-4> inode: 529201100 start_block : 0 start_off : 0 blkcnt : 0 extent-state : 0 Mar 29 14:04:26 fs kernel: ------------[ cut here ]------------ Mar 29 14:04:26 fs kernel: kernel BUG at fs/xfs/support/debug.c:106! Mar 29 14:04:26 fs kernel: invalid operand: 0000 [#1] Mar 29 14:04:26 fs kernel: SMP Mar 29 14:04:26 fs kernel: Modules linked in: ide_cd cdrom siimage piix e1000 qla2300 qla2xxx scsi_transport_fc dm_sna pshot dm_mod Mar 29 14:04:26 fs kernel: CPU: 0 Mar 29 14:04:26 fs kernel: EIP: 0060:[<c024e85b>] Not tainted VLI Mar 29 14:04:26 fs kernel: EFLAGS: 00010246 (2.6.11.6-sara1) Mar 29 14:04:26 fs kernel: EIP is at cmn_err+0xab/0xc0 Mar 29 14:04:26 fs kernel: eax: 00000000 ebx: c0363b40 ecx: 00000001 edx: 00000293 Mar 29 14:04:26 fs kernel: esi: 00000000 edi: c0476d00 ebp: e0ac9814 esp: e0ac97f4 Mar 29 14:04:26 fs kernel: ds: 007b es: 007b ss: 0068 Mar 29 14:04:26 fs kernel: Process dsmc (pid: 13791, threadinfo=e0ac8000 task=ed7ca530) Mar 29 14:04:26 fs kernel: Stack: c0353ef6 c035a3b0 c0476d00 00000293 00000000 00000000 00000000 e0ac9974 Mar 29 14:04:26 fs kernel: e0ac9858 c01f73cc 00000000 c0363b40 f71102e0 1f8af7cc 00000000 00000000 Mar 29 14:04:26 fs kernel: 00000000 00000000 00000000 00000000 00000000 00000000 e0ac9974 023caf60 Mar 29 14:04:26 fs kernel: Call Trace: Mar 29 14:04:26 fs kernel: [<c010354f>] show_stack+0x7f/0xa0 Mar 29 14:04:26 fs kernel: [<c01036ff>] show_registers+0x15f/0x1d0 Mar 29 14:04:26 fs kernel: [<c0103924>] die+0xf4/0x180 Mar 29 14:04:26 fs kernel: [<c0103ddc>] do_invalid_op+0xbc/0xd0 Mar 29 14:04:26 fs kernel: [<c01031bb>] error_code+0x2b/0x30 Mar 29 14:04:26 fs kernel: [<c01f73cc>] xfs_bmap_search_extents+0xfc/0x120 Mar 29 14:04:26 fs kernel: [<c01f88be>] xfs_bmapi+0x26e/0x15f0 Mar 29 14:04:26 fs kernel: [<c022361e>] xfs_iomap+0x16e/0x4b0 Mar 29 14:04:26 fs kernel: [<c0244ae0>] __linvfs_get_block+0x90/0x2b0 Mar 29 14:04:26 fs kernel: [<c0244d42>] linvfs_get_block+0x42/0x50 Mar 29 14:04:26 fs kernel: [<c0175fe6>] do_mpage_readpage+0x1b6/0x4e0 Mar 29 14:04:26 fs kernel: [<c0176449>] mpage_readpages+0x139/0x160 Mar 29 14:04:26 fs kernel: [<c013a226>] read_pages+0x126/0x130 Mar 29 14:04:26 fs kernel: [<c013a322>] __do_page_cache_readahead+0xf2/0x150 Mar 29 14:04:26 fs kernel: [<c013a554>] blockable_page_cache_readahead+0x64/0x80 Mar 29 14:04:26 fs kernel: [<c013a6cf>] page_cache_readahead+0x15f/0x2d0 Mar 29 14:04:26 fs kernel: [<c0133ba8>] do_generic_mapping_read+0x578/0x590 Mar 29 14:04:26 fs kernel: [<c0133ead>] __generic_file_aio_read+0x1ed/0x220 Mar 29 14:04:26 fs kernel: [<c024b0b0>] xfs_read+0x130/0x300 Mar 29 14:04:26 fs kernel: [<c02478be>] linvfs_aio_read+0x8e/0xa0 Mar 29 14:04:26 fs kernel: [<c01521f3>] do_sync_read+0xa3/0xd0 Mar 29 14:04:26 fs kernel: [<c01522d0>] vfs_read+0xb0/0x130 Mar 29 14:04:26 fs kernel: [<c015259b>] sys_read+0x4b/0x80 Mar 29 14:04:26 fs kernel: [<c0102717>] syscall_call+0x7/0xb Mar 29 14:04:26 fs kernel: Code: 89 54 24 08 8b 55 f0 8b 04 95 a0 3f 3a c0 89 44 24 04 e8 d9 88 ec ff 8b 55 ec b8 84 3 f 3a c0 e8 3c 3f 0f 00 8b 45 f0 85 c0 75 08 <0f> 0b 6a 00 cc a3 35 c0 83 c4 14 5b 5e 5f 5d c3 90 8d 74 26 00 I figured that maybe this happens because we don't use xfs_freeze. For some reason, xfs_freeze freezes the filesystem so good that lvcreate -s does not work either (hangs until you unfreeze manually). We are running on linux 2.6.11.6 with LVM2 and XFS. lvm2 utilities are version 2.01.04 xfsprogs are version 2.6.20 Greets, --Walter
I found out that xfs_freeze results in an XFS_IOC_FREEZE in xfs/linux-2.6/xfs_ioctl.c This calls freeze_bdev() We are using device mapper and I suspect that dm_suspend() should be called instead. dm_suspend() is in drivers/md/dm.c and the comments for this function describe exactly what you want xfs_freeze to do: <QUOTE> /* * We need to be able to change a mapping table under a mounted * filesystem. For example we might want to move some data in * the background. Before the table can be swapped with * dm_bind_table, dm_suspend must be called to flush any in * flight bios and ensure that any further io gets deferred. */ int dm_suspend(struct mapped_device *md) </QUOTE> This function is also called when you do 'dmsetup suspend /dev/mapper/some_lv' from the shell. As a workaround, I am now using 'dmsetup suspend' rather than 'xfs_freeze -f'.
OK suspend is definately not the correct way to do it. I found an e-mail on the web from Alasdair Kergon saying that it's a problem with the LVM2 userspace tools. But this was from Oct 2004 and apparently has not been fixed yet. Altogether it doesn't seem to be an XFS problem (at least, as far as I can see).
Almost same setup (SLES 9 SP1 with kernel 2.6.11.12, LVM2 volume with XFS, Snapshots taken using EVMS 2.5.2) and same error here: Jun 26 06:51:09 ko-cluster kernel: Access to block zero: fs: <dm-34> inode: 762352251 start_block : 0 start_off : 0 blkcnt : 0 extent-state : 0 Jun 26 06:51:09 ko-cluster kernel: ------------[ cut here ]------------ Jun 26 06:51:09 ko-cluster kernel: kernel BUG at fs/xfs/support/debug.c:106! Jun 26 06:51:09 ko-cluster kernel: invalid operand: 0000 [#1] Jun 26 06:51:09 ko-cluster kernel: SMP Jun 26 06:51:09 ko-cluster kernel: Modules linked in: ipt_MASQUERADE nls_iso8859_1 nls_cp437 w83627hf eeprom w83781d adm1021 i2c_sensor i2c_i801 i2c_dev st sr_mod nls_cp850 nls_utf8 smbfs vfat fat nfsd bond0 tg3 ipt_TCPMSS ipt_TOS e100 mii i2c_core hw_random uhci_hcd evdev ipt_state ipt_LOG e1000 usbcore ipt_REJECT iptable_mangle iptable_filter ip_nat_ftp iptable_nat ip_conntrack_ftp ip_conntrack ip_tables ide_cd cdrom capability commoncap xfs exportfs ext3 jbd dm_snapshot dm_mod aic7xxx sd_mod scsi_mod Jun 26 06:51:09 ko-cluster kernel: CPU: 0 Jun 26 06:51:09 ko-cluster kernel: EIP: 0060:[<f8f6c3f5>] Not tainted VLI Jun 26 06:51:09 ko-cluster kernel: EFLAGS: 00010246 (2.6.11.12MSTEST) Jun 26 06:51:09 ko-cluster kernel: EIP is at cmn_err+0xa5/0xd0 [xfs] Jun 26 06:51:09 ko-cluster kernel: eax: f8f83c84 ebx: f8f6f1e8 ecx: cae1f898 edx: 00000293 Jun 26 06:51:09 ko-cluster kernel: esi: 00000000 edi: f8f88140 ebp: 00000000 esp: cae1f894 Jun 26 06:51:09 ko-cluster kernel: ds: 007b es: 007b ss: 0068 Jun 26 06:51:09 ko-cluster kernel: Process bpbkar (pid: 4866, threadinfo=cae1e000 task=f5823540) Jun 26 06:51:09 ko-cluster kernel: Stack: f8f6efdc f8f6efa3 f8f88140 00000293 00000000 00000000 cae1fa04 ea147310 Jun 26 06:51:09 ko-cluster kernel: f8f16f08 00000000 f8f6f1e8 f6299ee0 2d70927b 00000000 00000000 00000000 Jun 26 06:51:09 ko-cluster kernel: 00000000 00000000 00000000 00000000 00000000 00000000 ea147360 ea147310 Jun 26 06:51:09 ko-cluster kernel: Call Trace: Jun 26 06:51:09 ko-cluster kernel: [<f8f16f08>] xfs_bmap_search_extents+0xf8/0x120 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f8f18382>] xfs_bmapi+0x2b2/0x1750 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f886c9e8>] dm_request+0x68/0xa0 [dm_mod] Jun 26 06:51:09 ko-cluster kernel: [<c0133350>] autoremove_wake_function+0x0/0x50 Jun 26 06:51:09 ko-cluster kernel: [<c024aed7>] generic_make_request+0xb7/0x220 Jun 26 06:51:09 ko-cluster kernel: [<c0133350>] autoremove_wake_function+0x0/0x50 Jun 26 06:51:09 ko-cluster kernel: [<c0133301>] finish_wait+0x21/0x70 Jun 26 06:51:09 ko-cluster kernel: [<f886c954>] __split_bio+0xf4/0x120 [dm_mod] Jun 26 06:51:09 ko-cluster kernel: [<f8f4212e>] xfs_imap_to_bmap+0x2e/0x2b0 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f8f42535>] xfs_iomap+0x185/0x4b0 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f8f62ae3>] __linvfs_get_block+0x83/0x320 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f8f62daf>] linvfs_get_block+0x2f/0x40 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<c018228c>] do_mpage_readpage+0x18c/0x490 Jun 26 06:51:09 ko-cluster kernel: [<c0291913>] nf_iterate+0x63/0xb0 Jun 26 06:51:09 ko-cluster kernel: [<c0111bd5>] smp_send_reschedule+0x55/0x70 Jun 26 06:51:09 ko-cluster kernel: [<c01e2a10>] radix_tree_node_alloc+0x10/0x60 Jun 26 06:51:09 ko-cluster kernel: [<c01e2c5b>] radix_tree_insert+0x8b/0x110 Jun 26 06:51:09 ko-cluster kernel: [<c013e26a>] add_to_page_cache+0xba/0xe0 Jun 26 06:51:09 ko-cluster kernel: [<c01825e6>] mpage_readpages+0x56/0x120 Jun 26 06:51:09 ko-cluster kernel: [<f8f62d80>] linvfs_get_block+0x0/0x40 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<c0145210>] read_pages+0xf0/0x100 Jun 26 06:51:09 ko-cluster kernel: [<f8f62d80>] linvfs_get_block+0x0/0x40 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<c0142db5>] __alloc_pages+0xc5/0x3b0 Jun 26 06:51:09 ko-cluster kernel: [<c01452fe>] __do_page_cache_readahead+0xde/0x150 Jun 26 06:51:09 ko-cluster kernel: [<c0145500>] blockable_page_cache_readahead+0x40/0x60 Jun 26 06:51:09 ko-cluster kernel: [<c01456a0>] page_cache_readahead+0x180/0x2a0 Jun 26 06:51:09 ko-cluster kernel: [<c013ebe5>] do_generic_mapping_read+0x355/0x510 Jun 26 06:51:09 ko-cluster kernel: [<c013eda0>] file_read_actor+0x0/0xf0 Jun 26 06:51:09 ko-cluster kernel: [<c013f082>] __generic_file_aio_read+0x1f2/0x230 Jun 26 06:51:09 ko-cluster kernel: [<c013eda0>] file_read_actor+0x0/0xf0 Jun 26 06:51:09 ko-cluster kernel: [<f8f68ff1>] xfs_read+0x141/0x300 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<f8f656c5>] linvfs_aio_read+0x85/0xb0 [xfs] Jun 26 06:51:09 ko-cluster kernel: [<c015e4c2>] do_sync_read+0xb2/0x100 Jun 26 06:51:09 ko-cluster kernel: [<c0105f2b>] do_IRQ+0x3b/0x70 Jun 26 06:51:09 ko-cluster kernel: [<c0133350>] autoremove_wake_function+0x0/0x50 Jun 26 06:51:09 ko-cluster kernel: [<c015e5e3>] vfs_read+0xd3/0x130 Jun 26 06:51:09 ko-cluster kernel: [<c015e8b1>] sys_read+0x41/0x70 Jun 26 06:51:09 ko-cluster kernel: [<c0103cd3>] syscall_call+0x7/0xb Jun 26 06:51:09 ko-cluster kernel: Code: b8 40 81 f8 f8 89 44 24 08 8b 04 ad a0 3c f8 f8 89 44 24 04 e8 8d 29 1b c7 8b 54 24 0c b8 84 3c f8 f8 e8 ff 6b 38 c7 85 ed 75 08 <0f> 0b 6a 00 c3 ef f6 f8 8b 5c 24 10 8b 74 24 14 8b 7c 24 18 8b
Can you still reproduce any of these? Defintively feels like a dm issue to me.
I no longer have access to a test system with lvm+xfs, sorry.
Closing as it's not reproducible anymore.