Hi,
Firstly I don't actually know that this is the fault of xfs. I wouldn't be able
to figure out who or what is to blame in a million years. At any rate I'd like
if the xfs people could look into this becuase it might be an xfs issue and
even if it isn't it impacts xfs in a big way.
So far I've only seen this problem in a qemu/kvm fedora vm (the host machine is
also fedora and the problem has not [yet] occured on the host machine). I'm
running the most up to date fedora kernel. I know that this problem occurs with
kernels 4.4.3 and 4.4.2 and I think some previous ones so this could be a
kernel regression. OTO the problem did not arise with the rescue kernel (which
is a stripped down 4.2.8 kernel, but that could be because of less resource
usage). Also frequency of occurence can be reduced by setting vm.swappiness to
1 or 0 and presumably it can be increased by increasing vm.swappiness.
Anyway so about 80% of the time when I run xfsdump in the guest machine it
fails part way and the following stack trace appears in the log.
THank you for your time.
Mar 11 22:29:11 ourdeploy kernel: XFS (vdb1): Mounting V5 Filesystem
Mar 11 22:29:11 ourdeploy kernel: XFS (vdb1): Starting recovery
(logdev: internal)
Mar 11 22:29:11 ourdeploy kernel: XFS (vdb1): Ending recovery (logdev:
internal)
Mar 11 22:29:17 ourdeploy audit[1]: SERVICE_STOP pid=1 uid=0
auid=4294967295 ses=4294967295 msg='unit=rolekit comm="systemd"
exe="/us
r/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 11 22:37:14 ourdeploy kernel: page:ffffea00000267c0 count:0
mapcount:-127 mapping: (null) index:0x0
Mar 11 22:37:14 ourdeploy kernel: flags: 0x1ffff800000000()
Mar 11 22:37:14 ourdeploy kernel: page dumped because:
VM_BUG_ON_PAGE(atomic_read(&page->_count) == 0)
Mar 11 22:37:14 ourdeploy kernel: ------------[ cut here ]------------
Mar 11 22:37:14 ourdeploy kernel: kernel BUG at include/linux/mm.h:342!
Mar 11 22:37:14 ourdeploy kernel: invalid opcode: 0000 [#1] SMP
Mar 11 22:37:14 ourdeploy kernel: Modules linked in: nls_utf8 cifs
dns_resolver fscache nf_conntrack_netbios_ns nf_conntrack_broadcas
t nf_conntrack_tftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6
xt_conntrack ip_set nfnetlink ebtable_broute bridge stp llc
ebtable_filt
er ebtable_nat ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6
nf_nat_ipv6 ip6table_raw ip6table_mangle ip6table_security ip6t
able_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4
nf_nat_ipv4 nf_nat nf_conntrack iptable_raw iptable_mangle iptabl
e_security snd_hda_codec_generic snd_hda_intel snd_hda_codec ppdev
snd_hda_core iosf_mbi crct10dif_pclmul snd_hwdep crc32_pclmul snd_
pcm joydev snd_timer virtio_balloon parport_pc snd parport acpi_cpufreq
i2c_piix4 soundcore tpm_tis tpm nfsd nfs_acl lockd grace auth
_rpcgss sunrpc xfs libcrc32c virtio_console
Mar 11 22:37:14 ourdeploy kernel: virtio_net virtio_blk qxl
drm_kms_helper ttm drm crc32c_intel serio_raw virtio_pci ata_generic
vir
tio_ring virtio pata_acpi
Mar 11 22:37:14 ourdeploy kernel: CPU: 0 PID: 1458 Comm: xfsdump Not
tainted 4.4.4-301.fc23.x86_64 #1
Mar 11 22:37:14 ourdeploy kernel: Hardware name: QEMU Standard PC
(i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014
Mar 11 22:37:14 ourdeploy kernel: task: ffff88003e1d9e00 ti:
ffff880028ccc000 task.ti: ffff880028ccc000
Mar 11 22:37:14 ourdeploy kernel: RIP:
0010:[<ffffffff811b5008>] [<ffffffff811b5008>] __free_pages+0x38/0x40
Mar 11 22:37:14 ourdeploy kernel: RSP: 0018:ffff880028ccfa48 EFLAGS:
00010246
Mar 11 22:37:14 ourdeploy kernel: RAX: 0000000000000044 RBX:
ffff88003f547900 RCX: 0000000000000006
Mar 11 22:37:14 ourdeploy kernel: RDX: 0000000000000000 RSI:
0000000000000000 RDI: ffff880042c0dff0
Mar 11 22:37:14 ourdeploy kernel: RBP: ffff880028ccfa48 R08:
0000000000000000 R09: 0000000000000254
Mar 11 22:37:14 ourdeploy kernel: R10: 0000000000000001 R11:
0000000000000254 R12: 0000000000000001
Mar 11 22:37:14 ourdeploy kernel: R13: ffffffffa0180d4d R14:
ffff880028ccfb38 R15: 0000000000000001
Mar 11 22:37:14 ourdeploy kernel: FS: 00007f57b020a780(0000)
GS:ffff880042c00000(0000) knlGS:0000000000000000
Mar 11 22:37:14 ourdeploy kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Mar 11 22:37:14 ourdeploy kernel: CR2: 00007f21861ec024 CR3:
0000000030025000 CR4: 00000000000406f0
Mar 11 22:37:14 ourdeploy kernel: Stack:
Mar 11 22:37:14 ourdeploy kernel: ffff880028ccfa70 ffffffffa017fed3
ffff880036d073c0 0000000000000000
Mar 11 22:37:14 ourdeploy kernel: 0000000000010015 ffff880028ccfab0
ffffffffa0180d4d ffff88003f547900
Mar 11 22:37:14 ourdeploy kernel: 0000000000010015 0000000000000001
0000000000010014 ffff880036d073c0
Mar 11 22:37:14 ourdeploy kernel: Call Trace:
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa017fed3>]
xfs_buf_free+0x73/0x130 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa0180d4d>]
xfs_buf_get_map+0x22d/0x280 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa018187d>]
xfs_buf_read_map+0x2d/0x180 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa0181a22>]
xfs_buf_readahead_map+0x52/0x70 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa0158bc1>]
xfs_btree_reada_bufs+0x61/0x80 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa01962d8>]
xfs_bulkstat_ichunk_ra.isra.4+0xe8/0x140 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa016e8b3>] ?
xfs_inobt_get_rec+0x33/0xc0 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa0196973>]
xfs_bulkstat+0x303/0x670 [xfs]
Mar 11 22:37:14 ourdeploy kernel: [<ffffffff81244c3d>] ?
__dentry_kill+0x13d/0x1b0
Mar 11 22:37:14 ourdeploy kernel: [<ffffffffa0196640>] ?
xfs_bulkstat_one_int+0x310/0x310 [xfs]