Best XFS mailinglist,
We have been using XFS on semi-large servers the last 2.5 years [ 2 x 600 GB ]
With that in mind we have bought two large servers with general specs:
- 24 x 250 GB disks (leaving 4.5 TB netto when using RAID5+2hotspare)
- 2 x 3Ware hardware RAID controllers
- 6 GB ram
- Dual CPU on TYAN mainboard
- Debian 3 using 2.4.22 with XFS patches
Main purpose is serving NFS v3 and some Samba v3 to >300 users. The total
amount of
files exceeds 5 milion, most of them are below 1 MB. Loadavg on a normal
working day
varies between 0.5 <-> 2, which seems quite good & low to me.
The servers are running for 2 weeks now but last night one 1.1 TB partition
went down -xfs_force_shutdown(sd(8,48),0x8)-. I couldnt umount it so I had
to reboot the server. Not a single sign of hardware error messages from the
raid drivers.
The partitions are mkfs'ed using: mkfs -V -t xfs -f -d su=65536,sw=8 /dev/sdd
Logfiles and various info follows:
/proc/partitions
major minor #blocks name rio rmerge rsect ruse wio wmerge wsect wuse
running use aveq
8 0 976790016 sda 3097 15187 146001 155810 6292 94489 804001 265190 0
66830 421000
8 16 1220987520 sdb 33 76 649 180 3 62 513 70 0 250 250
8 32 976790016 sdc 274 76 2521 2450 4 63 529 40 0 2490 2490
8 48 1220987520 sdd 9456 174151 1468331 1372750 4402 42143 367990 505730 0
232020 1878480
/proc/interrupts
26: 357 13379 IO-APIC-level 3ware Storage Controller
27: 132 9000 IO-APIC-level 3ware Storage Controller
Elevators optimized for writing are:
/dev/sd[a-d]
elevator ID 8
read_latency: 32
write_latency: 8192
/proc/version
Linux version 2.4.22-xfs (root@kingfish) (gcc version 3.3.2 (Debian)) #12 SMP
Thu Nov 27 17:34:34 CET 2003
/var/log/messages
Dec 12 01:54:58 kingfish -- MARK --
Dec 12 01:56:22 kingfish kernel: f6bd7c68 f6bd7cc4 c01bbb60 c03259af 00000001
00000000 c03259a3 00000662
Dec 12 01:56:22 kingfish kernel: c01bcbfa 00000000 00000001 f707cc00
f4aaeeac 000a8d43 0000000d 00000001
Dec 12 01:56:22 kingfish kernel: 000a8ce0 00000030 00000000 00000001
00000000 ee73dd10 00000000 f6bd7d3c
Dec 12 01:56:22 kingfish kernel: Call Trace:
Dec 12 01:56:22 kingfish kernel: [xfs_free_ag_extent+1056/1808]
xfs_free_ag_extent+0x420/0x710 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_free_extent+186/224]
xfs_free_extent+0xba/0xe0 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_free_extent+186/224]
xfs_free_extent+0xba/0xe0 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_efd_init+124/126] xfs_efd_init+0x7c/0x7e
[kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_trans_get_efd+54/80]
xfs_trans_get_efd+0x36/0x50 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_bmap_finish+306/448]
xfs_bmap_finish+0x132/0x1c0 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_itruncate_finish+513/1008]
xfs_itruncate_finish+0x201/0x3f0 [kernel]
Dec 12 01:56:22 kingfish kernel: [xfs_inactive+1270/1360]
xfs_inactive+0x4f6/0x550 [kernel]
Dec 12 01:56:22 kingfish kernel: [vn_rele+167/176] vn_rele+0xa7/0xb0 [kernel]
Dec 12 01:56:22 kingfish kernel: [linvfs_clear_inode+25/48]
linvfs_clear_inode+0x19/0x30 [kernel]
Dec 12 01:56:22 kingfish kernel: [clear_inode+261/320] clear_inode+0x105/0x140
[kernel]
Dec 12 01:56:22 kingfish kernel: [iput+214/736] iput+0xd6/0x2e0 [kernel]
Dec 12 01:56:22 kingfish kernel: [vfs_unlink+395/640] vfs_unlink+0x18b/0x280
[kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd_unlink+285/576] nfsd_unlink+0x11d/0x240
[kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd3_proc_remove+132/272]
nfsd3_proc_remove+0x84/0x110 [kernel]
Dec 12 01:56:22 kingfish kernel: [nfs3svc_decode_diropargs+134/240]
nfs3svc_decode_diropargs+0x86/0xf0 [kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd_dispatch+204/485]
nfsd_dispatch+0xcc/0x1e5 [kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd_dispatch+0/485] nfsd_dispatch+0x0/0x1e5
[kernel]
Dec 12 01:56:22 kingfish kernel: [svc_process+853/1360]
svc_process+0x355/0x600 [kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd+495/832] nfsd+0x1ef/0x340 [kernel]
Dec 12 01:56:22 kingfish kernel: [arch_kernel_thread+46/64]
arch_kernel_thread+0x2e/0x40 [kernel]
Dec 12 01:56:22 kingfish kernel: [nfsd+0/832] nfsd+0x0/0x340 [kernel]
Dec 12 01:56:22 kingfish kernel: xfs_force_shutdown(sd(8,48),0x8) called from
line 4049 of file xfs_bmap.c. Return address = 0xc0223ad8
The recovery upon reboot:
XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1634 of file xfs_alloc.c.
Caller 0xc01bcbfa
f6af1bac f6af1c08 c01bbb60 c03259af 00000001 00000000 c03259a3 00000662
c01bcbfa 00000000 00000001 f62b1c00 f689fe8c 000a8d43 0000000d 00000001
000a8ce0 00000030 00000000 00000001 00000000 f68abc50 f68b4e98 f6af1c80
Call Trace:
[<c01bbb60>] xfs_free_ag_extent+0x420/0x710 [kernel]
[<c01bcbfa>] xfs_free_extent+0xba/0xe0 [kernel]
[<c01bcbfa>] xfs_free_extent+0xba/0xe0 [kernel]
[<c01eae9c>] xfs_efd_init+0x7c/0x7e [kernel]
[<c020b066>] xfs_trans_get_efd+0x36/0x50 [kernel]
[<c0201e96>] xlog_recover_process_efi+0x186/0x1f0 [kernel]
[<c0201f77>] xlog_recover_process_efis+0x77/0x90 [kernel]
[<c01f1913>] xfs_iget+0x143/0x1a0 [kernel]
[<c02035cd>] xlog_recover_finish+0x1d/0x26a [kernel]
[<c01fa64d>] xfs_log_mount_finish+0x2d/0x30 [kernel]
[<c0204dec>] xfs_mountfs+0x6ac/0xd20 [kernel]
[<c01f7e20>] xfs_ioinit+0x20/0x40 [kernel]
[<c020cb5d>] xfs_mount+0x27d/0x420 [kernel]
[<c02236f1>] vfs_mount+0x41/0x50 [kernel]
[<c022322f>] linvfs_read_super+0x8f/0x270 [kernel]
[<c0147b4d>] alloc_super+0x3d/0x1b0 [kernel]
[<c014856e>] get_sb_bdev+0x18e/0x290 [kernel]
[<c015ca98>] alloc_vfsmnt+0x98/0xe0 [kernel]
[<c01488ff>] do_kern_mount+0x11f/0x130 [kernel]
[<c015dd94>] do_add_mount+0x74/0x160 [kernel]
[<c014e79a>] path_lookup+0x3a/0x40 [kernel]
[<c015e060>] do_mount+0x130/0x180 [kernel]
[<c015dedd>] copy_mount_options+0x5d/0xb0 [kernel]
[<c015e526>] sys_mount+0xb6/0x120 [kernel]
[<c01076df>] system_call+0x33/0x38 [kernel]
Ending XFS recovery on filesystem: sd(8,48) (dev: 8/48)
Best regards,
Leroy van Logchem
The Netherlands
WL | delft hydraulics
|