xfs
[Top] [All Lists]

[Bug 321] New: XFS internal error, xfs_force_shutdown on 1.2TB fileserve

To: xfs-master@xxxxxxxxxxx
Subject: [Bug 321] New: XFS internal error, xfs_force_shutdown on 1.2TB fileserver.
From: bugzilla-daemon@xxxxxxxxxxx
Date: Thu, 25 Mar 2004 02:50:31 -0800
Sender: linux-xfs-bounce@xxxxxxxxxxx
http://oss.sgi.com/bugzilla/show_bug.cgi?id=321

           Summary: XFS internal error, xfs_force_shutdown on 1.2TB
                    fileserver.
           Product: Linux XFS
           Version: 1.3.x
          Platform: All
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: High
         Component: XFS kernel code
        AssignedTo: xfs-master@xxxxxxxxxxx
        ReportedBy: ericy@xxxxxxxxxxxx
                CC: ericy@xxxxxxxxxxxx


I have a few 1.2TB (6*200GB) fileserver running RH7.3 with kernel 2.4.25 running
3ware 7506-8 RAID 1 * 3. The fileservers mainly contains users maildir with lots
of subdirectories. Unfortunately I have been suffered from continuous filesystem
corruption, following are the error messages from log:

Mar 22 02:27:48 fs5-12 kernel: XFS internal error XFS_WANT_CORRUPTED_RETURN at
line 295 of file xfs_alloc.c.  Caller 0xc01a04ed
Mar 22 02:27:49 fs5-12 kernel: ccda9944 c019f70d c02f1cca 00000001 00000000
c02f1cbe 00000127 c01a04ed 
Mar 22 02:27:49 fs5-12 kernel:        00000000 00000000 00000000 00000001
dad40f70 dad40eec 000039d3 c01a04ed 
Mar 22 02:27:49 fs5-12 kernel:        dad40f70 dad40eec 000039d3 00000007
000039d9 00000001 00000001 000041b4 
Mar 22 02:27:49 fs5-12 kernel: Call Trace:    [<c019f70d>] [<c01a04ed>]
[<c01a04ed>] [<c019f7e6>] [<c01a1dfa>]
Mar 22 02:27:49 fs5-12 kernel:   [<c01afb51>] [<c01b15d8>] [<c01b2a6d>]
[<c01b2e5d>] [<c022a08f>] [<c01c0e41>]
Mar 22 02:27:49 fs5-12 kernel:   [<c029e560>] [<c029e56b>] [<c01c2a34>]
[<c014eb72>] [<c01dee0c>] [<c01c8a36>]
Mar 22 02:27:49 fs5-12 kernel:   [<c01c07ae>] [<c0201dc9>] [<c0201ddd>]
[<c01f2bc4>] [<c01fb30d>] [<c014538f>]
Mar 22 02:27:49 fs5-12 kernel:   [<c0143fb9>] [<c018dba6>] [<c02d7c8f>]
[<c0192ac4>] [<c0188f27>] [<c02d7008>]
Mar 22 02:27:49 fs5-12 kernel:   [<c0188d2b>] [<c0188b30>] [<c0105696>] 
[<c0188b30>]
Mar 22 02:27:49 fs5-12 kernel: xfs_force_shutdown(sd(8,7),0x8) called from line
1070 of file xfs_trans.c.  Return address = 0xc01fe096
Mar 22 02:27:49 fs5-12 kernel: Filesystem "sd(8,7)": Corruption of in-memory
data detected.  Shutting down filesystem: sd(8,7)
Mar 22 02:27:49 fs5-12 kernel: Please umount the filesystem, and rectify the
problem(s)
Mar 22 02:27:49 fs5-12 kernel: nfsd: non-standard errno: -990
Mar 22 02:36:42 fs5-12 kernel: nfsd: last server has exited
Mar 22 02:36:42 fs5-12 kernel: nfsd: unexporting all filesystems
Mar 22 02:36:42 fs5-12 kernel: rpciod: active tasks at shutdown?!
Mar 22 02:37:40 fs5-12 kernel: XFS mounting filesystem sd(8,7)
Mar 22 02:37:41 fs5-12 kernel: Starting XFS recovery on filesystem: sd(8,7)
(dev: sd(8,7))
Mar 22 02:37:45 fs5-12 kernel: Ending XFS recovery on filesystem: sd(8,7) (dev:
sd(8,7))
Mar 22 03:03:54 fs5-12 kernel: 0x0: 6c 61 6e 67 75 61 67 65 3d 30 2c 75 73 0a 
00 00 
Mar 22 03:03:54 fs5-12 kernel: Filesystem "sd(8,7)": XFS internal error
xfs_da_do_buf(2) at line 2272 of file xfs_da_btree.c.  Caller 0xc01bea5e
Mar 22 03:03:54 fs5-12 kernel: dd3adbb4 c01be8cb c02f2026 00000001 d64e5800
c02f1f2d 000008e0 c01bea5e 
Mar 22 03:03:54 fs5-12 kernel:        c01bea5e ffffffff 0000000f 00000018
00000000 d64e5800 dd3adc04 00000001 
Mar 22 03:03:54 fs5-12 kernel:        00000000 d64e5800 05f6a598 00000000
00000000 00000000 d2676e80 00000001 
Mar 22 03:03:54 fs5-12 kernel: Call Trace:    [<c01be8cb>] [<c01bea5e>]
[<c01bea5e>] [<c01dd591>] [<c01bea5e>]
Mar 22 03:03:54 fs5-12 kernel:   [<c01c21dd>] [<c01c21dd>] [<c01c210b>]
[<c01c08d9>] [<c014ed01>] [<c01ed04c>]
Mar 22 03:03:54 fs5-12 kernel:   [<c01f1c7e>] [<c01f0c3f>] [<c01fb27f>]
[<c0142eb0>] [<c0143f21>] [<c0143fb9>]
Mar 22 03:03:54 fs5-12 kernel:   [<c018bf60>] [<c02c07e5>] [<c0282f9c>]
[<c02d73a4>] [<c02d7c8f>] [<c0191cf8>]
Mar 22 03:03:54 fs5-12 kernel:   [<c0188f27>] [<c02d7008>] [<c0188d2b>]
[<c0188b30>] [<c0105696>] [<c0188b30>]
Mar 22 03:03:54 fs5-12 kernel: nfsd: non-standard errno: -990

[root@fileserver root]# lspci
00:00.0 Host bridge: Intel Corp.: Unknown device 254c (rev 01)
00:00.1 Class ff00: Intel Corp. e7500 [Plumas] DRAM Controller Error Reporting
(rev 01)
00:02.0 PCI bridge: Intel Corp. e7500 [Plumas] HI_B Virtual PCI Bridge (F0) 
(rev 01)
00:02.1 Class ff00: Intel Corp. e7500 [Plumas] HI_B Virtual PCI Bridge (F1) 
(rev 01)
00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev 42)
00:1f.0 ISA bridge: Intel Corp. 82801CA ISA Bridge (LPC) (rev 02)
00:1f.1 IDE interface: Intel Corp. 82801CA IDE U100 (rev 02)
00:1f.3 SMBus: Intel Corp. 82801CA/CAM SMBus (rev 02)
01:1c.0 PIC: Intel Corp. 82870P2 P64H2 I/OxAPIC (rev 04)
01:1d.0 PCI bridge: Intel Corp. 82870P2 P64H2 Hub PCI Bridge (rev 04)
01:1e.0 PIC: Intel Corp. 82870P2 P64H2 I/OxAPIC (rev 04)
01:1f.0 PCI bridge: Intel Corp. 82870P2 P64H2 Hub PCI Bridge (rev 04)
02:01.0 Ethernet controller: Intel Corp. 82545EM Gigabit Ethernet Controller
(rev 01)
03:01.0 RAID bus controller: 3ware Inc 3ware 7000-series ATA-RAID (rev 01)
04:03.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
04:04.0 Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 0d)
04:05.0 Ethernet controller: Intel Corp. 82540EM Gigabit Ethernet Controller
(rev 02)

I tried to run xfs_repair to clean the filesystem, unfortunately it couldn't 
finish:

Phase 6 - check inode connectivity...
        - resetting contents of realtime bitmap and summary inodes
        - ensuring existence of lost+found directory
        - traversing filesystem starting at / ...
rebuilding directory inode 128
rebuilding directory inode 806216170
rebuilding directory inode 805641623
rebuilding directory inode 253488352
rebuilding directory inode 1782429002

fatal error -- couldn't map inode 1208851203, err = 22

After xfs_repair failed each time, I can remount it and get it run for few days
then it will failed again.

[root@fileserver root]# xfs_info /dev/sda6
meta-data=/filesystem1           isize=256    agcount=16, agsize=4457911 blks
         =                       sectsz=512
data     =                       bsize=4096   blocks=71326576, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal               bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks
realtime =none                   extsz=65536  blocks=0, rtextents=0

I saw quite alot of post on XFS internal error on different .c files, but
doesn't seem to have a fix for all yet. Please let me know for any information
required.

Regards,
Eric Yu
Outblaze Ltd



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.


<Prev in Thread] Current Thread [Next in Thread>
  • [Bug 321] New: XFS internal error, xfs_force_shutdown on 1.2TB fileserver., bugzilla-daemon <=