Null Pointer/kswapd oops with 2.4.18+XFS-1.1

[Poul Petersen <petersp@roguewave.com>]

	I've had a fairly long history of strange crashes and oops's over
the past year running XFS on our NFS server. Last Monday, I upgraded our NFS
server to kernel 2.4.18 with XFS 1.1. This evening, the kswapd oops'd -
which is actually an oops I have seen in the past. The other type of crash
we sometimes see is a hard lock, no oops and no NMI watchdog output. Any
ideas on what might be wrong or how to diagnose it? The system is a Dell
2550 dual P-III (1Ghz) running RedHat 7.2. The system is hooked up to a
Zzyzx RocketStor 2000 through a Qlogic 2200 Fibre channel card, if that has
any relevance. The XFS file-systems were xfs_repair'd last monday when I did
the upgrade. Here is the kysmoops output:

Thanks for any suggestions.

-poul

Unable to handle kernel NULL pointer dereference at virtual address 00000030
c019caf3
*pde = 00000000
Oops: 0000
CPU:    1
EIP:    0010:[xfs_alloc_lookup+259/832]    Not tainted
EIP:    0010:[<c019caf3>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010246
eax: 00000000   ebx: 00000000   ecx: cef91e4c   edx: 00000000
esi: f7b29c00   edi: db9eb000   ebp: 00000000   esp: f7ee3970
ds: 0018   es: 0018   ss: 0018
Process kswapd (pid: 5, stackpage=f7ee3000)
Stack: cef91e4c 00000000 00000000 dc68b000 c01b456e 00000033 f7b29c00
00000000 
       00000001 00000033 0000043a cef91f64 00000000 f7ee39d0 0005dabd
00000a54 
       0005dabd c0198e0e cef91e4c 00000000 f7ee39d0 f7ee39d0 00000000
00000000 
Call Trace: [xfs_btree_check_sblock+62/208] [xfs_alloc_fixup_trees+110/560]
[xfs_alloc_ag_vextent_near+912/2896] [xfs_alloc_ag_vextent+54/240]
[xfs_alloc_vextent+807/960] 
Call Trace: [<c01b456e>] [<c0198e0e>] [<c0199640>] [<c0199066>] [<c019b507>]

   [<c01a9234>] [<c027c3dc>] [<f881461a>] [<c01b3325>] [<c01b3325>]
[<c01acc26>] 
   [<c0247380>] [<c01f6e4b>] [<c011e08b>] [<c021d3a0>] [<c01f53c9>]
[<c01f17fb>] 
   [<c01f5370>] [<c01f0847>] [<c01f5370>] [<c01f55cc>] [<c01f5370>]
[<c0132dc7>] 
   [<c01330b2>] [<c013312c>] [<c01331d1>] [<c0133246>] [<c0133381>]
[<c01332e0>] 
   [<c0105000>] [<c0105876>] [<c01332e0>] 
Code: 8b 70 30 50 55 56 8b 54 24 54 52 e8 2d 7a 01 00 83 c4 1c 85 

>>EIP; c019caf3 <xfs_alloc_lookup+103/340>   <=====
Trace; c01b456e <xfs_btree_check_sblock+3e/d0>
Trace; c0198e0e <xfs_alloc_fixup_trees+6e/230>
Trace; c0199640 <xfs_alloc_ag_vextent_near+390/b50>
Trace; c0199066 <xfs_alloc_ag_vextent+36/f0>
Trace; c019b507 <xfs_alloc_vextent+327/3c0>
Trace; c01a9234 <xfs_bmap_alloc+1684/19e0>
Trace; c027c3dc <kfree_skbmem+c/70>
Trace; f881461a <[qla2200]qla2x00_next+da/174>
Trace; c01b3325 <xfs_bmbt_get_state+25/30>
Trace; c01b3325 <xfs_bmbt_get_state+25/30>
Trace; c01acc26 <xfs_bmapi+7f6/1210>
Trace; c0247380 <scsi_io_completion+1d0/480>
Trace; c01f6e4b <xfs_strategy+4fb/6e0>
Trace; c011e08b <do_softirq+7b/e0>
Trace; c021d3a0 <generic_make_request+110/120>
Trace; c01f53c9 <linvfs_pb_bmap+59/e0>
Trace; c01f17fb <pagebuf_delalloc_convert+5b/d0>
Trace; c01f5370 <linvfs_pb_bmap+0/e0>
Trace; c01f0847 <pagebuf_write_full_page+87/d0>
Trace; c01f5370 <linvfs_pb_bmap+0/e0>
Trace; c01f55cc <linvfs_write_full_page+4c/80>
Trace; c01f5370 <linvfs_pb_bmap+0/e0>
Trace; c0132dc7 <shrink_cache+237/3d0>
Trace; c01330b2 <shrink_caches+52/90>
Trace; c013312c <try_to_free_pages+3c/60>
Trace; c01331d1 <kswapd_balance_pgdat+51/a0>
Trace; c0133246 <kswapd_balance+26/40>
Trace; c0133381 <kswapd+a1/ba>
Trace; c01332e0 <kswapd+0/ba>
Trace; c0105000 <_stext+0/0>
Trace; c0105876 <kernel_thread+26/30>
Trace; c01332e0 <kswapd+0/ba>
Code;  c019caf3 <xfs_alloc_lookup+103/340>
00000000 <_EIP>:
Code;  c019caf3 <xfs_alloc_lookup+103/340>   <=====
   0:   8b 70 30                  mov    0x30(%eax),%esi   <=====
Code;  c019caf6 <xfs_alloc_lookup+106/340>
   3:   50                        push   %eax
Code;  c019caf7 <xfs_alloc_lookup+107/340>
   4:   55                        push   %ebp
Code;  c019caf8 <xfs_alloc_lookup+108/340>
   5:   56                        push   %esi
Code;  c019caf9 <xfs_alloc_lookup+109/340>
   6:   8b 54 24 54               mov    0x54(%esp,1),%edx
Code;  c019cafd <xfs_alloc_lookup+10d/340>
   a:   52                        push   %edx
Code;  c019cafe <xfs_alloc_lookup+10e/340>
   b:   e8 2d 7a 01 00            call   17a3d <_EIP+0x17a3d> c01b4530
<xfs_btree_check_sblock+0/d0>
Code;  c019cb03 <xfs_alloc_lookup+113/340>
  10:   83 c4 1c                  add    $0x1c,%esp
Code;  c019cb06 <xfs_alloc_lookup+116/340>
  13:   85 00                     test   %eax,(%eax)