"nathans@xxxxxxxxxxxx" wrote:
Yes this could very well be the same problem... first very high level
glance I would say you loaded the system down till kmalloc failed.
We don't do a very good job of checking to see if the allocation fails,
if it does it would leave the pb_addr field with a null pointer.
I've seen the slab magic error before, mostly when writing into areas of memory
that weren't mine at the time. I wonder if ext2 also has some merry little code
paths that don't check for failed kmallocs.
The may also be the case where the linux vfs has already free'ed an inode and
is trying to do is again. Ted's changes to the inode/vnode counts should fix
that case.
> View Incident:
> http://co-op.engr.sgi.com/BugWorks/code/bwxquery.cgi?search=Search&wlong=1&view_type=Bug&wi=791621
>
> Submitter : nathans Submitter Domain : engr
> Assigned Engineer : mostek Assigned Domain : sgi.com
> Assigned Group : xfs-linux Category : software
> Customer Reported : F Priority : 1
> Project : xfs-linux Status : open
> Description :
> I've seen this panic in the XFS code a few times now, in both
> big-endian and little-endian builds (just before Daniel's
> BE-only checkin). Only happens under very high load (test is
> using sendmail to queue-remote/save-local mail messages to an
> xfs fs), and only happens sporadically (increased load seems
> to increase the chance of it happening).
>
> kernel is built with debug on, pagebuf meta on.
>
> Unable to handle kernel NULL pointer dereference at virtual address 00000060
> printing eip:
> c01c5c73
> *pde = 00000000
> Entering kdb (0xc2c6a000) on processor 0 Panic: Oops
> due to panic @ 0xc01c5c73
> eax = 0xc551dd40 ebx = 0x00000000 ecx = 0x00000008 edx = 0x00000000
> esi = 0x00000000 edi = 0x00000020 esp = <kernel> eip = 0xc01c5c73
> ebp = 0xc2c6be10 ss = 0xc37727e4 cs = 0x00000010 eflags = 0x00010246
> ds = <kernel> es = 0x00000018 origeax = 0xffffffff ®s = 0xc2c6bdd0
> [0]kdb> bt
> EBP EIP Function(args)
> 0xc2c6be10 0xc01c5c73 xfs_inobp_check+0x33( 0xc4a9d000, 0xc551dd40,
> 0xc37727e4, 0xc551d480, 0x1560 )
> 0xc2c6be78 0xc01c9395 xfs_iunlink_remove+0x5c9( 0xc37727e4, 0xc22413c0,
> 0xc4a9d000, 0xc22413c0, 0xc251f3b4 )
> 0xc2c6be94 0xc01c9518 xfs_ifree+0x174( 0xc37727e4, 0xc22413c0, 0xc0394400,
> 0x0, 0xc251f3b4 )
> 0xc2c6bec0 0xc01eef8c xfs_inactive+0x470( 0xc22413d8, 0x0, 0xc251f3b4,
> 0xc0394740, 0xc28e59c0 )
> 0xc2c6bef0 0xc0205241 vn_rele+0x13d( 0xc251f3b4, 0xc1f73100, 0xc2c6bf1c,
> 0xc015f6b8 )
> 0xc2c6bf00 0xc02019c7 linvfs_put_inode+0x17( 0xc1f73100, 0xc28e59c0,
> 0xc1f73100, 0xc28e59c0 )
> 0xc2c6bf1c 0xc015f6b8 iput+0x38( 0xc1f73100, 0xc2f9bd80, 0xc1f73100,
> 0xc2c6bf48 )
> 0xc2c6bf30 0xc015ce67 dput+0xa7( 0xc28e59c0, 0xc2c6a000, 0xc2f9bd80,
> 0xc2c6a000 )
> 0xc2c6bf48 0xc014c330 __fput+0x40( 0xc2f9bd80, 0xc2f9bd80, 0xc2f9bd80,
> 0xc28e59c0, 0x0 )
> 0xc2c6bf64 0xc014c3ab _fput+0x6f( 0xc2f9bd80, 0xc24ac8e4, 0xc2c6a000,
> 0xc2f9bd80 )
> 0xc2c6bf7c 0xc014ac1c filp_close+0x64( 0xc2f9bd80, 0xc24ac8e0, 0xc2c6a000,
> 0x80f49f0, 0x80aa7a0 )
> 0xc2c6bfac 0xc014ad8b do_close+0x163( 0x3, 0x1, 0xbfffd650, 0xc010a538, 0x3 )
> 0xc2c6bfbc 0xc014ae3a sys_close+0xe( 0x3, 0x0, 0x4016848c, 0x80f49f0,
> 0x80aa7a0 )
> 0xc2c6bfd8 0xc010a538 system_call+0x34
> [0]kdb> id xfs_inobp_check
> xfs_inobp_check: pushl %ebp
> xfs_inobp_check+0x1: movl %esp,%ebp
> xfs_inobp_check+0x3: pushl %edi
> xfs_inobp_check+0x4: pushl %esi
> xfs_inobp_check+0x5: pushl %ebx
> xfs_inobp_check+0x6: movl 0x8(%ebp),%eax
> xfs_inobp_check+0x9: xorl %esi,%esi
> xfs_inobp_check+0xb: movzwl 0x1aa(%eax),%edi
> xfs_inobp_check+0x12: movzbl 0xa2(%eax),%ecx
> xfs_inobp_check+0x19: sarl %cl,%edi
> xfs_inobp_check+0x1b: cmpl %edi,%esi
> xfs_inobp_check+0x1d: jnl xfs_inobp_check+0x78
> xfs_inobp_check+0x1f: nop
> xfs_inobp_check+0x20: movl 0x8(%ebp),%eax
> xfs_inobp_check+0x23: movzwl 0x90(%eax),%ebx
> xfs_inobp_check+0x2a: movl 0xc(%ebp),%eax
> [0]kdb> id
> xfs_inobp_check+0x2d: imull %esi,%ebx
> xfs_inobp_check+0x30: addl 0x28(%eax),%ebx
> xfs_inobp_check+0x33: cmpl $0x0,0x60(%ebx)
> ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^
> xfs_inobp_check+0x37: jne xfs_inobp_check+0x73
> xfs_inobp_check+0x39: pushl %eax
> xfs_inobp_check+0x3a: pushl $0xc02d3a40
> xfs_inobp_check+0x3f: movl 0x8(%ebp),%eax
> xfs_inobp_check+0x42: pushl %eax
> xfs_inobp_check+0x43: pushl $0x1
> xfs_inobp_check+0x45: call xfs_fs_cmn_err
> xfs_inobp_check+0x4a: addl $0x10,%esp
> xfs_inobp_check+0x4d: cmpl $0x0,0xc03946c8
> xfs_inobp_check+0x54: je xfs_inobp_check+0x73
> xfs_inobp_check+0x56: cmpl $0x0,0x60(%ebx)
> xfs_inobp_check+0x5a: jne xfs_inobp_check+0x73
> xfs_inobp_check+0x5c: pushl $0xd6
>
> void
> xfs_inobp_check(
> xfs_mount_t *mp,
> xfs_buf_t *bp)
> {
> int i;
> int j;
> xfs_dinode_t *dip;
>
> j = mp->m_inode_cluster_size >> mp->m_sb.sb_inodelog;
>
> for (i = 0; i < j; i++) {
> dip = (xfs_dinode_t *)((char *)XFS_BUF_PTR(bp) +
> (i * mp->m_sb.sb_inodesize));
> if (INT_ISZERO(dip->di_next_unlinked, ARCH_UNKNOWN)) {
> ^^^^^^^^^^^^^^^^^^^^^
>
> adding a conditional printk in the while loop (before the "if")
> reveals that the panic happens when i==0, mp->m_sb.sb_inodesize==256,
> bp=0xc3f37800 and dip==0x00000000 (urk).
>
> and with CONFIG_PAGE_BUF_META defined, we compile with
> #define XFS_BUF_PTR(bp) (caddr_t)((bp)->pb_addr)
>
> so, looks like a pagebuf bug.
>
> and on a possibly-related note, the same kernel also panics
> when running the test on ext2, but in that case I get a bunch
> of "Bad slab magic" errors before it falls over (also on the
> close() syscall path)...
>
> troppo.melbourne.sgi.com login: kmem_alloc: Bad slab magic (corrupt)
> (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> kmem_alloc: Bad slab magic (corrupt) (name=inode_cache)
> Unable to handle kernel paging request at virtual address 08069e58
> printing eip:
> c0135305
> *pde = 013bf067
> *pte = 00000000
> Entering kdb (0xc2ad0000) on processor 0 Panic: Oops
> due to panic @ 0xc0135305
> eax = 0x08069e48 ebx = 0xc4553ee0 ecx = 0xc41a1fe0 edx = 0x08069e48
> esi = 0xc11e5be0 edi = 0xc11e5c00 esp = <kernel> eip = 0xc0135305
> ebp = 0xc2ad1efc ss = 0xc0166624 cs = 0x00000010 eflags = 0x00010046
> ds = <kernel> es = 0x00000018 origeax = 0xffffffff ®s = 0xc2ad1eb0
> [0]kdb> bt
> EBP EIP Function(args)
> 0xc2ad1efc 0xc0135305 kmem_cache_free+0x159( 0xc11e5be0, 0xc41a1e40,
> 0xc5d794e0, 0xc41a1e40, 0xc5d794e0 )
> 0xc2ad1f1c 0xc015fa5b iput+0x3db( 0xc41a1e40, 0xc1cd0a80, 0xc41a1e40,
> 0xc2ad1f48 )
> 0xc2ad1f30 0xc015ce67 dput+0xa7( 0xc5d794e0, 0xc2ad0000, 0xc1cd0a80,
> 0xc2ad0000 )
> 0xc2ad1f48 0xc014c330 __fput+0x40( 0xc1cd0a80, 0xc1cd0a80, 0xc1cd0a80,
> 0xc5d794e0, 0x0 )
> 0xc2ad1f64 0xc014c3ab _fput+0x6f( 0xc1cd0a80, 0xc209f004, 0xc2ad0000,
> 0xc1cd0a80 )
> 0xc2ad1f7c 0xc014ac1c filp_close+0x64( 0xc1cd0a80, 0xc209f000, 0xc2ad0000,
> 0x40166ba0, 0x808905c )
> 0xc2ad1fac 0xc014ad8b do_close+0x163( 0x0, 0x1, 0xbfffe7e0, 0xc010a538, 0x0 )
> 0xc2ad1fbc 0xc014ae3a sys_close+0xe( 0x0, 0x0, 0x4016848c, 0x40166ba0,
> 0x808905c )
> 0xc2ad1fd8 0xc010a538 system_call+0x34
> [0]kdb> ps
> Task Addr Pid Parent [*] cpu State Thread Command
> 0xc11fa000 00000001 00000000 0 000 stop 0xc11fa344 init
> 0xc5fc4000 00000002 00000001 0 000 stop 0xc5fc4344 kswapd
> 0xc5fc2000 00000003 00000001 0 000 stop 0xc5fc2344 kflushd
> 0xc5fc0000 00000004 00000001 0 000 stop 0xc5fc0344 kupdate
> 0xc5d0e000 00000400 00000001 0 000 stop 0xc5d0e344 portmap
> 0xc4450000 00000440 00000001 0 000 stop 0xc4450344 rpciod
> 0xc44a6000 00000441 00000001 0 000 stop 0xc44a6344 lockd
> 0xc4722000 00000460 00000001 0 000 stop 0xc4722344 syslogd
> 0xc45b0000 00000471 00000001 0 000 stop 0xc45b0344 klogd
> 0xc4684000 00000487 00000001 0 000 stop 0xc4684344 atd
> 0xc4bcc000 00000503 00000001 0 000 stop 0xc4bcc344 crond
> 0xc5d1a000 00000523 00000001 0 000 stop 0xc5d1a344 inetd
> 0xc431c000 00000558 00000001 0 000 stop 0xc431c344 rpc.rquotad
> 0xc46ae000 00000569 00000001 0 000 stop 0xc46ae344 rpc.mountd
> 0xc470c000 00000580 00000001 0 000 stop 0xc470c344 nfsd
> 0xc46f2000 00000581 00000001 0 000 stop 0xc46f2344 nfsd
> 0xc476c000 00000582 00000001 0 000 stop 0xc476c344 nfsd
> 0xc47aa000 00000583 00000001 0 000 stop 0xc47aa344 nfsd
> 0xc47a8000 00000584 00000001 0 000 stop 0xc47a8344 nfsd
> 0xc47b6000 00000585 00000001 0 000 stop 0xc47b6344 nfsd
> 0xc47b0000 00000586 00000001 0 000 stop 0xc47b0344 nfsd
> 0xc47d6000 00000587 00000001 0 000 stop 0xc47d6344 nfsd
> [0]more>
> 0xc4e34000 00000652 00000001 0 000 stop 0xc4e34344 pmcd
> 0xc0580000 00000664 00000652 0 000 stop 0xc0580344 pmdasimple
> 0xc4568000 00000672 00000652 0 000 stop 0xc4568344 pmdatrace
> 0xc5e74000 00000837 00000001 0 000 stop 0xc5e74344 pmlogger
> 0xc0556000 00000839 00000001 0 000 stop 0xc0556344 nsrexecd
> 0xc437e000 00000859 00000001 0 000 stop 0xc437e344 mingetty
> 0xc4a6e000 00000860 00000001 0 000 stop 0xc4a6e344 mingetty
> 0xc4a6c000 00000861 00000001 0 000 stop 0xc4a6c344 mingetty
> 0xc4a6a000 00000862 00000001 0 000 stop 0xc4a6a344 mingetty
> 0xc4a66000 00000863 00000001 0 000 stop 0xc4a66344 mingetty
> 0xc0546000 00000864 00000001 0 000 stop 0xc0546344 mingetty
> 0xc0fba000 00000865 00000001 0 000 stop 0xc0fba344 getty
> 0xc480a000 00000907 00000523 0 000 stop 0xc480a344 in.rlogind
> 0xc419a000 00000908 00000907 0 000 stop 0xc419a344 login
> 0xc4160000 00000909 00000908 0 000 stop 0xc4160344 tcsh
> 0xc4102000 00000941 00000909 0 000 stop 0xc4102344 sh
> 0xc5b5c000 00008851 00000941 0 000 stop 0xc5b5c344 sudo
> 0xc33ce000 00008852 00008851 0 000 stop 0xc33ce344 run_sendmail
> 0xc43cc000 00008870 00008852 0 000 stop 0xc43cc344 lpbm
> 0xc1d98000 00009055 00008870 0 000 stop 0xc1d98344 time
> 0xc481e000 00009065 00008870 0 000 stop 0xc481e344 time
> 0xc2dae000 00009105 00008870 0 000 stop 0xc2dae344 time
> 0xc0b5a000 00009107 00009105 0 000 stop 0xc0b5a344 lpbm_user_run
> [0]more>
> 0xc0fe4000 00009286 00000001 0 000 stop 0xc0fe4344 sendmail
> 0xc3b2c000 00009292 00000001 0 000 stop 0xc3b2c344 sendmail
> 0xc2da6000 00009294 00000001 0 000 stop 0xc2da6344 sendmail
> 0xc3a88000 00009299 00000001 0 000 stop 0xc3a88344 sendmail
> 0xc2d90000 00009317 00000001 0 000 stop 0xc2d90344 sendmail
> 0xc0086000 00009320 00000001 0 000 stop 0xc0086344 sendmail
> 0xc05b2000 00009326 00000001 0 000 stop 0xc05b2344 sendmail
> 0xc1cfa000 00009328 00000001 0 000 stop 0xc1cfa344 sendmail
> 0xc5716000 00009333 00000001 0 000 stop 0xc5716344 sendmail
> 0xc5092000 00009335 00000001 0 000 stop 0xc5092344 sendmail
> 0xc5d4e000 00009339 00000001 0 000 stop 0xc5d4e344 sendmail
> 0xc527e000 00009341 00000001 0 000 stop 0xc527e344 sendmail
> 0xc2c10000 00009347 00000001 0 000 stop 0xc2c10344 sendmail
> 0xc055c000 00009349 00000001 0 000 stop 0xc055c344 sendmail
> 0xc0ab8000 00009352 00000001 0 000 stop 0xc0ab8344 sendmail
> 0xc4ce6000 00009353 00000001 0 000 stop 0xc4ce6344 sendmail
> 0xc3dcc000 00009355 00000001 0 000 stop 0xc3dcc344 sendmail
> 0xc5662000 00009371 00000001 0 000 stop 0xc5662344 sendmail
> 0xc2460000 00009375 00000001 0 000 stop 0xc2460344 sendmail
> 0xc27dc000 00009379 00000001 0 000 stop 0xc27dc344 sendmail
> 0xc0a88000 00009380 00000001 0 000 stop 0xc0a88344 sendmail
> 0xc1292000 00009390 00000001 0 000 stop 0xc1292344 sendmail
> 0xc51a6000 00009394 00000001 0 000 stop 0xc51a6344 sendmail
> [0]more>
> 0xc3574000 00009396 00000001 0 000 stop 0xc3574344 sendmail
> 0xc165c000 00009398 00000001 0 000 stop 0xc165c344 sendmail
> 0xc1512000 00009400 00000001 0 000 stop 0xc1512344 sendmail
> 0xc2296000 00009402 00000001 0 000 stop 0xc2296344 sendmail
> 0xc208c000 00009404 00000001 0 000 stop 0xc208c344 sendmail
> 0xc3eb8000 00009406 00000001 0 000 stop 0xc3eb8344 sendmail
> 0xc2be6000 00009408 00000001 0 000 stop 0xc2be6344 sendmail
> 0xc4144000 00009410 00000001 0 000 stop 0xc4144344 sendmail
> 0xc5c80000 00009412 00000001 0 000 stop 0xc5c80344 sendmail
> 0xc254c000 00009414 00000001 0 000 stop 0xc254c344 sendmail
> 0xc13ba000 00009418 00000001 0 000 stop 0xc13ba344 sendmail
> 0xc18ea000 00009419 00000001 0 000 stop 0xc18ea344 sendmail
> 0xc370a000 00009438 00000001 0 000 stop 0xc370a344 sendmail
> 0xc42c2000 00009440 00000001 0 000 stop 0xc42c2344 sendmail
> 0xc42a6000 00009442 00000001 0 000 stop 0xc42a6344 sendmail
> 0xc0a1a000 00009444 00000001 0 000 stop 0xc0a1a344 sendmail
> 0xc0c7c000 00009446 00000001 0 000 stop 0xc0c7c344 sendmail
> 0xc56e0000 00009448 00000001 0 000 stop 0xc56e0344 sendmail
> 0xc3672000 00009450 00000001 0 000 stop 0xc3672344 sendmail
> 0xc58d8000 00009451 00000001 0 000 stop 0xc58d8344 sendmail
> 0xc5cf4000 00009455 00000001 0 000 stop 0xc5cf4344 sendmail
> 0xc42c6000 00009457 00000001 0 000 stop 0xc42c6344 sendmail
> 0xc4fac000 00009458 00000001 0 000 stop 0xc4fac344 sendmail
> [0]more>
> 0xc562c000 00009460 00000001 0 000 stop 0xc562c344 sendmail
> 0xc3fc8000 00009463 00000001 0 000 stop 0xc3fc8344 sendmail
> 0xc2656000 00009465 00000001 0 000 stop 0xc2656344 sendmail
> 0xc5dec000 00009466 00000001 0 000 stop 0xc5dec344 sendmail
> 0xc14a8000 00009469 00000001 0 000 stop 0xc14a8344 sendmail
> 0xc294c000 00009471 00000001 0 000 stop 0xc294c344 sendmail
> 0xc4c0e000 00009473 00000001 0 000 stop 0xc4c0e344 sendmail
> 0xc453c000 00009474 00000001 0 000 stop 0xc453c344 sendmail
> 0xc486a000 00009476 00000001 0 000 stop 0xc486a344 sendmail
> 0xc163e000 00009477 00000001 0 000 stop 0xc163e344 sendmail
> 0xc42e6000 00009479 00000001 0 000 stop 0xc42e6344 sendmail
> 0xc3188000 00009481 00000001 0 000 stop 0xc3188344 sendmail
> 0xc5080000 00009484 00000001 0 000 stop 0xc5080344 sendmail
> 0xc4946000 00009486 00000001 0 000 stop 0xc4946344 sendmail
> 0xc1d5c000 00009487 00000001 0 000 stop 0xc1d5c344 sendmail
> 0xc5d04000 00009490 00000001 0 000 stop 0xc5d04344 sendmail
> 0xc45de000 00009491 00000001 0 000 stop 0xc45de344 sendmail
> 0xc5754000 00009493 00000001 0 000 stop 0xc5754344 sendmail
> 0xc5140000 00009495 00000001 0 000 stop 0xc5140344 sendmail
> 0xc147c000 00009496 00000001 0 000 stop 0xc147c344 sendmail
> 0xc3354000 00009497 00000001 0 000 stop 0xc3354344 sendmail
> 0xc1c1a000 00009499 00000001 0 000 stop 0xc1c1a344 sendmail
> 0xc1b42000 00009504 00000001 0 000 stop 0xc1b42344 sendmail
> [0]more>
> 0xc56de000 00009505 00000001 0 000 stop 0xc56de344 sendmail
> 0xc5762000 00009506 00000001 0 000 stop 0xc5762344 sendmail
> 0xc1cca000 00009508 00000001 0 000 stop 0xc1cca344 sendmail
> 0xc58e8000 00009510 00000001 0 000 stop 0xc58e8344 sendmail
> 0xc47b8000 00009511 00000001 0 000 stop 0xc47b8344 sendmail
> 0xc24d6000 00009514 00000001 0 000 stop 0xc24d6344 sendmail
> 0xc2732000 00009517 00000001 0 000 stop 0xc2732344 sendmail
> 0xc379c000 00009519 00000001 0 000 stop 0xc379c344 sendmail
> 0xc064e000 00009522 00000001 0 000 stop 0xc064e344 sendmail
> 0xc2bb2000 00009523 00000001 0 000 stop 0xc2bb2344 sendmail
> 0xc32bc000 00009526 00000001 0 000 stop 0xc32bc344 sendmail
> 0xc40cc000 00009529 00000001 0 000 stop 0xc40cc344 sendmail
> 0xc12c0000 00009531 00000001 0 000 stop 0xc12c0344 sendmail
> 0xc5270000 00009533 00000001 0 000 stop 0xc5270344 sendmail
> 0xc1c70000 00009534 00000001 0 000 stop 0xc1c70344 sendmail
> 0xc14a0000 00009537 00000001 0 000 stop 0xc14a0344 sendmail
> 0xc1cb2000 00009539 00000001 0 000 stop 0xc1cb2344 sendmail
> 0xc523a000 00009540 00000001 0 000 stop 0xc523a344 sendmail
> 0xc16ea000 00009542 00000001 0 000 stop 0xc16ea344 sendmail
> 0xc1eb8000 00009544 00000001 0 000 stop 0xc1eb8344 sendmail
> 0xc42f8000 00009546 00000001 0 000 stop 0xc42f8344 sendmail
> 0xc4042000 00009549 00000001 0 000 stop 0xc4042344 sendmail
> 0xc4ba8000 00009551 00000001 0 000 stop 0xc4ba8344 sendmail
> [0]more>
> 0xc1b5c000 00009553 00000001 0 000 stop 0xc1b5c344 sendmail
> 0xc51f8000 00009555 00000001 0 000 stop 0xc51f8344 sendmail
> 0xc3e9c000 00009557 00000001 0 000 stop 0xc3e9c344 sendmail
> 0xc4700000 00009559 00000001 0 000 stop 0xc4700344 sendmail
> 0xc2a5c000 00009560 00000001 0 000 stop 0xc2a5c344 sendmail
> 0xc3320000 00009561 00000001 0 000 stop 0xc3320344 sendmail
> 0xc2fd0000 00009562 00000001 0 000 stop 0xc2fd0344 sendmail
> 0xc368e000 00009567 00000001 0 000 stop 0xc368e344 sendmail
> 0xc264c000 00009569 00000001 0 000 stop 0xc264c344 sendmail
> 0xc5572000 00009571 00000001 0 000 stop 0xc5572344 sendmail
> 0xc5998000 00009573 00000001 0 000 stop 0xc5998344 sendmail
> 0xc0660000 00009574 00000001 0 000 stop 0xc0660344 sendmail
> 0xc5a68000 00009577 00000001 0 000 stop 0xc5a68344 sendmail
> 0xc1d18000 00009578 00000001 0 000 stop 0xc1d18344 sendmail
> 0xc2766000 00009580 00000001 0 000 stop 0xc2766344 sendmail
> 0xc18c2000 00009583 00000001 0 000 stop 0xc18c2344 sendmail
> 0xc2c24000 00009585 00000001 0 000 stop 0xc2c24344 sendmail
> 0xc3d6a000 00009587 00000001 0 000 stop 0xc3d6a344 sendmail
> 0xc4020000 00009588 00000001 0 000 stop 0xc4020344 sendmail
> 0xc3ba0000 00009589 00000001 0 000 stop 0xc3ba0344 sendmail
> 0xc48b8000 00009594 00000001 0 000 stop 0xc48b8344 sendmail
> 0xc3476000 00009595 00000001 0 000 stop 0xc3476344 sendmail
> 0xc215c000 00009596 00000001 0 000 stop 0xc215c344 sendmail
> [0]more>
> 0xc207a000 00009598 00000001 0 000 stop 0xc207a344 sendmail
> 0xc5744000 00009603 00000001 0 000 stop 0xc5744344 sendmail
> 0xc35fc000 00009605 00000001 0 000 stop 0xc35fc344 sendmail
> 0xc3e58000 00009606 00000001 0 000 stop 0xc3e58344 sendmail
> 0xc459a000 00009608 00000001 0 000 stop 0xc459a344 sendmail
> 0xc309e000 00009610 00000001 0 000 stop 0xc309e344 sendmail
> 0xc3698000 00009612 00000001 0 000 stop 0xc3698344 sendmail
> 0xc433e000 00009614 00000001 0 000 stop 0xc433e344 sendmail
> 0xc4338000 00009615 00000001 0 000 stop 0xc4338344 sendmail
> 0xc4bf6000 00009618 00000001 0 000 stop 0xc4bf6344 sendmail
> 0xc225e000 00009619 00000001 0 000 stop 0xc225e344 sendmail
> 0xc1782000 00009621 00000001 0 000 stop 0xc1782344 sendmail
> 0xc4dac000 00009623 00000001 0 000 stop 0xc4dac344 sendmail
> 0xc22d4000 00009624 00000001 0 000 stop 0xc22d4344 sendmail
> 0xc38a4000 00009626 00000001 0 000 stop 0xc38a4344 sendmail
> 0xc22f6000 00009629 00000001 0 000 stop 0xc22f6344 sendmail
> 0xc3a34000 00009632 00000001 0 000 stop 0xc3a34344 sendmail
> 0xc479a000 00009633 00000001 0 000 stop 0xc479a344 sendmail
> 0xc1ae4000 00009636 00000001 0 000 stop 0xc1ae4344 sendmail
> 0xc3560000 00009638 00000001 0 000 stop 0xc3560344 sendmail
> 0xc4ad4000 00009639 00000001 0 000 stop 0xc4ad4344 sendmail
> 0xc1642000 00009641 00000001 0 000 stop 0xc1642344 sendmail
> 0xc1380000 00009643 00000001 0 000 stop 0xc1380344 sendmail
> [0]more>
> 0xc05a4000 00009646 00000001 0 000 stop 0xc05a4344 sendmail
> 0xc1640000 00009648 00000001 0 000 stop 0xc1640344 sendmail
> 0xc43d2000 00009649 00000001 0 000 stop 0xc43d2344 sendmail
> 0xc1b14000 00009650 00000001 0 000 stop 0xc1b14344 sendmail
> 0xc0acc000 00009652 00000001 0 000 stop 0xc0acc344 sendmail
> 0xc3b38000 00009654 00000001 0 000 stop 0xc3b38344 sendmail
> 0xc0d8a000 00009656 00000001 0 000 stop 0xc0d8a344 sendmail
> 0xc05ea000 00009657 00000001 0 000 stop 0xc05ea344 sendmail
> 0xc1e04000 00009660 00000001 0 000 stop 0xc1e04344 sendmail
> 0xc2dc2000 00009661 00000001 0 000 stop 0xc2dc2344 sendmail
> 0xc0ffe000 00009663 00000001 0 000 stop 0xc0ffe344 sendmail
> 0xc136a000 00009665 00000001 0 000 stop 0xc136a344 sendmail
> 0xc1978000 00009667 00000001 0 000 stop 0xc1978344 sendmail
> 0xc14e6000 00009669 00000001 0 000 stop 0xc14e6344 sendmail
> 0xc5070000 00009671 00000001 0 000 stop 0xc5070344 sendmail
> 0xc5684000 00009673 00000001 0 000 stop 0xc5684344 sendmail
> 0xc2b84000 00009675 00000001 0 000 stop 0xc2b84344 sendmail
> 0xc2cba000 00009677 00000001 0 000 stop 0xc2cba344 sendmail
> 0xc4ea4000 00009679 00000001 0 000 stop 0xc4ea4344 sendmail
> 0xc094c000 00009682 00000001 0 000 stop 0xc094c344 sendmail
> 0xc2e8e000 00009685 00000001 0 000 stop 0xc2e8e344 sendmail
> 0xc0566000 00009688 00000001 0 000 stop 0xc0566344 sendmail
> 0xc2836000 00009690 00000001 0 000 stop 0xc2836344 sendmail
> [0]more>
> 0xc3756000 00009691 00000001 0 000 stop 0xc3756344 sendmail
> 0xc3ed6000 00009693 00000001 0 000 stop 0xc3ed6344 sendmail
> 0xc4b8c000 00009695 00000001 0 000 stop 0xc4b8c344 sendmail
> 0xc2956000 00009701 00000001 0 000 stop 0xc2956344 sendmail
> 0xc1492000 00009702 00000001 0 000 stop 0xc1492344 sendmail
> 0xc54d6000 00009703 00000001 0 000 stop 0xc54d6344 sendmail
> 0xc2e4c000 00009704 00000001 0 000 stop 0xc2e4c344 sendmail
> 0xc2c52000 00009706 00000001 0 000 stop 0xc2c52344 sendmail
> 0xc15a4000 00009707 00000001 0 000 stop 0xc15a4344 sendmail
> 0xc396c000 00009710 00000001 0 000 stop 0xc396c344 sendmail
> 0xc1ab8000 00009712 00000001 0 000 stop 0xc1ab8344 sendmail
> 0xc14de000 00009715 00000001 0 000 stop 0xc14de344 sendmail
> 0xc583c000 00009716 00000001 0 000 stop 0xc583c344 sendmail
> 0xc1846000 00009717 00000001 0 000 stop 0xc1846344 sendmail
> 0xc5002000 00009718 00000001 0 000 stop 0xc5002344 sendmail
> 0xc5bf8000 00009720 00000001 0 000 stop 0xc5bf8344 sendmail
> 0xc160c000 00009722 00000001 0 000 stop 0xc160c344 sendmail
> 0xc29d2000 00009723 00000001 0 000 stop 0xc29d2344 sendmail
> 0xc23b6000 00009726 00000001 0 000 stop 0xc23b6344 sendmail
> 0xc4b86000 00009730 00000001 0 000 stop 0xc4b86344 sendmail
> 0xc5320000 00009732 00000001 0 000 stop 0xc5320344 sendmail
> 0xc07ea000 00009734 00000001 0 000 stop 0xc07ea344 sendmail
> 0xc31b6000 00009735 00000001 0 000 stop 0xc31b6344 sendmail
> [0]more>
> 0xc1be0000 00009738 00000001 0 000 stop 0xc1be0344 sendmail
> 0xc5ad4000 00009741 00000001 0 000 stop 0xc5ad4344 sendmail
> 0xc525a000 00009742 00000001 0 000 stop 0xc525a344 sendmail
> 0xc4536000 00009744 00000001 0 000 stop 0xc4536344 sendmail
> 0xc5a02000 00009747 00000001 0 000 stop 0xc5a02344 sendmail
> 0xc3074000 00009749 00000001 0 000 stop 0xc3074344 sendmail
> 0xc1f5e000 00009752 00000001 0 000 stop 0xc1f5e344 sendmail
> 0xc5b42000 00009758 00009107 0 000 stop 0xc5b42344 lpbm_user_run
> 0xc2ad0000 00009760 00000001 1 000 run 0xc2ad0344*sendmail
> 0xc5a3c000 00009761 00009400 0 000 run 0xc5a3c344 sendmail
> 0xc3cb0000 00009762 00009379 0 000 run 0xc3cb0344 sendmail
> 0xc5006000 00009763 00009762 0 000 stop 0xc5006344 sendmail
> 0xc09e8000 00009764 00009761 0 000 stop 0xc09e8344 sendmail
> 0xc0a0a000 00009765 00009380 0 000 run 0xc0a0a344 sendmail
> 0xc09e2000 00009766 00009765 0 000 stop 0xc09e2344 sendmail
> 0xc5470000 00009767 00009451 0 000 run 0xc5470344 sendmail
> 0xc3204000 00009768 00009410 0 000 run 0xc3204344 sendmail
> 0xc3040000 00009769 00009768 0 000 stop 0xc3040344 sendmail
> 0xc3946000 00009770 00009767 0 000 stop 0xc3946344 sendmail
> [0]kdb>
>
> I haven't seen this second panic on an XFS filesystem, however.
>
> cheers.
--
Russell Cattelan
cattelan@xxxxxxxxxxx
|