[Top] [All Lists]

Re: xfs lockup on 2.4.6-SMP kernel with 1.1TB filesystem

To: Jani Jaakkola <jjaakkol@xxxxxxxxxxxxxx>
Subject: Re: xfs lockup on 2.4.6-SMP kernel with 1.1TB filesystem
From: Steve Lord <lord@xxxxxxx>
Date: Tue, 17 Jul 2001 10:53:41 -0500
Cc: linux-xfs@xxxxxxxxxxx
In-reply-to: Message from Jani Jaakkola <jjaakkol@cs.helsinki.fi> of "Tue, 17 Jul 2001 18:30:53 +0300." <Pine.LNX.4.33.0107171356130.3797-100000@hallikari.cs.Helsinki.FI>
Sender: owner-linux-xfs@xxxxxxxxxxx
Could you possibly try the cvs tree, Linus was still working deadlocks
out of the memory allocation/reclaim end of things up until 2.4.7-pre2.
XFS and ext2 will almost certainly push things in different directions.

Another issue here is that you may actually be creating inode numbers with
greated than 32 bits in them with a filesystem this size. If you run 
xfs_growfs -n on the mount point of the filesystem and run the attached
perl script with the following arguments it will tell you how many bits
your inodes can consume.

        inode_size.pl blk-size inode-size agcount blocks

xfs_growfs -n /usr
meta-data=/usr                   isize=256    agcount=8, agsize=103670 blks
data     =                       bsize=4096   blocks=829355, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=0
naming   =version 2              bsize=4096  
log      =internal               bsize=4096   blocks=1500
realtime =none                   extsz=65536  blocks=0, rtextents=0

inode_size.pl 4096  256 8  829355 
Blocksize: 4096
Inodesize: 256
Number of allocation groups: 8
Number of data blocks (in filesystem blocks) : +829355

Number of inodes in a filesystem block: 16 - bits: 4
Number of datablocks in an allocation group +103669 - bits: 17 
Number of allocation groups: 8 - bits: 3
Total number of bits required  for an inode: 24

You can play with numbers to make the number of bits <= 32, increasing
the inode size will be the thing which does it for you, also if you
did not end up with 4GB allocation groups you should attempt to get
them setup that way. Unfortunately this means mkfs to fix.

I do have some plans to make this issue go away for large filesystems, but
you beat me to it!


> OK, this seems to be deterministic and not a HW-problem..
> I am testing xfs on a brand new linux file server having a 1.1T xfs
> filesystem in dual processor (Intel III Xeon 1GHz) motherboard
> with 1G memory.
> When using xfs with linux kernel 2.4.6 (compiled with egcs-2.91.66) and
> Linux software RAID5 all processes accessing the /dev/md0 device lockup in
> D-state when trying to write files from network with tar (kupdated, tar,
> pagebuf_daemon and /bin/sync) typically after few hundred megabytes have
> been written. Kdb tells me that all processes are hanging forever in
> get_active_stripe().
> All SCSI drives are still working, when accessed directly
> through /dev/sda-g.. Oops, seems that they too get stuck after a while
> (perhaps trying to write some pages to disk).
> The same problem does not come up when using ext2.
> /bin/free tells me this:
>              total       used       free     shared    buffers     cached
> Mem:        899328     324772     574556          0     134952     115272
> -/+ buffers/cache:      74548     824780
> Swap:      2097136          0    2097136
> Only thing I can do, is to reboot with /sbin/reboot -nf.
> I've have seen other reports in mailing list with this kind of problem.
> Are there known solutions? I think I might try the cvs version...
> - Jani

Attachment: inode_size.pl
Description: inode_size.pl

<Prev in Thread] Current Thread [Next in Thread>