Date: Thu, 30 Aug 2001 19:04:29 +0200 (CEST)
From: Seth Mos <knuffie@xs4all.nl>
To: Paul Schutte <paul@it.up.ac.za>
cc: XFS mailing list <linux-xfs@oss.sgi.com>
Subject: Re: Kernel hangs with mongo.pl benchmark.
In-Reply-To: <3B8E6E8E.D2082F15@it.up.ac.za>
Message-ID: <Pine.BSI.4.10.10108301856550.16049-100000@xs3.xs4all.nl>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-linux-xfs@oss.sgi.com
Precedence: bulk

On Thu, 30 Aug 2001, Paul Schutte wrote:

> Hi,
> 
> I found a problem while doing benchmarks with mongo.pl
> 
> The sympthom is that the machine just hangs. No error messages.
> 
> I ran "vmstat 1" in a window while doing the benchmark. I noticed that
> the "bo" becomes 0 for a while before it
> completely locks up. I can force an immediate lockup by doing a "sync".
> 
> I also noticed (by doing 'watch df -k') in another term that the used
> space increased as the files are created.
> The used space then starts decreasing. At this point you can start
> walking to the server room ...
> 
> When running reiserfs on the partitions, it went through without a
> problem.
> (I would have been amused if their own benchmark failed)
> 
> With ext2 I got "Out of disk space" errors. Ext2 actually ran out of
> inodes on this test and not space.
> 
> With that in mind I played with the various inode options in mkfs.xfs.
> I tried maxpct=0 and then maxpct=90 in combinations with size=2048.
> None of these resolved the problem.
> 
> I ran the test on 4 different machines with the same result on 3 of the
> machines.
> 
> I use debian 2.2r3 (stable) on all the machines.
> 
> I compiled the kernels with egcs-1.1.2 which I installed spesificly for
> compiling kernels.
> I also tried kernels compiled with gcc 2.95.4 and gcc version 3.0.2
> 20010825 (Debian prerelease)
> I tried kernels 2.4.7,2.4.8-pre4,2.4.8,2.4.9,2.4.10-pre1,2.4.10-pre2 all
> with the same result.
> All these kernels are from the CVS tree.
> 
> The machine that ran successfully was my home PC.
> Celeron 300A (run at 100MHz FSB thus 450MHz) 128M ram, 30G seagate
> baracuda IDE hard drive.
> Kernel 2.4.8 build with egcs-1.1.2.
> 
> The machines that does'nt work:
> Dell PowerEdge 4400 1GHz Xeon,1G RAM, Seagate Cheetha 15k RPM on PERC
> 3/Di controller no RAID settings.

You mean scsi mode? Or a JBOD config. Did you need to use the patch for
this raid controller.

> Dell 1400C   PIII 866MHz,128M RAM,  2x9G Fujitsu 10k RPM drives on
> Adaptec controller.
> Custom build 2x500MHz PIII, 768M RAM, 9G Seagate Cheetha on DPT 3755
> controller no RAID settings.

The common factor seems to be adaptec hardware ;-)

> I tried to pin point the problem, but without success.
> 
> Unfortunetly I can't get kdb to compile.
> It fails with error:
> 
> /home/paul/test/linux-2.4-xfs/linux/include/linux/dis-asm.h:148: storage
> size of `display_endian' isn't known
> make[3]: *** [kdb_bt.o] Error 1
> 
> I think you will be able to reproduce the hanging problem.
> 
> I ran ./mongo.pl xfs /dev/sdc1 /testfs xfs 6 from the mongo benchmark
> suite found at www.reiserfs.com
> 
> When I do ./mongo.pl xfs /dev/sdc1 /testfs xfs 1 (single process), it
> works fine.
> 
> I added a definition to mongo.pl for xfs:
> 
>     if ( $FILESYSTEM eq "xfs" ) {
>         system("mkfs.xfs -f -l size=8192b $DEVICE") ;
>         system("mount -t xfs -o logbufs=8 $DEVICE $TESTDIR") ;
>     }

Have you tried using less logbufs, I can imagine that during deletion
(which is slow) the buffering might become problematic.

Can you try it with less buffering like 2(default) or 4?

Cheers
Seth