[Fwd: Re: Kernel hangs with mongo.pl benchmark.]

[Paul Schutte <paul@it.up.ac.za>]

Seth Mos wrote:

> On Thu, 30 Aug 2001, Paul Schutte wrote:
>
> > Hi,
> >
> > I found a problem while doing benchmarks with mongo.pl
> >
> > The sympthom is that the machine just hangs. No error messages.
> >
> > I ran "vmstat 1" in a window while doing the benchmark. I noticed that
> > the "bo" becomes 0 for a while before it
> > completely locks up. I can force an immediate lockup by doing a "sync".
> >
> > I also noticed (by doing 'watch df -k') in another term that the used
> > space increased as the files are created.
> > The used space then starts decreasing. At this point you can start
> > walking to the server room ...
> >
> > When running reiserfs on the partitions, it went through without a
> > problem.
> > (I would have been amused if their own benchmark failed)
> >
> > With ext2 I got "Out of disk space" errors. Ext2 actually ran out of
> > inodes on this test and not space.
> >
> > With that in mind I played with the various inode options in mkfs.xfs.
> > I tried maxpct=0 and then maxpct=90 in combinations with size=2048.
> > None of these resolved the problem.
> >
> > I ran the test on 4 different machines with the same result on 3 of the
> > machines.
> >
> > I use debian 2.2r3 (stable) on all the machines.
> >
> > I compiled the kernels with egcs-1.1.2 which I installed spesificly for
> > compiling kernels.
> > I also tried kernels compiled with gcc 2.95.4 and gcc version 3.0.2
> > 20010825 (Debian prerelease)
> > I tried kernels 2.4.7,2.4.8-pre4,2.4.8,2.4.9,2.4.10-pre1,2.4.10-pre2 all
> > with the same result.
> > All these kernels are from the CVS tree.
> >
> > The machine that ran successfully was my home PC.
> > Celeron 300A (run at 100MHz FSB thus 450MHz) 128M ram, 30G seagate
> > baracuda IDE hard drive.
> > Kernel 2.4.8 build with egcs-1.1.2.
> >
> > The machines that does'nt work:
> > Dell PowerEdge 4400 1GHz Xeon,1G RAM, Seagate Cheetha 15k RPM on PERC
> > 3/Di controller no RAID settings.
>
> You mean scsi mode? Or a JBOD config. Did you need to use the patch for
> this raid controller.
>

I did aplied the AACRAID patch. I don't use the onboard adaptec
controller.
No patch = no drives.

>
> > Dell 1400C   PIII 866MHz,128M RAM,  2x9G Fujitsu 10k RPM drives on
> > Adaptec controller.
> > Custom build 2x500MHz PIII, 768M RAM, 9G Seagate Cheetha on DPT 3755
> > controller no RAID settings.
>
> The common factor seems to be adaptec hardware ;-)
>
> > I tried to pin point the problem, but without success.
> >
> > Unfortunetly I can't get kdb to compile.
> > It fails with error:
> >
> > /home/paul/test/linux-2.4-xfs/linux/include/linux/dis-asm.h:148: storage
> > size of `display_endian' isn't known
> > make[3]: *** [kdb_bt.o] Error 1
> >
> > I think you will be able to reproduce the hanging problem.
> >
> > I ran ./mongo.pl xfs /dev/sdc1 /testfs xfs 6 from the mongo benchmark
> > suite found at www.reiserfs.com
> >
> > When I do ./mongo.pl xfs /dev/sdc1 /testfs xfs 1 (single process), it
> > works fine.
> >
> > I added a definition to mongo.pl for xfs:
> >
> >     if ( $FILESYSTEM eq "xfs" ) {
> >         system("mkfs.xfs -f -l size=8192b $DEVICE") ;
> >         system("mount -t xfs -o logbufs=8 $DEVICE $TESTDIR") ;
> >     }
>
> Have you tried using less logbufs, I can imagine that during deletion
> (which is slow) the buffering might become problematic.
>

It is still in the creation phase. No deletes. I asume the decreasing in
size
is because there are some
packing activity going on. The files are of 100 bytes avarage size.


>
> Can you try it with less buffering like 2(default) or 4?
>

I repeated the test on the 4400 with logbufs=2.

It still gave up.

>
> Cheers
> Seth

Paul