xfs
[Top] [All Lists]

RE: database filesystem (ab)use [Was: Re: realtime ( wasRE:unexpected hi

To: "Stewart Smith" <stewart@xxxxxxxxx>
Subject: RE: database filesystem (ab)use [Was: Re: realtime ( wasRE:unexpected high fragmentation, any ideas? )]
From: "Scott Fagg" <scott.fagg@xxxxxxxxxxx>
Date: Fri, 8 Apr 2005 13:44:19 +1000
Cc: "Steve Lord" <lord@xxxxxxx>, <linux-xfs@xxxxxxxxxxx>
Sender: linux-xfs-bounce@xxxxxxxxxxx
Thread-index: AcU75DuuAT+kcEI9SXelq2jUs4sKUAAACLaA
Thread-topic: database filesystem (ab)use [Was: Re: realtime ( wasRE:unexpected high fragmentation, any ideas? )]
> -----Original Message-----
> From: Stewart Smith [mailto:stewart@xxxxxxxxx] 
> Sent: Friday, 8 April 2005 12:39 PM
> To: Scott Fagg
> Cc: Steve Lord; linux-xfs@xxxxxxxxxxx
> Subject: RE: database filesystem (ab)use [Was: Re: realtime ( 
> wasRE:unexpected high fragmentation, any ideas? )]
> 
> On Fri, 2005-04-08 at 09:47 +1000, Scott Fagg wrote:
> > Using InnoDB and binary logs gets about 1MB/s throughput on 
> the server,
> > but 5MB/s on the desktop.
> 
> is dma on? right drivers for the server? sounds like a problem along
> those lines.

I'll look into that. 

> 
> > I've also tried toggling between mysql v3 and v5, the FC3 
> kernel RPM and
> > a vanilla kernel from kernel.org, but all that has had 
> little impact.
> 
> performance of mysql 3 through 5 should be about the same or better

That's what i've been seeing on the servers that seem to be behaving.

> 
> > Other processes can write to disc at much higher rates than 
> mysql, so
> > perhaps i should take this problem to a different forum .. 
> 
> this is interesting... maybe try a bonnie++ run on the disk.

I ran bonnie++ with what seemed like reasonable values and did one run
each, toggling the '-b' option to run with and without fsync();

Good server using XFS (older hardware):

./bonnie++ -d /tmp/ -s 2000 -m bneuxs11 -r 1000 -x 1
bneuxs11,2000M,10349,97,23391,16,13092,13,10242,87,43113,27,482.7,2,16,(
lots of +'s deleted)

./bonnie++ -d /tmp/ -s 2000 -m bneuxs11 -r 1000 -x 1 -b
bneuxs11,2000M,10449,97,20486,14,12483,13,11574,98,43418,27,525.5,3,16,(
lots of +'s deleted)

Mis-behaving Server using ext3fs (new machine):

./bonnie++ -d /tmp/ -s 4000 -m syduxs08 -r 2000 -x 1 
syduxs08,4000M,36927,79,43812,14,22258,5,45767,85,61616,5,372.3,0,16,(lo
ts of +'s deleted)

./bonnie++ -d /tmp/ -s 4000 -m syduxs08 -r 2000 -x 1 -b
syduxs08,4000M,39005,84,37419,12,23580,5,50082,92,61995,5,263.4,0,16,(lo
ts of +'s deleted)

I think those results are reasonable, the better machine giving slightly
better numbers.

My insert script produces results like this :

On the good server:

- myisam, no bin-log = 61 seconds, kb/s varied a lot
- myisam, bin-log = 69 seconds, kb/s varied a lot
- innodb, no bin-log = 230 seconds, 3000kb/s
- innodb, bin-log = 230 seconds, 3000kb/s

On the mis-behaving:

- myisam, no bin-log = 24 seconds, 5000kb/s
- myisam, bin-log = 24 seconds, 5000kb/s
- innodb, no bin-log = 24 minutes, 800kb/s
- innodb, bin-log = 24 minutes, 800kb/s

Dud innodb file ?

> 
> 
> You may also need to tune mysql parameters - check out the 
> manual or the

The only tuning i've attempted is to take some hints from the
my-huge.cnf sample file, but that had no discernable impact, possibly
because i'm not really putting much load on the system.

> MySQL Administrator GUI tool (which also does some performance
> monitoring).
> 
> -- 
> Stewart Smith, Software Engineer
> MySQL AB, www.mysql.com
> Office: +14082136540 Ext: 6616
> VoIP: 6616@xxxxxxxxxxxxxxxx
> Mobile: +61 4 3 8844 332
> 
> Jumpstart your cluster:
> http://www.mysql.com/consulting/packaged/cluster.html
> 


<Prev in Thread] Current Thread [Next in Thread>
  • RE: database filesystem (ab)use [Was: Re: realtime ( wasRE:unexpected high fragmentation, any ideas? )], Scott Fagg <=