[Top] [All Lists]

Re: Performance problem - reads slower than writes

To: Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx>
Subject: Re: Performance problem - reads slower than writes
From: Brian Candler <B.Candler@xxxxxxxxx>
Date: Sun, 5 Feb 2012 09:05:02 +0000
Cc: Christoph Hellwig <hch@xxxxxxxxxxxxx>, xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date:from:to :cc:subject:message-id:references:mime-version:content-type :in-reply-to; s=sasl; bh=D/EHMfi5yyFJeFBWFSoNKlHfEOQ=; b=uCUVNne vK60l+VcGBnLZwp/qXn3KLqZagubF2Y8QuG3X8ORYB0V34il98/7xdHgX+Fyc+Vc ipLRdm2kAGv7aA88k8OqPzncRXNUjsbEzsIUZe9158QkHGrz1iG2IyBfJ/ywH3Wh R7iKVaXF3Vw1JhZyWkEtkwPdtqv+gzxCd5dk=
Domainkey-signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:from:to:cc :subject:message-id:references:mime-version:content-type :in-reply-to; q=dns; s=sasl; b=CPNDJStsls4pESGvd9b5jbWQHeNs2/Hrt ZOz8Ek3+s98CKGzxrYo0v0LVfVF4rGzUsTldUsDTsgbcJHnCUxb8J6h+u+JmobjX Do+u67D9ouC0AixglfHOx+9xwY7C7U0Exbq7xQ292vjg2mnzDCbUafy64gwVhaOZ EFXT/UloOw=
In-reply-to: <4F2E10C1.3040200@xxxxxxxxxxxxxxxxx>
References: <20120131103126.GA46170@xxxxxxxx> <20120131145205.GA6607@xxxxxxxxxxxxx> <20120203115434.GA649@xxxxxxxx> <4F2C38BE.2010002@xxxxxxxxxxxxxxxxx> <20120203221015.GA2675@xxxxxxxx> <4F2D016C.9020406@xxxxxxxxxxxxxxxxx> <20120204112436.GA3167@xxxxxxxx> <4F2D2953.2020906@xxxxxxxxxxxxxxxxx> <20120204200417.GA3362@xxxxxxxx> <4F2E10C1.3040200@xxxxxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
On Sat, Feb 04, 2012 at 11:16:49PM -0600, Stan Hoeppner wrote:
> When you lose a disk in this setup, how do you rebuild the replacement
> drive?  Do you simply format it and then move 3TB of data across GbE
> from other Gluster nodes?

Basically, yes. When you read a file, it causes the mirror to synchronise
that particular file. To force the whole brick to come back into sync you
run find+stat across the whole filesystem.

> Even if the disk is only 1/3rd full, such a
> restore seems like an expensive and time consuming operation.  I'm
> thinking RAID has a significant advantage here.

Well, if you lose a 3TB disk in a RAID-1 type setup, then the whole disk has
to be copied block by block (whether it contains data or not). So the
consideration here is network bandwidth.

I am building with 10GE, but even 1G would be just about sufficient to carry
the peak bandwidth of a single one of these disks.  (dd on the raw disk
gives 120MB/s at the start and 60MB/s at the end)

The whole manageability aspect certainly needs to be considered very
seriously though.  With RAID1 or RAID10, dealing with a failed disk is
pretty much pull and plug; with Gluster we'd be looking at having to mkfs
the new filesystem, mount it at the right place, and then run the self-heal. 
This will have to be weighed against the availability advantages of being
able to take an entire storage node out of service.



<Prev in Thread] Current Thread [Next in Thread>