On 2/3/2012 4:10 PM, Brian Candler wrote:
> On Fri, Feb 03, 2012 at 01:42:54PM -0600, Stan Hoeppner wrote:
>> You've hit the peak read rate of these Hitachi drives. As others
>> pointed out, if you need more read performance than the dozen of these
>> you plan to RAID stripe, then you'll need to swap them for units with a
>> faster spindle:
>> 7.2k 1.21x
>> 10k 1.68x
>> 15k 2.53x
>> or with SSDs, which will yield an order of magnitude increase. Your
>> stated need is 20M 500-800KB files, or 20GB if my math is correct.
> Thanks for your suggestion, but unfortunately your maths isn't correct: 20M
> x 0.65MB = 13TB. And that's just one of many possible datasets like this.
Wow, you're right. How did I miss so many zeros? Got in hurry I guess.
> I'm aware that I'm working with low-performance drives. This is intentional:
> we need low power consumption so we can get lots in a rack, and large
> capacity at low cost.
SSDs would fulfill criteria 1/2 but obviously not 3/4.
> Fortunately our workload will also parallelise easily, and throwing it
> across 24 spindles will be fine. But obviously I want to squeeze the most
> performance out of each spindle we have first. I'm very happy to have found
> the bottleneck that was troubling me :-)
Will you be using mdraid or hardware RAID across those 24 spindles?