xfs
[Top] [All Lists]

Re: Dealing with a very large file -1,5TB

To: Simon Matter <simon.matter@xxxxxxxxxxxxxxxx>
Subject: Re: Dealing with a very large file -1,5TB
From: Christoph Klocker <cklocker@xxxxxx>
Date: Thu, 25 Sep 2003 15:57:41 +0200
Cc: linux-xfs@xxxxxxxxxxx
In-reply-to: <3466.10.1.200.117.1064496571.squirrel@xxxxxxxxxxxxxxxxxxxxxxx>
References: <3F72E2D4.1030109@xxxxxx> <3466.10.1.200.117.1064496571.squirrel@xxxxxxxxxxxxxxxxxxxxxxx>
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.5b) Gecko/20030723 Thunderbird/0.1
I had a look at the dd man page, but I couldn't find out how I use it to check the write speed, can you tell me how I should do it.

/Christoph

_____________
Klocker Christoph
KTH-AMT
10044 Stockholm



Simon Matter wrote:

Did you check the raw write speed of your devices using dd? When I did
tests in the past I got almost the same speed with XFS like raw. Until you
are sure your devices perform much better, XFS should not be involved too
much.

Simon

Hi

I am developing a system to stream very large uncompressed videofiles
which are about 1,5TB.
I need a bandwith of 195MB/s.
my system is now:
kernel: 2.4.18
xeon 2,4GHz
1GB RAM
3ware escalade controller 12 s-ata harddisks, seagate barracuda 7200.7 -
120GB - SCSI

when I do different tests with bonnie++ I get very good results for a
2GB file, readperformance 249mb/s,
but when the files get larger the speed is going down significantly.
at 10GB - 216MB/s
at 100GB - 187MB/s
at 1TB - 161MB/s
the write performance stays at 183MB/s any time, up to 1TB.
As my harddisk have a sustained rate of 32-50 MB/s it should not be the
case that they influence the results.
I am now thinking maybe the filesystem influences the result when the
disks are getting fuller. I thought also about that the logfiles get
bigger and bigger and that maybe a reason.

I had a look if its possible to switch of the logging completely (there
is no reason to have it because I just have one file) , but I think its
not possible with xfs.

Is there a special parameter configuration for having just one file at
this size (1,5TB)

/Christoph

_____________
Klocker Christoph
KTH-AMT
10044 Stockholm







<Prev in Thread] Current Thread [Next in Thread>