[Top] [All Lists]

Re: Defrag pitfals (was Re: Support for XFS File systems)

Subject: Re: Defrag pitfals (was Re: Support for XFS File systems)
From: "D. Stimits" <stimits@xxxxxxxxx>
Date: Wed, 12 Feb 2003 14:48:07 -0700
Cc: linux-xfs@xxxxxxxxxxx
In-reply-to: <Pine.LNX.4.44.0302121952080.7758-100000@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
References: <Pine.LNX.4.44.0302121952080.7758-100000@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
Reply-to: stimits@xxxxxxxxx
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2b) Gecko/20021018
Bogdan Costescu wrote:

On Wed, 12 Feb 2003, Chris Wedgwood wrote:
Scientific simulations that produce large amounts of data over a long
time. Of course, "large amounts" and "long time" is relative, in our case
being "several hundreds megabytes" and "days". The file is created at the
beginning of the simulation and data is appended to it for the whole
simulation duration; data is never rewritten during the simulation.
Depending on data size, simulations can finish sooner or later and produce
files that are of different sizes, however in order to ease later backup,
we limit the size of one file to approximately 650 MB. Writting to disk is not a problem, as the rate at which data is produced is very low. However,
reading the data becomes a problem and we need to do it to either analize
the data or transfer it from there to other computer or CD.

I am curious if the data is binary, or if it is in such a form that SQL storage could be used? That would solve a lot of problems (or perhaps alter them) for replication, and especially for fragmentation choices and questions. Postgresql might be a good choice, network enabling it and having nodes write to postgresql either via sockets or a daemon running on your master node. If you need to sort or view non-binary data in more than one way, SQL would be a blessing.

I am also curious what kind of Postgresql or MySQL performance differences people have found under XFS, compared to any other file system?

D. Stimits, stimits AT attbi DOT com

<Prev in Thread] Current Thread [Next in Thread>