Hello and thanks for your reply.
Am Dienstag 19 Oktober 2010 03:42 CEST, Dave Chinner <david@xxxxxxxxxxxxx>
> What's the layout of you storage? how many disks, size, etc.
> How much RAM, number of CPUs in your server?
Server hardware is a tricky question as there are 4 different servers
I have tested XFS on. Three of those are running as a VM guest under
VMWare ESX. One is on pure hardware.
All of the servers have Xeon Processors, every CentOS installation
limited to only use 2 Cores (vCPUs in VMWare).
Listing by server (VMs flagged with a *):
Intel(R) Xeon(TM) CPU 3.00GHz - 8GB DDR2 ECC - 2x single HDDs, no RAID, 10k
rpm SAS (80 + 140GB)
* Intel(R) Xeon(TM) MV CPU 3.20GHz - 4GB DDR2 ECC - Storage is a SAN (via
FibreChannel) (90 GB exclusive)
* Intel(R) Xeon(R) CPU 5160 @ 3.00GHz - 10GB DDR2 ECC - 2x RAID0 with 10k
rpm SAS (74GB each)
* Intel(R) Xeon(R) CPU X5560 @ 2.80GHz - 16 GB DDR3 ECC - Storage is a SAN
(via FibreChannel) (80GB exclusive)
With the SANs I checked if they are under heavy load (they are not).
Of those VMs only I is situated in an environment where there is load
from other VMs.
> Using information from a 7 year old web page about how someone
> optimised their filesystem for a specific bonnie++ workload
> is not really a good place to start when it comes to real-world
Well it gave me some pointers and after some thinking and testing I
ended up with those options. I have to admit that my knowledge of
XFS is based only on that and some other webpages so I wouldn't
know where and how to improve performance.
> You need understand why the performance falls through the floor
> first, then work out what needs optimising. It may be that your sort
> algorithm has exponential complexity, or you are running out of RAM,
> or something else. It may not have anything at all to do with the
It isn't really a sorting algorithm. Every file has 3 elements that can be
used to categorise it. So the actual sorting amounts to a linear search
(read) for those elements and a move/copy&delete (after possible
creation of 4 directories in which to sort/categorise).
[categorisation is done in this pattern: /<unique id>/<year>/fixed/(a|b)]
After some testing it seems copy&delete is 4x slower than reading the file.
> The symptoms you describe sound like the difference between
> operation in cache vs out of cache (i.e. RAM speed vs disk speed),
> and if so then no amount of filesystem tweaking will fix it.
I see your point but I can't see much caching with inflating tar/bzip2
(and that is where XFS really shines here ...).
> However, if you describe you system and the algorithms you are using
> in more detail, we might be able to help identify the cause...
The only addition I can make to the algorithms is this:
My application is written in Java and I have tried three different approaches
to the copy & delete matter
- java.io -> Read + immediate Write, then Delete (via Readers and Writers)
- java.io -> renameTo (applicable not for all directories/paths)
- java.nio -> FileChannels (transferTo)
All of which show the same issues. So my reasoning was either hardware
or FS. But seeing as the degeneration als happens with the SANs I thought
it might be more of a FS specific issue.
Application is mostly single threaded (at least at the point I'm having
If you would like more information feel free to ask.
Thanks and Greetings from Germany,