On Donnerstag, 21. Juli 2011 Dave Chinner wrote:
> If you are writing files that grow like this, then you are doing
> something wrong. If the app can't do it's IO differently, then this
> is exactly the reason we have userspace-controlled preallocation
> interfaces.
>
> Filesystems cannot prevent user stupidity from screwing something
> up....
This can happen if you copy a syslog server over to a new disk, then let
it start it's work again. Many files that start small and grow. Luckily,
the logs are rotated latest monthly, so it shouldn't be too bad.
> > And files >64KiB are immediately fragmented
> > then. At this time, there are only 16384 * 2KiB = 32MiB used, which
> > is 3,125% of the disk. I can't believe my numbers, are they true?
>
> No, because most filesystems have a 4k block size.
I just meant pure disk usage. Of 1GB, only 32MB are used, and this worst
case example hits us badly.
> Not to mention
> that fragmentation is likely to be limited to the single AG the files
> in the directory belong to. i.e. even if we can't allocation a sunit
> aligned chunk in an AG, we won't switch to another AG just to do
> sunit aligned allocation.
This is good to know also, thanks.
> > OK, this is a worst case scenario, and as you've said before, any
> > filesystem can be considered full at 85% fill grade. But it's
> > incredible how quickly you could fuck up a filesystem when using
> > su/sw and writing small files.
>
> Well, don't use a filesystem that is optimised for storing large
> sizes, large files and high bandwidth for storing lots of small
> files, then. Indeed, the point of not packing the files is so they
> -don't fragemnt as they grow-. XFS is not designed to be optimal
> for small filesystems or small files. In most cases it will deal
> with them just fine, so in reality your concerns are mostly
> unfounded...
Yes, I just wanted to know about the corner cases, and how XFS behaves.
Actually, we're changing over to using NetApps, and with their WAFL
anyway I should drop all su/sw usage and just use 4KB blocks.
And even when XFS is optimized for large files, there are often small
ones. Think of a mysql server with hundreds of DBs and
innodb_file_per_table set. Even when some DBs are large, there are many
small files.
But this thread has drifted a bit. XFS does great work, and now I
understand the background a bit more. Thanks, Dave.
--
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc
it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531
// Haus zu verkaufen: http://zmi.at/langegg/
signature.asc
Description: This is a digitally signed message part.
|