[Top] [All Lists]

Re: unexpected high fragmentation, any ideas?

To: Steve Lord <lord@xxxxxxx>
Subject: Re: unexpected high fragmentation, any ideas?
From: Marc Lehmann <schmorp@xxxxxxxxxx>
Date: Mon, 4 Apr 2005 21:09:57 +0200
Cc: Russell Cattelan <cattelan@xxxxxxx>, Chris Wedgwood <cw@xxxxxxxx>, linux-xfs@xxxxxxxxxxx
In-reply-to: <42518613.2060103@xfs.org>
References: <20050403004653.GA981@schmorp.de> <20050403050542.GB5727@taniwha.stupidest.org> <20050403135805.GC24559@schmorp.de> <42500F94.1070603@thebarn.com> <20050403004653.GA981@schmorp.de> <20050403050542.GB5727@taniwha.stupidest.org> <20050403135805.GC24559@schmorp.de> <4250000C.5070709@xfs.org> <20050403215230.GA919@schmorp.de> <42518613.2060103@xfs.org>
Sender: linux-xfs-bounce@xxxxxxxxxxx
On Mon, Apr 04, 2005 at 01:23:15PM -0500, Steve Lord <lord@xxxxxxx> wrote:
> Marc Lehmann wrote:
> >Is this somewthing inside xfs or is this just setting the st_blksize stat
> >data field? If the latter, then its unlikely to help, as I configured
> >mythtv to use fairly large write's already (minimum 512KB, usually around
> >2MB). But thanks for the tip, it might help some other XFS filesystems I
> >have (although it isn't a problem there).
> >
> This is inside XFS and is different than doing a large I/O. What is 
> happening

[very detailed explanations skipped]

Thanks a lot for explaining it in that detail!

I'll try chris's patch and see how that improves things.

> The realtime allocator uses a different binary chop algorithm which while
> wasteful, makes it very hard to fragment realtime files. Hmm, buffered works

Hmm, is the realtime code ready for use then? I was under the (likely
wrong) impression that the realtime code is not yet ready.

However, I *guess* I could easily live with, say, 128MB (or even
1GB) realtime extents and live with the internal frgamentation that
will occur:  If my interpretation is right (not likely) the realtime
allocator more-or-less treats these realtime extents as the basic unit of
allocation, so internal fragmentation will be extremely high, but external
fragmentation is low?

> on realtime, maybe you could try hacking mythtv to set the realtime flag in
> inodes and see what happens.

I'll try.

> make it more likely for there to be unused space just beyond the last
> extent allocated for a file. Implementing that would take some time by
> someone experienced inside the allocator (Glen are you reading this ;-)
> at the end of the day, there may actually not be much of a payoff.

Quite possible. I think both suggestions might work out.

Thanks again!

                The choice of a
      -----==-     _GNU_
      ----==-- _       generation     Marc Lehmann
      ---==---(_)__  __ ____  __      pcg@xxxxxxxx
      --==---/ / _ \/ // /\ \/ /      http://schmorp.de/
      -=====/_/_//_/\_,_/ /_/\_\      XX11-RIPE

<Prev in Thread] Current Thread [Next in Thread>