xfs
[Top] [All Lists]

Re: Fragmentation and hole punching

To: "Bradley C. Kuszmaul" <kuszmaul@xxxxxxxxx>
Subject: Re: Fragmentation and hole punching
From: Eric Sandeen <sandeen@xxxxxxxxxxx>
Date: Sat, 05 Jan 2013 23:22:30 -0600
Cc: Dave Chinner <david@xxxxxxxxxxxxx>, xfs@xxxxxxxxxxx
In-reply-to: <CAKSyJXchcnnpD6aiCXnaSF3ym1cvGZjYPGKKCgiOHY59vbuntQ@xxxxxxxxxxxxxx>
References: <CAKSyJXchcnnpD6aiCXnaSF3ym1cvGZjYPGKKCgiOHY59vbuntQ@xxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/17.0 Thunderbird/17.0
On 1/5/13 6:52 PM, Bradley C. Kuszmaul wrote:
> I'm using hole punching now in my code.  Should I have any concerns about 
> fragmentation?
> 
> Here's how I use it:
> I perform pwrite() operations of various sizes. These are "blocks" in
> my application. These blocks are always 4KB-aligned, but they are of
> many different sizes. Most writes are near a megabyte in size, but
> hardly ever exactly a megabyte.
>  I also punch holes.  The holes always correspond exactly to a previously 
> written pwrite().
>  A pwrite() always writes into a hole.  (It never overwrites current data).

Just out of curiosity - what is the end goal here, why all the 
punching-and-rewriting?

> For example I might write
>  1MB at offset 1MB
> then write 3MB at offset 2MB
> then punch a hole of size 1MB at offset 1MB
> then do some more activity elsewhere in the file
> then write 512K at an offset of 1MB (partially filling the hole that I had 
> punched)
> 
> I have a lot of experience running this code on xfs without the hole
> punching. The pattern of the pwrite()'s are the same with or without
> hole punching. So I'd hope that the hole punching didn't make things
> any worse, and maybe made things better.

Once you free up blocks, there is always a chance that some other file
will end up claiming them, so I can see fragmentation downsides; not sure
I see any upside there.

OTOH, fragmentation may not matter at all, depending on your access patterns 
etc.

My first guess for the hole-punching reason is to maximize available space;
that makes me think you are really utilizing all available space; that makes me
think you might really get files intermingled on disk if you're running
things this way.

But if you're seeking around in files to read at random offsets anyway,
it might not matter.

-Eric

> Can I expect that this will work reasonably well?  
> 
> By the way, all of your help has been outstanding.
> 
> -Bradley
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs
> 

<Prev in Thread] Current Thread [Next in Thread>