Re: how to free a used ag?

[Markus Lidel <Markus.Lidel@xxxxxxxxxxxxxxxxx>]

Hello,

Steve Lord wrote:
> Markus Lidel wrote:
> > Steve Lord wrote:
> > > Markus Lidel wrote:
> > > > could someone help me with freeing a used ag?
> > > > I want to do it inside the kernel and while the fs is mounted. I've 
> > > > already looked at the code, and it looks to me that freeing the agi 
> > > > of an ag is the easier part, so i want to start with it. I thought i 
> > > > could look at each inode of the agi i want to free, but i think i 
> > > > must iterate over each inode when freeing the used blocks anyway. So 
> > > > i iterate over each inode (starting with the root inode) and move 
> > > > the inode to a new one if it is inside the ag which should be freed. 
> > > > The second step is to look at the inode itself. If the data block is 
> > > > inside the ag it could be moved outside the ag too.
> > > > If i missed something until here, please let me know:-)
> > > Make sure you traverse all the inodes - there will probably be inodes
> > > in different allocation groups which have extents in the targeted ag.
> > > When you move an inode you need to fix up all the directories which
> > > point at it. It is going to get a new inode number, so you need to walk
> > > the directory tree looking for that inode number and replacing it. 
> > > Remember,
> > > hard links mean an inode number could show up more than once.
> > Thanks you very much for the advice... But do you think it's better to 
> > work on the on-disk structure, or do you think it's better to work 
> > with the in-core inodes?
> You had said you wanted to do this on a live filesystem. I think running
> everything through transactions is going to be more robust, and being
> able to offline part of a running filesystem is a more useful feature
> than doing it on an unmounted fs.

Okay, i want to do it transactional...
But to e. g. locate an inode i could do it using xfs_iget (in-core) or 
split the inode into agno and agino and locate the inode using the 
on-disk API, right?

Probably in my previous e-mail i've wrote a little bit confusing :-)

> The two potential approaches here are really to add some extensions to
> the kernel and use thinks like bulkstat and swapext to do the work on
> the live filesystem. Everything will be transactional, if the machine
> goes down or crashes in the middle, you should end up with a working

Think i prefer this way :-)

> filesystem. The alternative is to use something like libxfs and manipulate
> the fs from user space - if you crash you are left with bits and pieces of
> a filesystem.

If the kernel approach is to expensive (in term of code size), it should 
be not a big deal to split it and put some logic into user space...

So i think i take the first way, and decide later if it is necessary to 
remove some parts from kernel...

> So taking the use the transaction system approach:
> Add the logic to turn off new allocates in the allocation group,
> sounds like you are making progress on this.

Oh, no i could remove a unused AG from the filesystem, but i do this on 
the on-disk structure (like the growfs stuff)... I don't know if setting 
the per AG freelist to 0 would prevent the allocator to allocate 
something in this AG... It only work in best case, e. g. nothing is 
allocated in the AG :-)

> The defragmenter uses swapext to move extents from one file to another.
> I would build on this approach, so have a user space app which scans
> the fs for target inodes. Use the code from the defragmenter,
> without the logic which says it has to be less fragmented afterwards,
> to move the space to elsewhere in the fs - since allocations will not
> be allowed in the ag you are emptying, you end up with an inode and
> extents which are in other allocation groups.

Okay, i will look at the swapext to find out how it works...

> This leaves directories with blocks in the ag to rebuild, a mkdir
> of a temporary dir and some renames should take care of that.
> Other things to watch out for:
>   There is the log in the middle allocation group, you cannot empty
>   that one.
>   Sometimes when you free space it actually wants to allocate blocks
>   in that ag to put the free space tree into. Eventually the free space
>   btree will collapse, but there could be some interim steps where an
>   allocate is required to free space. There will need to be special
>   logic to allow these allocates to happen.

Okay, thanks for the advice!



Best regards,


Markus Lidel
------------------------------------------
Markus Lidel (Senior IT Consultant)

Shadow Connect GmbH
Carl-Reisch-Weg 12
D-86381 Krumbach
Germany

Phone:  +49 82 82/99 51-0
Fax:    +49 82 82/99 51-11

E-Mail: Markus.Lidel@xxxxxxxxxxxxxxxxx
URL:    http://www.shadowconnect.com