[Top] [All Lists]

Re: [RFD] : Allowing xfs_fsr to defragment portions of a file

To: xfs@xxxxxxxxxxx, Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: [RFD] : Allowing xfs_fsr to defragment portions of a file
From: Dhruvesh Rathore <adrscube@xxxxxxxxx>
Date: Tue, 3 Mar 2015 20:18:45 +0530
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=zUpZu0btiS4cicjeuh7Xna6Zl2XufqGMK1mkRGq7o4M=; b=d6B4yJZFgxqB1QyCfbBY15A1al38wZjFw9saQEX2mCpTcC51FBDLwq6oZtrWZDkrOu lgi4kCRouIumkNTDHuFp55Ww4+/UkwfVobBJbVEthUzgMMXPdNyg8y2SJe/Vx4VwQZvP IEpHJ0doKi/F9QeK0BUJCtY9UZ8/a2ZTMfzALkDm1IRffT5axfEwttFIZrCtD5wFvYQn EqVqXLaKgVkNn4UJdpN66X+1FcsbQ/KreMS2nzz2bxgaNdUJE6YFPX+WCuhIC++RLpxb urNfB93fnYLh5GR+W4P5x86CSyAhNzngHfZAxb6KAPkuMWLVQfWVBWSZCHHYHeEx63WA PXiw==
In-reply-to: <CALJmpHOZe44t5Mimv213vrQV60Jp6FGpAYpT36bmawKZina56g@xxxxxxxxxxxxxx>
References: <CALJmpHP+PwdV1n8VLusxqz0Wb=1PLL52frEMBn8+4h0v5+7Ssg@xxxxxxxxxxxxxx> <20150216234838.GC4251@dastard> <CALJmpHOZe44t5Mimv213vrQV60Jp6FGpAYpT36bmawKZina56g@xxxxxxxxxxxxxx>
On Tue, Feb 17, 2015 at 5:18 AM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
> On Sun, Feb 15, 2015 at 08:08:46PM +0530, Dhruvesh Rathore wrote:

>> A new function will have to hence focus on swapping the existing
>> extents from the
>> fragmented file. For this a new_swap_extent() function will be needed,
>> which will take arguments
>> as the current extent and the location(in free space) where it needs
>> to be swapped to.
> You can't swap an in use extent with a free space location. You need
> to preallocate space in a temporary file, copy the data across to
> it, and then swap the extent ranges between the two files.
>> For the above functionality Dave suggested that we have a look at
>> and its usage.
> The extent swap between two files is exactly what that ioctl does.
> Look at the API, especially the struct move_extent that defines the
> information passed to the kernel to perform the extent swap
> operation. The XFS APi will be similar.

Over the past few days we have been going over the XFS API for
EXT4_IOC_MOVE_EXT available in misc/e4defrag.c, to lay down the
groundwork of the required XFS API.


These are a few things that we felt do not be changed in the new API:

1) The original file will be opened in fsrfile() in xfs_fsr.c, similar to the
original file being opened in file_defrag() in e4defrag.c

2) The check for sufficient freespace done in fsrfile_common() in
xfs_fsr.c achieves the same objective as the check_freesize() in


On the same lines, we have identified the following additions
that will be needed in the XFS API for fsr/xfs_fsr.c

1) The read_fd_bmap() function in xfs_fsr.c which gets the block map
of range of original file (map) and the defragmented file (outmap), will
need to be added to when we are performing partial defragmentation.

A function get_file_extents() is called in e4defrag.c twice, which populates
a struct fiemap, struct fiemap_extent and struct fiemap_extent_list
for both original file and donor file (temp file).

The operation of this function will need to be added in xfs_fsr.c as the
EXT4_IOC_MOVE_EXT ioctl primarily uses these fiemap structures.
We will need both the fiemap structures and the existing part of read_fd_bmap( )
in this new XFS API.
(P.S We have already achieved the above using XFS_IOC_FIEMAPFS, and have
got extent information in fiemap structures)

2) The temp file getting created in xfs_fsr.c, will be made similar to
the donor file of e4defrag.c.
We were a bit unsure about the function of join_extents() in
e4defrag.c, which finds a continuous region (extents group), and how
it will fit in the XFS API.
Is it similar to the process of preallocating extents[toff, tlen] in temp file ?

3) Now instead of copying data from [orig, off, len] to [temp, toff,
tlen] and then calling
XFS_IOC_SWPEXT, we will need to simulate the ioctl EXT4_IOC_MOVE_EXT.

In call_defrag( ) in e4defrag.c, the struct fiemap_extent_list
structure, which is basically
a link list of extent information is used.
In our XFS API we will similarly use this structure in a loop and call
the IOCTL for
each of the extents (off donor file) which needs to be filled by
swapping data blocks.

In this loop the fields orig_start, donor_start, len and moved_len of
struct move_extent
will be populated from the fiemap_extent_list structure before each ioctl call.


Before all of the above we will need to select the extents that can be
i.e the best possible choice, and then mark them.
For this we decided to create another data structure - an array
(user_array[ ]), which
will store extent size information (blocks).
For instance say a file has 5 extents then
user_array[0] : save size of extent 1
user_array[1] : save size of extents 1, 2
user_array[2] : save size of extents 1, 2, 3
user_array[3] : save size of extents 1, 2, 3, 4
user_array[4] : save size of extents 1, 2 3, 4, 5
This way we could perform the selection efficiently, by necessary
comparison with
available space.


As said, we have already added to the XFS API, and have got extent info
in fiemap structures.
We would love to hear your comments or directions on the above points. After
confirmation from you, we will finish creating the XFS API on the above lines.

> The process in fsr is this:
>         1. bulkstat original file inode [bstat] for change
>            detection whilst doing copying operation.
>         2. get block map of range [off, len] of orignal file
>            that needs to be swapped
>         3. create temp file
>         4. preallocate extent [toff, tlen] in temp file
>         5. check the number of extents in [toff, tlen] is less than
>            the number of extents in [off, len] in the original file.
>            If it's not an improvement, abort.
>         6. copy data from [orig, off, len] to [temp, toff, tlen]
>         7. ioctl(orig_fd, XFS_IOC_SWAPEXT_RANGE, [bstat, off, len, temp, 
> toff])
> The kernel side of the ioctl is somewhat more complex, and we'll get
> to that once the fsr infrastructure is ready to use it.
> Cheers,
> Dave.


<Prev in Thread] Current Thread [Next in Thread>