Re: Question about continous blocks for inode due to 'no space left on device' problem

[Michel Verbraak <info1stsetup@xxxxxxxxx>]

2016-02-08 15:35 GMT+01:00 Brian Foster <bfoster@xxxxxxxxxx>:

On Mon, Feb 08, 2016 at 02:09:15PM +0100, Michel Verbraak wrote:
> 2016-02-05 23:30 GMT+01:00 Dave Chinner <david@xxxxxxxxxxxxx>:
>
> > On Fri, Feb 05, 2016 at 02:13:00PM +0100, Michel Verbraak wrote:
> > > All,
> > >
> > > Recently we ran into a problem where our filesystem (300GB in size)
> > > reported 'no space left on device' (ENOSPC) but when we looked at disk
> > > space usage and inode usage it was around 52% for disk space and 11% for
> > .......
>
>
> >
> > > Store 03 (root@server):~# xfs_db -c freesp /dev/sdb
> > >  from   to extents blocks  pct
> > >Â Â Â Â1Â Â Â Â1 3282633 3282633Â Â9.03
> > >Â Â Â Â2Â Â Â Â3 3416223 8372325Â 23.03
> > >Â Â Â Â4Â Â Â Â7 6175009 24700036Â 67.94
> >
> > Yup, no more aligned 4 block extents in the filesystem.
> >
> > Dave Thank you for your answer but are you able to explain a bit more on
> how you determine there are no aligned 4 block extends available?
> It apparently is not the 'from 4 to 7' line telling there are at least
> continuous extents of 4 blocks big. Or is it because 24700036 divided
> by 6175009
> is exactly 4 and not a bit more?
> When I calculate the ratio between the individual AG freesp they were all
> four exactly 4.
>

I'm not sure there is a way to determine whether these extents are
aligned or not short of walking the free space btree structures with

..<snip>Â

Â

BTW, a metadump of your fs while it's in this ENOSPC state would be
useful towards testing something like that, if you'd be willing to
provide one. Note that xfs_metadump has obfuscation that hides filenames
and whatnot for privacy purposes.

If we do run into problem again, we do hope not as it is on a production system, we will create the xfs_metadump and provide it to this list.

Thank you for the extra explanation.

Cheers Michel.
Â

Brian

> As said we would like to detect this problem from happening again. And we
> would like this detection to be done by our monitoring service.
> One of my colleagues mentions the following calculation to find out if
> there are enough aligned 4 block extents:
>
> Starting from line "from 4 to 7" calculate: <blocks-column> - (4 *
> <extents-column>) > 0. When not there is not enough free space for new
> inodes.
>
> On another system we have which does not see these problems yet we have:
>
>  from   to extents blocks  pct
>Â Â Â Â1Â Â Â Â1 3483987 3483987Â Â7.83
>Â Â Â Â2Â Â Â Â3 3560607 8709838Â 19.58
>Â Â Â Â4Â Â Â Â7 6402893 26486634Â 59.55
>Â Â Â Â8Â Â Â 15Â 418450 5799206Â 13.04
>
> 26486634 / 6402893 = 4.136666 and
> 26486634 - (4 * 6402893) = 875062 > 0 and
> 5799205 - (4 * 418450) = 4125405 > 0
>
>
> > into this condition.
> >
> > Cheers,
> >
> > Dave.
> > --
> > Dave Chinner
> > david@xxxxxxxxxxxxx
> >
>
> Michel.

> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs