Question about continous blocks for inode due to 'no space left on device' problem

[Michel Verbraak]

2016-02-08 15:35 GMT+01:00 Brian Foster <bfoster at redhat.com>:

> On Mon, Feb 08, 2016 at 02:09:15PM +0100, Michel Verbraak wrote:
> > 2016-02-05 23:30 GMT+01:00 Dave Chinner <david at fromorbit.com>:
> >
> > > On Fri, Feb 05, 2016 at 02:13:00PM +0100, Michel Verbraak wrote:
> > > > All,
> > > >
> > > > Recently we ran into a problem where our filesystem (300GB in size)
> > > > reported 'no space left on device' (ENOSPC) but when we looked at
> disk
> > > > space usage and inode usage it was around 52% for disk space and 11%
> for
> > > .......
> >
> >
> > >
> > > > Store 03 (root at server):~# xfs_db -c freesp /dev/sdb
> > > >    from      to extents  blocks    pct
> > > >       1       1 3282633 3282633   9.03
> > > >       2       3 3416223 8372325  23.03
> > > >       4       7 6175009 24700036  67.94
> > >
> > > Yup, no more aligned 4 block extents in the filesystem.
> > >
> > > Dave Thank you for your answer but are you able to explain a bit more
> on
> > how you determine there are no aligned 4 block extends available?
> > It apparently is not the 'from 4 to 7' line telling there are at least
> > continuous extents of 4 blocks big. Or is it because 24700036 divided
> > by 6175009
> > is exactly 4 and not a bit more?
> > When I calculate the ratio between the individual AG freesp they were all
> > four exactly 4.
> >
>
> I'm not sure there is a way to determine whether these extents are
> aligned or not short of walking the free space btree structures with
> ..<snip>



> BTW, a metadump of your fs while it's in this ENOSPC state would be
> useful towards testing something like that, if you'd be willing to
> provide one. Note that xfs_metadump has obfuscation that hides filenames
> and whatnot for privacy purposes.
>
> If we do run into problem again, we do hope not as it is on a production
system, we will create the xfs_metadump and provide it to this list.

Thank you for the extra explanation.

Cheers Michel.


> Brian
>
> > As said we would like to detect this problem from happening again. And we
> > would like this detection to be done by our monitoring service.
> > One of my colleagues mentions the following calculation to find out if
> > there are enough aligned 4 block extents:
> >
> > Starting from line "from 4 to 7" calculate: <blocks-column> - (4 *
> > <extents-column>) > 0. When not there is not enough free space for new
> > inodes.
> >
> > On another system we have which does not see these problems yet we have:
> >
> >    from      to extents  blocks    pct
> >       1       1 3483987 3483987   7.83
> >       2       3 3560607 8709838  19.58
> >       4       7 6402893 26486634  59.55
> >       8      15  418450 5799206  13.04
> >
> > 26486634 / 6402893 = 4.136666 and
> > 26486634 - (4 * 6402893) = 875062 > 0 and
> > 5799205 - (4 * 418450) = 4125405 > 0
> >
> >
> > > into this condition.
> > >
> > > Cheers,
> > >
> > > Dave.
> > > --
> > > Dave Chinner
> > > david at fromorbit.com
> > >
> >
> > Michel.
>
> > _______________________________________________
> > xfs mailing list
> > xfs at oss.sgi.com
> > http://oss.sgi.com/mailman/listinfo/xfs
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20160208/3f96b8a8/attachment.html>