[Top] [All Lists]

Re: Slightly Urgent: XFS No Space Left On Device

To: Dave Hall <kdhall@xxxxxxxxxxxxxx>
Subject: Re: Slightly Urgent: XFS No Space Left On Device
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Thu, 2 Apr 2015 11:12:35 +1100
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <551C4CB8.7@xxxxxxxxxxxxxx>
References: <551993CF.4060908@xxxxxxxxxxxxxx> <20150330194510.GD28621@dastard> <551C4CB8.7@xxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2010-09-15)
On Wed, Apr 01, 2015 at 03:53:28PM -0400, Dave Hall wrote:
> Please pardon the 'top-post', but here is the additional information
> requested:
> This is a Dell R720xd dual 8-core Xeon system with 128GB RAM.  The
> RAID controller is Dell PERC H710 Mini with 12 2TB disks in RAID6.
> The OS is Debian 6 with kernel 3.2.0-0.bpo.4-amd64 #1 SMP Debian
> 3.2.65-1+deb7u2~bpo60+1 x86_64.

So defaults to inode32 allocation....

> From /proc/mounts:
>    /dev/sdb1 /data xfs
> rw,noexec,noatime,attr2,delaylog,allocsize=64k,logbsize=64k,sunit=128,swidth=1280,usrquota,prjquota
>    0 0

... and inode64 is not in the mount options.....

> The output from xfs_info was previously included, but is repeated here:
> # xfs_info /data
> meta-data=/dev/sdb1              isize=256    agcount=19,agsize=268435440 blks

Inode allocation requires contiguous free space of 16k aligned to 8k
boundaries to allocate new inode chunks. Also, 1TB AGs, so with
inode32, inodes can only be allocated in AG 0.

> Here are the more extensive freesp outputs for each of the 19 AGs:
>    # xfs_db -r /dev/sdb1 -c 'freesp -s -a0'
>        from      to extents  blocks    pct
>           1       1     747     747  19.68
>           2       3    1045    2496  65.77
>           4       7     138     552  14.55
>    total free extents 1930
>    total free blocks 3795
>    average free extent size 1.96632

And that says you have no correctly aligned free 16k extents that
can be allocated in AG 0. i.e. no more inodes can be allocated, and
that's where the ENOSPC is coming from.

Unmount, add the inode64 mount option, and you'll be able to
allocate inodes again as they will be allowed to be allocated in
any AG, not just AG 0.


Dave Chinner

<Prev in Thread] Current Thread [Next in Thread>