X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o98HHYcx051585 for ; Fri, 8 Oct 2010 12:17:34 -0500 X-ASG-Debug-ID: 1286558318-720101b10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-qy0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E04FD494E for ; Fri, 8 Oct 2010 10:18:38 -0700 (PDT) Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com [209.85.216.181]) by cuda.sgi.com with ESMTP id zBfPKtq9JkAzHCJI for ; Fri, 08 Oct 2010 10:18:38 -0700 (PDT) Received: by qyk10 with SMTP id 10so1421448qyk.5 for ; Fri, 08 Oct 2010 10:18:38 -0700 (PDT) Received: by 10.224.173.132 with SMTP id p4mr1772966qaz.294.1286558318020; Fri, 08 Oct 2010 10:18:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.229.221.12 with HTTP; Fri, 8 Oct 2010 10:17:52 -0700 (PDT) From: James Braid Date: Fri, 8 Oct 2010 18:17:52 +0100 Message-ID: X-ASG-Orig-Subj: ENOSPC at 90% with plenty of inodes Subject: ENOSPC at 90% with plenty of inodes To: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: mail-qy0-f181.google.com[209.85.216.181] X-Barracuda-Start-Time: 1286558319 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.43101 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi, On a fairly busy mail server hosting about 1000 cyrus mailboxes (each message is stored as a separate file), we are observing ENOSPC errors from the cyrus pop/imap daemons when attempting to create (tiny) files. e.g. Oct 8 12:18:49 xxx cyrus/pop3[14119]: IOERROR: creating /user/imap/proc/14119: No space left on device Oct 8 12:18:49 xxx cyrus/lmtpunix[14361]: IOERROR: creating quota file /user/imap/quota/m/user.xx.NEW: No space left on device Oct 8 12:18:49 xxx cyrus/imap[12804]: IOERROR: creating quota file /user/imap/quota/d/user.xxx.NEW: No space left on device The filesystem appears to have plenty of space available: # df -i /user Filesystem Inodes IUsed IFree IUse% Mounted on /dev/sdb1 102294688 10201313 92093375 10% /user # df /user Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb1 213014148 190023408 22990740 90% /user Any ideas for how can we go about debugging where ENOSPC is coming from? Unfortunately it's running a vintage 2.6.24 kernel (Ubuntu 8.04) - we're trying to get it upgraded to 10.04 (2.6.32) Thanks, James