From jweiner@redhat.com Tue Nov 1 05:53:40 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1AreeY113560 for ; Tue, 1 Nov 2011 05:53:40 -0500 X-ASG-Debug-ID: 1320144819-1970007b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 040A41CD7A3A for ; Tue, 1 Nov 2011 03:53:39 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 8IbFsCDpqbbbNiOB for ; Tue, 01 Nov 2011 03:53:39 -0700 (PDT) X-ASG-Whitelist: Barracuda Reputation Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pA1ArJUL023592 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 1 Nov 2011 06:53:19 -0400 Received: from dexter.home.cmpxchg.org (vpn1-4-47.ams2.redhat.com [10.36.4.47]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id pA1ArG33023303 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 1 Nov 2011 06:53:18 -0400 Date: Tue, 1 Nov 2011 11:52:57 +0100 From: Johannes Weiner To: Wu Fengguang Cc: Michal Hocko , Andrew Morton , Mel Gorman , Christoph Hellwig , Dave Chinner , Jan Kara , Rik van Riel , Minchan Kim , Chris Mason , "Theodore Ts'o" , Andreas Dilger , "Li, Shaohua" , "xfs@oss.sgi.com" , "linux-btrfs@vger.kernel.org" , "linux-ext4@vger.kernel.org" , "linux-mm@kvack.org" , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" X-ASG-Orig-Subj: Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones Subject: Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones Message-ID: <20111101105257.GF5819@redhat.com> References: <1317367044-475-1-git-send-email-jweiner@redhat.com> <1317367044-475-4-git-send-email-jweiner@redhat.com> <20110930142805.GC869@tiehlicka.suse.cz> <20111027155618.GA25524@localhost> <20111027161359.GA1319@redhat.com> <20111027204743.GA19343@localhost> <20111027221258.GA22869@localhost> <20111027231933.GB1319@redhat.com> <20111028203944.GB20607@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111028203944.GB20607@localhost> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1320144820 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Oct 29, 2011 at 04:39:44AM +0800, Wu Fengguang wrote: > [restore CC list] > > > > I'm trying to understand where the performance gain comes from. > > > > > > I noticed that in all cases, before/after patchset, nr_vmscan_write are all zero. > > > > > > nr_vmscan_immediate_reclaim is significantly reduced though: > > > > That's a good thing, it means we burn less CPU time on skipping > > through dirty pages on the LRU. > > > > Until a certain priority level, the dirty pages encountered on the LRU > > list are marked PageReclaim and put back on the list, this is the > > nr_vmscan_immediate_reclaim number. And only below that priority, we > > actually ask the FS to write them, which is nr_vmscan_write. > > Yes, it is. > > > I suspect this is where the performance improvement comes from: we > > find clean pages for reclaim much faster. > > That explains how it could reduce CPU overheads. However the dd's are > throttled anyway, so I still don't understand how the speedup of dd page > allocations improve the _IO_ performance. They are throttled in balance_dirty_pages() when there are too many dirty pages. But they are also 'throttled' in direct reclaim when there are too many clean + dirty pages. Wild guess: speeding up direct reclaim allows dirty pages to be generated faster and the writer can better saturate the BDI? Not all filesystems ignore all VM writepage requests, either. xfs e.g. ignores only direct reclaim but honors requests from kswapd. ext4 honors writepage whenever it pleases. On those, I can imagine the reduced writepage intereference to help. But that can not be the only reason as btrfs ignores writepage from the reclaim in general and still sees improvement. From jweiner@redhat.com Tue Nov 1 05:56:30 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1AuUXI113668 for ; Tue, 1 Nov 2011 05:56:30 -0500 X-ASG-Debug-ID: 1320144988-331f02070000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id F2DB2214D7C for ; Tue, 1 Nov 2011 03:56:28 -0700 (PDT) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id WG07EIza2AwCCCkJ for ; Tue, 01 Nov 2011 03:56:28 -0700 (PDT) X-ASG-Whitelist: Barracuda Reputation Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pA1AuFVj024615 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 1 Nov 2011 06:56:15 -0400 Received: from dexter.home.cmpxchg.org (vpn1-4-47.ams2.redhat.com [10.36.4.47]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id pA1AuDn0002230 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Tue, 1 Nov 2011 06:56:14 -0400 Date: Tue, 1 Nov 2011 11:55:53 +0100 From: Johannes Weiner To: Wu Fengguang Cc: Michal Hocko , Andrew Morton , Mel Gorman , Christoph Hellwig , Dave Chinner , Jan Kara , Rik van Riel , Minchan Kim , Chris Mason , "Theodore Ts'o" , Andreas Dilger , "Li, Shaohua" , "xfs@oss.sgi.com" , "linux-btrfs@vger.kernel.org" , "linux-ext4@vger.kernel.org" , "linux-mm@kvack.org" , "linux-fsdevel@vger.kernel.org" , "linux-kernel@vger.kernel.org" X-ASG-Orig-Subj: Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones Subject: Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones Message-ID: <20111101105553.GG5819@redhat.com> References: <1317367044-475-1-git-send-email-jweiner@redhat.com> <1317367044-475-4-git-send-email-jweiner@redhat.com> <20110930142805.GC869@tiehlicka.suse.cz> <20111028201829.GA20607@localhost> <20111031113321.GA30890@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111031113321.GA30890@localhost> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1320144989 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Oct 31, 2011 at 07:33:21PM +0800, Wu Fengguang wrote: > > //regression > > 3) much increased cpu %user and %system for btrfs > > Sorry I find out that the CPU time regressions for btrfs are caused by > some additional trace events enabled on btrfs (for debugging an > unrelated btrfs hang bug) which results in 7 times more trace event > lines: > > 2701238 /export/writeback/thresh=1000M/btrfs-1dd-4k-8p-2941M-1000M:10-3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+ > 19054054 /export/writeback/thresh=1000M/btrfs-1dd-4k-8p-2941M-1000M:10-3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+ > > So no real regressions. Phew :-) > Besides, the patchset also performs good on random writes: > > 3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+ 3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+ > ------------------------ ------------------------ > 1.65 -5.1% 1.57 MMAP-RANDWRITE-4K/btrfs-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X > 18.65 -6.4% 17.46 MMAP-RANDWRITE-4K/ext3-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X > 2.09 +1.2% 2.12 MMAP-RANDWRITE-4K/ext4-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X > 2.49 -0.3% 2.48 MMAP-RANDWRITE-4K/xfs-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X > 51.35 +0.0% 51.36 MMAP-RANDWRITE-64K/btrfs-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X > 45.20 +0.5% 45.43 MMAP-RANDWRITE-64K/ext3-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X > 44.77 +0.7% 45.10 MMAP-RANDWRITE-64K/ext4-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X > 45.11 +2.5% 46.23 MMAP-RANDWRITE-64K/xfs-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X > 211.31 +0.2% 211.74 TOTAL write_bw Hmm, mmapped IO page allocations are not annotated yet, so I expect this to be just runtime variations? > And writes to USB key: > > 3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+ 3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+ > ------------------------ ------------------------ > 5.94 +0.8% 5.99 UKEY-thresh=1G/btrfs-1dd-4k-8p-4096M-1024M:10-X > 2.64 -0.8% 2.62 UKEY-thresh=1G/ext3-10dd-4k-8p-4096M-1024M:10-X > 5.10 +0.3% 5.12 UKEY-thresh=1G/ext3-1dd-4k-8p-4096M-1024M:10-X > 3.26 -0.8% 3.24 UKEY-thresh=1G/ext3-2dd-4k-8p-4096M-1024M:10-X > 5.63 -0.5% 5.60 UKEY-thresh=1G/ext4-10dd-4k-8p-4096M-1024M:10-X > 6.04 -0.1% 6.04 UKEY-thresh=1G/ext4-1dd-4k-8p-4096M-1024M:10-X > 5.90 -0.2% 5.88 UKEY-thresh=1G/ext4-2dd-4k-8p-4096M-1024M:10-X > 2.45 +22.6% 3.00 UKEY-thresh=1G/xfs-10dd-4k-8p-4096M-1024M:10-X > 6.18 -0.4% 6.16 UKEY-thresh=1G/xfs-1dd-4k-8p-4096M-1024M:10-X > 4.81 +0.0% 4.81 UKEY-thresh=1G/xfs-2dd-4k-8p-4096M-1024M:10-X > 47.94 +1.1% 48.45 TOTAL write_bw > > In summary, I see no problem at all in these trivial writeback tests. > > Tested-by: Wu Fengguang Thanks! From ben@decadent.org.uk Tue Nov 1 09:14:24 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1EEOJA118451 for ; Tue, 1 Nov 2011 09:14:24 -0500 X-ASG-Debug-ID: 1320156862-6c6a039e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from shadbolt.e.decadent.org.uk (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D1A421F296FC for ; Tue, 1 Nov 2011 07:14:22 -0700 (PDT) Received: from shadbolt.e.decadent.org.uk (shadbolt.e.decadent.org.uk [88.96.1.126]) by cuda.sgi.com with ESMTP id ioNDighDKlwzRWqP for ; Tue, 01 Nov 2011 07:14:22 -0700 (PDT) Received: from [2001:470:1f08:1539:21c:bfff:fe03:f805] (helo=deadeye) by shadbolt.decadent.org.uk with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1RLF6a-0002tJ-Fn; Tue, 01 Nov 2011 14:14:20 +0000 Received: from ben by deadeye with local (Exim 4.76) (envelope-from ) id 1RLF6P-0006x2-R2; Tue, 01 Nov 2011 14:14:09 +0000 Message-ID: <1320156842.30281.28.camel@deadeye> X-ASG-Orig-Subj: Re: [PATCH] Fix possible memory corruption in xfs_readlink Subject: Re: [PATCH] Fix possible memory corruption in xfs_readlink From: Ben Hutchings To: Carlos Maiolino Cc: xfs@oss.sgi.com Date: Tue, 01 Nov 2011 14:14:02 +0000 Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-fJlQeUrvKum8YH3IBiQ1" X-Mailer: Evolution 3.0.3-2 Mime-Version: 1.0 X-SA-Exim-Connect-IP: 2001:470:1f08:1539:21c:bfff:fe03:f805 X-SA-Exim-Mail-From: ben@decadent.org.uk X-SA-Exim-Scanned: No (on shadbolt.decadent.org.uk); SAEximRunCond expanded to false X-Barracuda-Connect: shadbolt.e.decadent.org.uk[88.96.1.126] X-Barracuda-Start-Time: 1320156862 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79017 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --=-fJlQeUrvKum8YH3IBiQ1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, 2011-10-18 at 02:18 -0200, Carlos Maiolino wrote: > Fixes a possible memory corruption when the link is larger than > MAXPATHLEN and XFS_DEBUG is not enabled. This also remove the > S_ISLNK assert, since the inode mode is checked previously in > xfs_readlink_by_handle() and via VFS. >=20 > Signed-off-by: Carlos Maiolino > --- > fs/xfs/xfs_vnodeops.c | 11 ++++++++--- > 1 files changed, 8 insertions(+), 3 deletions(-) >=20 > diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c > index 51fc429..c3288be 100644 > --- a/fs/xfs/xfs_vnodeops.c > +++ b/fs/xfs/xfs_vnodeops.c > @@ -123,13 +123,18 @@ xfs_readlink( > =20 > xfs_ilock(ip, XFS_ILOCK_SHARED); > =20 > - ASSERT(S_ISLNK(ip->i_d.di_mode)); > - ASSERT(ip->i_d.di_size <=3D MAXPATHLEN); > - > pathlen =3D ip->i_d.di_size; pathlen is a signed int (32-bit) and di_size has signed 64-bit type. So, even if di_size was verified to be non-negative earlier (is it?)... > if (!pathlen) > goto out; > =20 > + if (pathlen > MAXPATHLEN) { ...pathlen may be negative here and will pass this check. Ben. > + xfs_alert(mp, "%s: inode (%llu) symlink length (%d) too long", > + __func__, (unsigned long long)ip->i_ino, pathlen); > + ASSERT(0); > + return XFS_ERROR(EFSCORRUPTED); > + } > + > + > if (ip->i_df.if_flags & XFS_IFINLINE) { > memcpy(link, ip->i_df.if_u1.if_data, pathlen); > link[pathlen] =3D '\0'; --=20 Ben Hutchings Computers are not intelligent. They only think they are. --=-fJlQeUrvKum8YH3IBiQ1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQIVAwUATq/+q+e/yOyVhhEJAQqMTg/9H3hIhbliIi6FQI1N0MqY4Sxmrjr745n0 WYsWL8IyTaPVEUiSlKiS5b/V4Lq7NheDVY32Pa/XroJo8CUZ1/sIlgVBgpYI/Rk0 2ovZ7ebcrNgqnrwTvmg2TdZkAXI083sofaZ7Ni/uAwO/CAf/qWDyym0SmJQ4iuMd AezzwqJ1XzTxwswasBFOoVDTQaDbMqtiRe8tocXQ4ClaKNvHIqJWra6iwlUb8UKn r9sqQuagzwy8gZdJT/h59BWAIOVHcjxKYBFXeJ/tAJ8HJ1meukeU4Hva2g1UbRso tF0PMbXyxIrMV9ugxlrPBkX4c0f/H1j4eWzZlHgYDMiEGANrljvZlh3061VPpuUd 1AB4WKSzd/Fii8rKce+CkJCGYe6HG6n1tA4Yj012zHSJGWiElkkoFR+2wGCvg3aT 9jsuerwjjCnnzLgBzoHtiQ+403vlDIDJI6wnVGvKLTr3RhZNdq4V6CBC6OlNa+Lr uQkgXaEdXmj5gQZX7PKLPJ6XJ/ijqUCWNr+427mT2y8m+SgBjw/U6eNWCW0lJV42 Tk44/xooP0IejKOj7kCY3i5rqhyJN/kk70p4qYb1+UbCwmGH1z6sdb1qIFZvz11v y73Wu6DbycX1zZ3p8F/eVt33Hd9tqSg7wuRRqK4aFK5teLoiMrk3uK/n5ZlMX67k AZ4HtievltQ= =E102 -----END PGP SIGNATURE----- --=-fJlQeUrvKum8YH3IBiQ1-- From wkendall@sgi.com Tue Nov 1 14:53:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1JrMZ6129282 for ; Tue, 1 Nov 2011 14:53:22 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id B91D98F8049 for ; Tue, 1 Nov 2011 12:53:19 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id A081F700075D for ; Tue, 1 Nov 2011 14:53:19 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 610C920DF3BB; Tue, 1 Nov 2011 14:53:19 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Subject: [PATCH 0/4] xfstests: refactor arg processing in common.dump Date: Tue, 1 Nov 2011 14:53:03 -0500 Message-Id: <1320177187-18871-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch series reworks the argument processing for the dump and restore helper routines in common.dump. Note this series depends on the following outstanding patches, as there are overlapping changes in _parse_args(): xfstests: add test 264 for testing xfsdump -D xfstests: add 265 and 266 for multiple media From wkendall@sgi.com Tue Nov 1 14:53:30 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1JrU16129301 for ; Tue, 1 Nov 2011 14:53:30 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 33B6C30404E; Tue, 1 Nov 2011 12:53:27 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 10871700075C; Tue, 1 Nov 2011 14:53:27 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 0799820DF3BB; Tue, 1 Nov 2011 14:53:27 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 1/4] xfstests: refactor cumulative restore tests Date: Tue, 1 Nov 2011 14:53:04 -0500 Message-Id: <1320177187-18871-2-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320177187-18871-1-git-send-email-wkendall@sgi.com> References: <1320177187-18871-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The cumulative restore tests call _do_restore_file_cum(), which requires a dump level to be passed in the -l option in order to determine whether the restore directory needs to be prepared or not. -l is not a valid xfsrestore option, so doing things this way prevents tests from passing options to xfsrestore. It's more straightforward to have the test call _prepare_restore_dir itself prior to starting a series of cumulative restores. Signed-off-by: Bill Kendall --- 064 | 3 ++- 065 | 3 ++- 264 | 5 +++-- common.dump | 6 ++---- 4 files changed, 9 insertions(+), 8 deletions(-) diff --git a/064 b/064 index 16960c9..bf8a7bc 100755 --- a/064 +++ b/064 @@ -97,12 +97,13 @@ while [ $i -le 9 ]; do done echo "Do the cumulative restores" +_prepare_restore_dir i=0 while [ $i -le 9 ]; do dump_file=$tmp.df.level$i echo "" echo "restoring from df.level$i" - _do_restore_file_cum -l $i + _do_restore_file_cum echo "ls -l restore_dir" ls -lR $restore_dir | _ls_size_filter | _check_quota_file let i=$i+1 diff --git a/065 b/065 index f21a87f..2ebe1cc 100755 --- a/065 +++ b/065 @@ -174,12 +174,13 @@ while [ $i -le $num_dumps ]; do done echo "Do the cumulative restores" +_prepare_restore_dir i=0 while [ $i -le $num_dumps ]; do dump_file=$tmp.df.level$i echo "" echo "restoring from df.level$i" - _do_restore_file_cum -l $i + _do_restore_file_cum echo "list restore_dir" _list_dir $restore_dir | _check_quota_file | tee $tmp.restorals.$i let i=$i+1 diff --git a/264 b/264 index 9544a58..1caa15f 100755 --- a/264 +++ b/264 @@ -63,10 +63,11 @@ _do_dump_file _add_and_append_dumpdir_fill dump_file=$tmp.df.1 _do_dump_file -l 1 -D +_prepare_restore_dir dump_file=$tmp.df.0 -_do_restore_file_cum -l 0 +_do_restore_file_cum dump_file=$tmp.df.1 -_do_restore_file_cum -l 1 +_do_restore_file_cum _ls_compare_sub _diff_compare diff --git a/common.dump b/common.dump index 3942e83..56b348a 100644 --- a/common.dump +++ b/common.dump @@ -1065,14 +1065,12 @@ _do_restore_file() # # Cumulative restore from a file -# Need to specify the dump level e.g. "-l 0" +# Must call _prepare_restore_dir before the first +# (and only the first) call to this function. # _do_restore_file_cum() { _parse_args $* - if echo $dump_args | grep '\-l0' >/dev/null; then - _prepare_restore_dir - fi echo "Restoring cumumlative from file..." opts="$_restore_debug -f $dump_file -r $restore_dir" -- 1.7.0.4 From wkendall@sgi.com Tue Nov 1 14:53:33 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1JrXNI129316 for ; Tue, 1 Nov 2011 14:53:33 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id E3EF230404E; Tue, 1 Nov 2011 12:53:32 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id BB128700075C; Tue, 1 Nov 2011 14:53:32 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id AFEC420DF3BB; Tue, 1 Nov 2011 14:53:32 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 2/4] xfstests: refactor xfsdump quota checking Date: Tue, 1 Nov 2011 14:53:05 -0500 Message-Id: <1320177187-18871-3-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320177187-18871-1-git-send-email-wkendall@sgi.com> References: <1320177187-18871-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Tests can enable/disable quota checking by passing -q or -Q to the various dump and restore helper routines. But -q and -Q are valid xfsdump/xfsrestore options, so in addition to being confusing, tests cannot use these options. Use --check-quota and --no-check-quota instead. Signed-off-by: Bill Kendall --- 061 | 3 ++- common.dump | 4 ++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/061 b/061 index 414675f..2b08e11 100755 --- a/061 +++ b/061 @@ -48,7 +48,8 @@ dump_file=src/dumpfile # override dump_file to checked-in dumpfile session_label="stress_056" # we have no quotas to restore # if we happen to run this on crackle then put the hostname back -_do_restore_file -Q | sed -e 's/HOSTNAME/crackle/g' -e 's#SCRATCH_DEV#/dev/dsk/dks0d2s1#' +_do_restore_file --no-check-quota | +sed -e 's/HOSTNAME/crackle/g' -e 's#SCRATCH_DEV#/dev/dsk/dks0d2s1#' _diff_compare_sub _ls_nodate_compare_sub diff --git a/common.dump b/common.dump index 56b348a..09f1a91 100644 --- a/common.dump +++ b/common.dump @@ -903,10 +903,10 @@ _parse_args() multi=$2 shift ;; - -q) + --check-quota) do_quota_check=true ;; - -Q) + --no-check-quota) do_quota_check=false ;; -l|-d) -- 1.7.0.4 From wkendall@sgi.com Tue Nov 1 14:53:38 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1JrcRP129331 for ; Tue, 1 Nov 2011 14:53:38 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 44FC030404E; Tue, 1 Nov 2011 12:53:38 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 34FFE700075C; Tue, 1 Nov 2011 14:53:38 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 2B83F20DF3BB; Tue, 1 Nov 2011 14:53:38 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 3/4] xfstests: allow dump file name to be passed as arg Date: Tue, 1 Nov 2011 14:53:06 -0500 Message-Id: <1320177187-18871-4-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320177187-18871-1-git-send-email-wkendall@sgi.com> References: <1320177187-18871-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean xfsdump tests using tapes (rather than files) can pass the tape pathname to the dump/restore helper functions using the -f option. Change _parse_args() so that this can be done for file-based tests as well, so that they don't have to set the global 'dump_file' variable before doing the dump or restore. Signed-off-by: Bill Kendall --- 061 | 5 ++--- 064 | 9 +++------ 065 | 9 +++------ 264 | 12 ++++-------- common.dump | 3 +++ 5 files changed, 15 insertions(+), 23 deletions(-) diff --git a/061 b/061 index 2b08e11..8f1d149 100755 --- a/061 +++ b/061 @@ -44,11 +44,10 @@ _supported_os IRIX Linux # _create_dumpdir_fill_perm (small dump) _create_dumpdir_fill_perm -dump_file=src/dumpfile # override dump_file to checked-in dumpfile -session_label="stress_056" +# override dump_file to checked-in dumpfile # we have no quotas to restore # if we happen to run this on crackle then put the hostname back -_do_restore_file --no-check-quota | +_do_restore_file --no-check-quota -f src/dumpfile -L stress_056 | sed -e 's/HOSTNAME/crackle/g' -e 's#SCRATCH_DEV#/dev/dsk/dks0d2s1#' _diff_compare_sub _ls_nodate_compare_sub diff --git a/064 b/064 index bf8a7bc..8e9e694 100755 --- a/064 +++ b/064 @@ -78,8 +78,7 @@ while [ $i -le 9 ]; do cat $tmp.dates.$i >>$seq.full fi - dump_file=$tmp.df.level$i - _do_dump_file -l $i + _do_dump_file -f $tmp.df.level$i -l $i let i=$i+1 done @@ -91,8 +90,7 @@ i=0 while [ $i -le 9 ]; do echo "" echo "restoring from df.level$i" - dump_file=$tmp.df.level$i - _do_restore_toc + _do_restore_toc -f $tmp.df.level$i let i=$i+1 done @@ -100,10 +98,9 @@ echo "Do the cumulative restores" _prepare_restore_dir i=0 while [ $i -le 9 ]; do - dump_file=$tmp.df.level$i echo "" echo "restoring from df.level$i" - _do_restore_file_cum + _do_restore_file_cum -f $tmp.df.level$i echo "ls -l restore_dir" ls -lR $restore_dir | _ls_size_filter | _check_quota_file let i=$i+1 diff --git a/065 b/065 index 2ebe1cc..c1dee5d 100755 --- a/065 +++ b/065 @@ -158,8 +158,7 @@ while [ $i -le $num_dumps ]; do echo "Listing of what files we have at level $i:" _list_dir $dump_dir | tee $tmp.ls.$i - dump_file=$tmp.df.level$i - _do_dump_file -l $i + _do_dump_file -f $tmp.df.level$i -l $i let i=$i+1 done @@ -168,8 +167,7 @@ i=0 while [ $i -le $num_dumps ]; do echo "" echo "restoring from df.level$i" - dump_file=$tmp.df.level$i - _do_restore_toc + _do_restore_toc -f $tmp.df.level$i let i=$i+1 done @@ -177,10 +175,9 @@ echo "Do the cumulative restores" _prepare_restore_dir i=0 while [ $i -le $num_dumps ]; do - dump_file=$tmp.df.level$i echo "" echo "restoring from df.level$i" - _do_restore_file_cum + _do_restore_file_cum -f $tmp.df.level$i echo "list restore_dir" _list_dir $restore_dir | _check_quota_file | tee $tmp.restorals.$i let i=$i+1 diff --git a/264 b/264 index 1caa15f..949e911 100755 --- a/264 +++ b/264 @@ -58,16 +58,12 @@ fi _create_dumpdir_fill # ensure file/dir timestamps precede dump timestamp sleep 2 -dump_file=$tmp.df.0 -_do_dump_file +_do_dump_file -f $tmp.df.0 _add_and_append_dumpdir_fill -dump_file=$tmp.df.1 -_do_dump_file -l 1 -D +_do_dump_file -f $tmp.df.1 -l 1 -D _prepare_restore_dir -dump_file=$tmp.df.0 -_do_restore_file_cum -dump_file=$tmp.df.1 -_do_restore_file_cum +_do_restore_file_cum -f $tmp.df.0 +_do_restore_file_cum -f $tmp.df.1 _ls_compare_sub _diff_compare diff --git a/common.dump b/common.dump index 09f1a91..552085f 100644 --- a/common.dump +++ b/common.dump @@ -889,6 +889,7 @@ _parse_args() -f) [ -z "$2" ] && _fail "missing argument for -f" dumptape=$2 + dump_file=$2 shift ;; -L) @@ -1080,6 +1081,8 @@ _do_restore_file_cum() _do_restore_toc() { + _parse_args $* + echo "Contents of dump ..." opts="$_restore_debug -f $dump_file -t" echo "xfsrestore $opts" | _dir_filter -- 1.7.0.4 From wkendall@sgi.com Tue Nov 1 14:53:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1JriNM129346 for ; Tue, 1 Nov 2011 14:53:44 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6CBAF30404E; Tue, 1 Nov 2011 12:53:44 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 5A5B0700075C; Tue, 1 Nov 2011 14:53:44 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 51BC120DF3BB; Tue, 1 Nov 2011 14:53:44 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 4/4] xfstests: refactor dump test argument parsing Date: Tue, 1 Nov 2011 14:53:07 -0500 Message-Id: <1320177187-18871-5-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320177187-18871-1-git-send-email-wkendall@sgi.com> References: <1320177187-18871-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The dump and restore helper functions all call the same _parse_args() function. xfsdump and xfsrestore have some common options, but others have different meanings/syntaxes or only exist in one program or the other. Split _parse_args() into dump and restore variants. Further, a test cannot pass most options to xfsrestore because many of the parsed args are not used by the restore helper functions. Change the helpers so that the parsed options are available (to be used in future tests). Also add a check for a missing --multi argument, and change a couple of callers to be consistent and use $* instead of "$@". Signed-off-by: Bill Kendall --- common.dump | 95 ++++++++++++++++++++++++++++++++++++++++++++--------------- 1 files changed, 71 insertions(+), 24 deletions(-) diff --git a/common.dump b/common.dump index 552085f..076517d 100644 --- a/common.dump +++ b/common.dump @@ -876,9 +876,10 @@ _dir_filter() } # +# Parse xfsdump arguments. # Note: requires a space between option letter and argument # -_parse_args() +_parse_dump_args() { OPTIND=0 dump_args="" @@ -897,10 +898,8 @@ _parse_args() session_label=$2 shift ;; - -o|-D|-F) - dump_args="$dump_args $1" - ;; --multi) + [ -z "$2" ] && _fail "missing argument for --multi" multi=$2 shift ;; @@ -910,6 +909,9 @@ _parse_args() --no-check-quota) do_quota_check=false ;; + -o|-D|-F) + dump_args="$dump_args $1" + ;; -l|-d) [ -z "$2" ] && _fail "missing argument for $1" dump_args="$dump_args $1$2" @@ -923,13 +925,55 @@ _parse_args() done } +# +# Parse xfsrestore arguments. +# Note: requires a space between option letter and argument +# +_parse_restore_args() +{ + OPTIND=0 + restore_args="" + while [ $# -gt 0 ] + do + case $1 + in + -f) + [ -z "$2" ] && _fail "missing argument for -f" + dumptape=$2 + dump_file=$2 + shift + ;; + -L) + [ -z "$2" ] && _fail "missing argument for -L" + session_label=$2 + shift + ;; + --multi) + [ -z "$2" ] && _fail "missing argument for --multi" + multi=$2 + shift + ;; + --check-quota) + do_quota_check=true + ;; + --no-check-quota) + do_quota_check=false + ;; + *) + _fail "invalid argument to common.dump function: $1" + ;; + esac + shift + done +} + # # Dump a subdir # _do_dump_sub() { - _parse_args $* + _parse_dump_args $* echo "Dumping to tape..." opts="$_dump_debug$dump_args -s $dump_sdir -f $dumptape -M $media_label -L $session_label $SCRATCH_MNT" @@ -942,7 +986,7 @@ _do_dump_sub() # _do_dump() { - _parse_args $* + _parse_dump_args $* echo "Dumping to tape..." opts="$_dump_debug$dump_args -f $dumptape -M $media_label -L $session_label $SCRATCH_MNT" @@ -956,7 +1000,7 @@ _do_dump() # _do_dump_min() { - _parse_args $* + _parse_dump_args $* echo "Dumping to tape..." onemeg=1048576 @@ -971,7 +1015,7 @@ _do_dump_min() # _do_dump_file() { - _parse_args $* + _parse_dump_args $* echo "Dumping to file..." opts="$_dump_debug$dump_args -f $dump_file -M $media_label -L $session_label $SCRATCH_MNT" @@ -984,7 +1028,7 @@ _do_dump_file() # _do_dump_multi_file() { - _parse_args "$@" + _parse_dump_args $* multi_args="" @@ -1025,12 +1069,11 @@ _prepare_restore() # _do_restore() { - _parse_args $* + _parse_restore_args $* _prepare_restore - echo "Restoring from tape..." - opts="$_restore_debug -f $dumptape -L $session_label $restore_dir" + opts="$_restore_debug$restore_args -f $dumptape -L $session_label $restore_dir" echo "xfsrestore $opts" | _dir_filter $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter } @@ -1040,12 +1083,12 @@ _do_restore() # _do_restore_min() { - _parse_args $* + _parse_restore_args $* _prepare_restore echo "Restoring from tape..." onemeg=1048576 - opts="$_restore_debug -m -b $onemeg -f $dumptape -L $session_label $restore_dir" + opts="$_restore_debug$restore_args -m -b $onemeg -f $dumptape -L $session_label $restore_dir" echo "xfsrestore $opts" | _dir_filter $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter } @@ -1055,11 +1098,11 @@ _do_restore_min() # _do_restore_file() { - _parse_args $* + _parse_restore_args $* _prepare_restore_dir echo "Restoring from file..." - opts="$_restore_debug -f $dump_file -L $session_label $restore_dir" + opts="$_restore_debug$restore_args -f $dump_file -L $session_label $restore_dir" echo "xfsrestore $opts" | _dir_filter $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter } @@ -1071,20 +1114,20 @@ _do_restore_file() # _do_restore_file_cum() { - _parse_args $* + _parse_restore_args $* echo "Restoring cumumlative from file..." - opts="$_restore_debug -f $dump_file -r $restore_dir" + opts="$_restore_debug$restore_args -f $dump_file -r $restore_dir" echo "xfsrestore $opts" | _dir_filter $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter } _do_restore_toc() { - _parse_args $* + _parse_restore_args $* echo "Contents of dump ..." - opts="$_restore_debug -f $dump_file -t" + opts="$_restore_debug$restore_args -f $dump_file -t" echo "xfsrestore $opts" | _dir_filter cd $SCRATCH_MNT # for IRIX which needs xfs cwd $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter_main |\ @@ -1103,7 +1146,7 @@ _do_restore_toc() # _do_restore_multi_file() { - _parse_args "$@" + _parse_restore_args $* _prepare_restore_dir multi_args="" @@ -1116,22 +1159,26 @@ _do_restore_multi_file() done echo "Restoring from file..." - opts="$_restore_debug $multi_args -L $session_label $restore_dir" + opts="$_restore_debug$restore_args $multi_args -L $session_label $restore_dir" echo "xfsrestore $opts" | _dir_filter $XFSRESTORE_PROG $opts 2>&1 | tee -a $here/$seq.full | _dump_filter } # # Do xfsdump piped into xfsrestore - xfsdump | xfsrestore +# Pass dump options in $1 and restore options in $2, if required. e.g.: +# _do_dump_restore "-o -F" "-R" +# _do_dump_restore "" "-R" # # Use -s as we want to dump and restore to the same xfs partition # _do_dump_restore() { - _parse_args $* + _parse_dump_args $1 + _parse_restore_args $2 _prepare_restore_dir echo "xfsdump|xfsrestore ..." - restore_opts="$_restore_debug - $restore_dir" + restore_opts="$_restore_debug$restore_args - $restore_dir" dump_opts="$_dump_debug$dump_args -s $dump_sdir - $SCRATCH_MNT" echo "xfsdump $dump_opts | xfsrestore $restore_opts" | _dir_filter $XFSDUMP_PROG $dump_opts 2>$tmp.dump.mlog | $XFSRESTORE_PROG $restore_opts 2>&1 | tee -a $here/$seq.full | _dump_filter -- 1.7.0.4 From wkendall@sgi.com Tue Nov 1 15:30:49 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA1KUm0Q130222 for ; Tue, 1 Nov 2011 15:30:49 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id AD35430404E; Tue, 1 Nov 2011 13:30:45 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 8F22E700075D; Tue, 1 Nov 2011 15:30:45 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 4D1C220DF3BB; Tue, 1 Nov 2011 15:30:45 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2] xfsdump: fix metadata restore on split files Date: Tue, 1 Nov 2011 15:30:43 -0500 Message-Id: <1320179443-21235-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean xfsrestore does not apply certain metadata until all of the file's data has been restored. This allows, for example, files with the immutable flag set to be restored properly. While testing multi-stream restores, I noticed that files split across multiple streams did not have their metadata restored. Looking into this further, it also can occur with just a single stream when the dump spans multiple tapes. Specifically it requires that end-of-tape is hit just as a file is finished (before a new file is begun), and the restore is performed as separate xfsrestore invocations (one per tape). The fix is to check to see if a file is completely restored whenever we hit the end of a media file. The current code is broken because it relies on all media files being applied during the same restore session. This also moves the S_ISREG() check into restore_complete_reg() rather than relying on callers to make the check. Signed-off-by: Bill Kendall --- Changed for v2: - Reworded commit message to correctly characterize the existing bug. restore/content.c | 50 +++++++++++++++++++++++++++++++++----------------- 1 files changed, 33 insertions(+), 17 deletions(-) diff --git a/restore/content.c b/restore/content.c index e5957bf..34fc4a0 100644 --- a/restore/content.c +++ b/restore/content.c @@ -2499,12 +2499,6 @@ content_stream_restore( ix_t thrdix ) } } - /* out of media files, so finish the last file that - * was being worked on. - */ - if ((strctxp->sc_bstat.bs_mode & S_IFMT) == S_IFREG) - restore_complete_reg(strctxp); - /* finally, choose one thread to do final processing * and cleanup. the winner waits, the losers all exit. * once the losers exit, the winner can perform cleanup. @@ -3337,6 +3331,7 @@ applynondirdump( drive_t *drivep, char *path2, filehdr_t *fhdrp ) { + rv_t rv = RV_UNKNOWN; bool_t fhcs; bool_t ehcs; bool_t ahcs; @@ -3367,18 +3362,24 @@ applynondirdump( drive_t *drivep, */ pi_bracketneededegrps( fileh, &first_egrp, &next_egrp ); + /* initialize the stream context + */ + memset(&strctxp->sc_bstat, 0, sizeof(bstat_t)); + strctxp->sc_path[0] = '\0'; + strctxp->sc_fd = -1; + for ( ; ; ) { drive_ops_t *dop = drivep->d_opsp; drive_mark_t drivemark; bstat_t *bstatp = &fhdrp->fh_stat; bool_t resyncpr = BOOL_FALSE; - rv_t rv; intgen_t rval; /* if a null file header, break */ if ( fhdrp->fh_flags & FILEHDR_FLAGS_NULL ) { - break; + rv = RV_OK; + goto applynondirdump_out; } /* if working on a different file than we were previously, @@ -3386,8 +3387,7 @@ applynondirdump( drive_t *drivep, */ if ( bstatp->bs_ino != strctxp->sc_bstat.bs_ino ) { - if ((strctxp->sc_bstat.bs_mode & S_IFMT) == S_IFREG) - restore_complete_reg(strctxp); + restore_complete_reg(strctxp); /* start new ino */ memcpy(&strctxp->sc_bstat, bstatp, sizeof(bstat_t)); @@ -3418,7 +3418,8 @@ applynondirdump( drive_t *drivep, case RV_OK: break; case RV_EOD: - return RV_OK; + rv = RV_OK; + goto applynondirdump_out; case RV_CORRUPT: rval = ( * dop->do_next_mark )( drivep ); if ( rval ) { @@ -3426,12 +3427,13 @@ applynondirdump( drive_t *drivep, "unable to resync media file: " "some portion of dump will NOT " "be restored\n") ); - return RV_OK; /* treat as EOD */ + rv = RV_OK; /* treat as EOD */ + goto applynondirdump_out; } resyncpr = BOOL_TRUE; break; default: - return rv; + goto applynondirdump_out; } /* update stats if appropriate @@ -3471,7 +3473,8 @@ applynondirdump( drive_t *drivep, case RV_OK: break; case RV_EOD: - return RV_OK; + rv = RV_OK; + goto applynondirdump_out; case RV_CORRUPT: rval = ( * dop->do_next_mark )( drivep ); if ( rval ) { @@ -3479,11 +3482,12 @@ applynondirdump( drive_t *drivep, "unable to resync media file: " "some portion of dump will NOT " "be restored\n") ); - return RV_OK; /* treat as EOD */ + rv = RV_OK; /* treat as EOD */ + goto applynondirdump_out; } resyncpr = BOOL_TRUE; default: - return rv; + goto applynondirdump_out; } if ( resyncpr && rv == RV_OK ) { @@ -3515,7 +3519,16 @@ applynondirdump( drive_t *drivep, preemptchk( ); } } - return RV_OK; + +applynondirdump_out: + + /* We've hit the end of this media file or encountered corruption. + * In either case, we may not be back to complete the metadata for + * this file, so attempt to complete it now. + */ + restore_complete_reg(strctxp); + + return rv; } /* ARGSUSED */ @@ -7507,6 +7520,9 @@ restore_complete_reg(stream_context_t *strcxtp) struct utimbuf utimbuf; intgen_t rval; + // only applies to regular files + if (!S_ISREG((strcxtp->sc_bstat.bs_mode))) + return BOOL_TRUE; if (fd < 0) return BOOL_TRUE; -- 1.7.0.4 From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 04:10:35 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA29AZti164371 for ; Wed, 2 Nov 2011 04:10:35 -0500 X-ASG-Debug-ID: 1320225031-2c3703d00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 38459217D3F for ; Wed, 2 Nov 2011 02:10:32 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id IEK7JmnDhikSaMVX for ; Wed, 02 Nov 2011 02:10:32 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLWpo-0007ej-5A; Wed, 02 Nov 2011 09:10:12 +0000 Date: Wed, 2 Nov 2011 05:10:12 -0400 From: Christoph Hellwig To: Wu Fengguang Cc: "linux-fsdevel@vger.kernel.org" , Peter Zijlstra , Jan Kara , Curt Wohlgemuth , Michael Rubin , Andrew Morton , Christoph Hellwig , Dave Chinner , Greg Thelen , Minchan Kim , Vivek Goyal , Andrea Righi , linux-mm , LKML , Tang Feng , Linus Torvalds , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: writeback tree status (for 3.2 merge window) Subject: Re: writeback tree status (for 3.2 merge window) Message-ID: <20111102091012.GA25879@infradead.org> References: <20111101074347.GA23644@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111101074347.GA23644@localhost> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320225032 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79093 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Nov 01, 2011 at 03:43:47PM +0800, Wu Fengguang wrote: > Hi, > > There are 3 patchsets sitting in the writeback tree. > > 1) IO-less dirty throttling v12 > https://github.com/fengguang/linux/commits/dirty-throttling-v12 > > 2) writeback reasons tracing from Curt Wohlgemuth > https://github.com/fengguang/linux/commits/writeback-reason > > 3) writeback queuing changes from Jan Kara and me > https://github.com/fengguang/linux/commits/requeue-io-wait > > They have been merged into this branch testing in linux-next for a while: > > https://github.com/fengguang/linux/commits/writeback-for-next > > Since (3) still has an unresolved issue (detailed in the below > links), it looks better to hold it back for this merge window. Given that we have run into issues so much with it that's proably the better idea. To fix the originally reported issue with the missing file size updates in XFS we plan to simply log all filesize changes in XFS. Currently I had this in the 3.3 queue, but in the worst case we might have to move this forward to 3.2. > The patches from (1,2) together with 2 tracing patches essential for > debugging (1) have been pushed to the "writeback-for-linus" branch: > > http://git.kernel.org/?p=linux/kernel/git/wfg/linux.git;a=shortlog;h=refs/heads/writeback-for-linus > > If no objections, I'll send a pull request to Linus soon. Please do so. I'm really looking forward to see the I/O-less balance_dirty_pages in ASAP. From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 04:20:17 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA29KGJt164688 for ; Wed, 2 Nov 2011 04:20:17 -0500 X-ASG-Debug-ID: 1320225615-6628033f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D6B66217A96 for ; Wed, 2 Nov 2011 02:20:15 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id JXRVIcXIlTHotmZr for ; Wed, 02 Nov 2011 02:20:15 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLWzU-0001qs-Ur; Wed, 02 Nov 2011 09:20:12 +0000 Date: Wed, 2 Nov 2011 05:20:12 -0400 From: Christoph Hellwig To: xfs@oss.sgi.com Cc: Jakub Bogusz X-ASG-Orig-Subj: [PATCH v2] xfsprogs: fix various incorrect printf formats Subject: [PATCH v2] xfsprogs: fix various incorrect printf formats Message-ID: <20111102092012.GA3808@infradead.org> References: <20111025115233.GB22103@infradead.org> <20111025155916.GA19818@stranger.qboosh.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111025155916.GA19818@stranger.qboosh.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320225615 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79095 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Reported-by: Jakub Bogusz Signed-off-by: Christoph Hellwig Index: xfsprogs/libxfs/freebsd.c =================================================================== --- xfsprogs.orig/libxfs/freebsd.c 2011-10-30 05:56:36.180125966 +0100 +++ xfsprogs/libxfs/freebsd.c 2011-10-30 05:57:45.957126722 +0100 @@ -126,15 +126,13 @@ platform_findsizes(char *path, int fd, l } if ((st.st_mode & S_IFMT) != S_IFCHR) { - fprintf(stderr, _("%s: " - "Not a device or file: \"%s\"n"), + fprintf(stderr, _("%s: Not a device or file: \"%s\"\n"), progname, path); exit(1); } if (ioctl(fd, DIOCGMEDIASIZE, &size) != 0) { - fprintf(stderr, _("%s: " - "DIOCGMEDIASIZE failed on \"%s\": %s\n"), + fprintf(stderr, _("%s: DIOCGMEDIASIZE failed on \"%s\": %s\n"), progname, path, strerror(errno)); exit(1); } Index: xfsprogs/repair/dinode.c =================================================================== --- xfsprogs.orig/repair/dinode.c 2011-10-30 05:56:36.132124261 +0100 +++ xfsprogs/repair/dinode.c 2011-10-30 05:57:45.957126722 +0100 @@ -1439,7 +1439,7 @@ _("mismatch between format (%d) and size } if (dino->di_format == XFS_DINODE_FMT_LOCAL) { do_warn( -_("mismatch between format (%d) and size (%" PRId64 ") in symlink inode %" PRIu64 "n"), +_("mismatch between format (%d) and size (%" PRId64 ") in symlink inode %" PRIu64 "\n"), dino->di_format, (__int64_t)be64_to_cpu(dino->di_size), lino); return 1; Index: xfsprogs/repair/dir2.c =================================================================== --- xfsprogs.orig/repair/dir2.c 2011-10-30 05:56:36.156126568 +0100 +++ xfsprogs/repair/dir2.c 2011-11-02 10:16:46.593593447 +0100 @@ -1174,7 +1174,7 @@ _("would have corrected entry count in d ino, sfp->hdr.count, i); } else { do_warn( -_("corrected entry count in directory %" PRIu64 "u, was %d, now %d\n"), +_("corrected entry count in directory %" PRIu64 ", was %d, now %d\n"), ino, sfp->hdr.count, i); sfp->hdr.count = i; *dino_dirty = 1; Index: xfsprogs/repair/phase6.c =================================================================== --- xfsprogs.orig/repair/phase6.c 2011-10-30 05:56:36.144127454 +0100 +++ xfsprogs/repair/phase6.c 2011-10-30 05:57:45.973171205 +0100 @@ -2071,7 +2071,7 @@ longform_dir2_entry_check_data( if (is_inode_free(irec, ino_offset)) { nbad++; if (entry_junked( - _("entry \"%s\" in directory inode %" PRIu64 " points to free inode " PRIu64), + _("entry \"%s\" in directory inode %" PRIu64 " points to free inode %" PRIu64), fname, ip->i_ino, inum)) { dep->name[0] = '/'; libxfs_dir2_data_log_entry(tp, bp, dep); Index: xfsprogs/repair/scan.c =================================================================== --- xfsprogs.orig/repair/scan.c 2011-10-30 05:56:36.168127218 +0100 +++ xfsprogs/repair/scan.c 2011-10-30 05:57:45.981169953 +0100 @@ -289,14 +289,14 @@ _("bad back (left) sibling pointer (saw */ set_bmap(agno, agbno, XR_E_MULT); do_warn( -_("inode 0x%" PRIu64 "bmap block 0x%" PRIu64 " claimed, state is %d\n"), +_("inode 0x%" PRIu64 "bmap block 0x%" PRIx64 " claimed, state is %d\n"), ino, bno, state); break; case XR_E_MULT: case XR_E_INUSE_FS: set_bmap(agno, agbno, XR_E_MULT); do_warn( -_("inode 0x%" PRIu64 " bmap block 0x%" PRIu64 " claimed, state is %d\n"), +_("inode 0x%" PRIu64 " bmap block 0x%" PRIx64 " claimed, state is %d\n"), ino, bno, state); /* * if we made it to here, this is probably a bmap block @@ -311,7 +311,7 @@ _("inode 0x%" PRIu64 " bmap block 0x%" P case XR_E_BAD_STATE: default: do_warn( -_("bad state %d, inode 0x%" PRIu64 " bmap block 0x%" PRIu64 "\n"), +_("bad state %d, inode 0x%" PRIu64 " bmap block 0x%" PRIx64 "\n"), state, ino, bno); break; } @@ -338,7 +338,7 @@ _("bad state %d, inode 0x%" PRIu64 " bma if (numrecs > mp->m_bmap_dmxr[0] || (isroot == 0 && numrecs < mp->m_bmap_dmnr[0])) { do_warn( -_("inode 0x%" PRIu64 " bad # of bmap records (%u, min - %u, max - %u)\n"), +_("inode %" PRIu64 " bad # of bmap records (%u, min - %u, max - %u)\n"), ino, numrecs, mp->m_bmap_dmnr[0], mp->m_bmap_dmxr[0]); return(1); From fengguang.wu@intel.com Wed Nov 2 10:14:06 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2FE6HE179300 for ; Wed, 2 Nov 2011 10:14:06 -0500 X-ASG-Debug-ID: 1320246845-71b503bc0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mga14.intel.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 90F191F2D111 for ; Wed, 2 Nov 2011 08:14:05 -0700 (PDT) Received: from mga14.intel.com (mga14.intel.com [143.182.124.37]) by cuda.sgi.com with ESMTP id Dp8fHI6CW55XnJ2S for ; Wed, 02 Nov 2011 08:14:05 -0700 (PDT) Received: from azsmga001.ch.intel.com ([10.2.17.19]) by azsmga102.ch.intel.com with ESMTP; 02 Nov 2011 08:14:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.69,444,1315206000"; d="scan'208";a="69754244" Received: from unknown (HELO localhost.localdomain) ([10.255.21.180]) by azsmga001.ch.intel.com with ESMTP; 02 Nov 2011 08:13:57 -0700 Received: from wfg by localhost.localdomain with local (Exim 4.69) (envelope-from ) id 1RLcVn-0001zQ-HE; Wed, 02 Nov 2011 23:13:55 +0800 Date: Wed, 2 Nov 2011 23:13:55 +0800 From: Wu Fengguang To: Christoph Hellwig Cc: "linux-fsdevel@vger.kernel.org" , Peter Zijlstra , Jan Kara , Curt Wohlgemuth , Michael Rubin , Andrew Morton , Christoph Hellwig , Dave Chinner , Greg Thelen , Minchan Kim , Vivek Goyal , Andrea Righi , linux-mm , LKML , Tang Feng , Linus Torvalds , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: writeback tree status (for 3.2 merge window) Subject: Re: writeback tree status (for 3.2 merge window) Message-ID: <20111102151355.GA6717@localhost> References: <20111101074347.GA23644@localhost> <20111102091012.GA25879@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111102091012.GA25879@infradead.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Barracuda-Connect: mga14.intel.com[143.182.124.37] X-Barracuda-Start-Time: 1320246845 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79117 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 02, 2011 at 05:10:12AM -0400, Christoph Hellwig wrote: > On Tue, Nov 01, 2011 at 03:43:47PM +0800, Wu Fengguang wrote: > > Hi, > > > > There are 3 patchsets sitting in the writeback tree. > > > > 1) IO-less dirty throttling v12 > > https://github.com/fengguang/linux/commits/dirty-throttling-v12 > > > > 2) writeback reasons tracing from Curt Wohlgemuth > > https://github.com/fengguang/linux/commits/writeback-reason > > > > 3) writeback queuing changes from Jan Kara and me > > https://github.com/fengguang/linux/commits/requeue-io-wait > > > > They have been merged into this branch testing in linux-next for a while: > > > > https://github.com/fengguang/linux/commits/writeback-for-next > > > > Since (3) still has an unresolved issue (detailed in the below > > links), it looks better to hold it back for this merge window. > > Given that we have run into issues so much with it that's proably > the better idea. > > To fix the originally reported issue with the missing file size updates > in XFS we plan to simply log all filesize changes in XFS. Currently I > had this in the 3.3 queue, but in the worst case we might have to move > this forward to 3.2. OK. Anyway I'll report any progress early on (3). > > The patches from (1,2) together with 2 tracing patches essential for > > debugging (1) have been pushed to the "writeback-for-linus" branch: > > > > http://git.kernel.org/?p=linux/kernel/git/wfg/linux.git;a=shortlog;h=refs/heads/writeback-for-linus > > > > If no objections, I'll send a pull request to Linus soon. > > Please do so. I'm really looking forward to see the I/O-less > balance_dirty_pages in ASAP. Done, thanks! Thanks, Fengguang From aelder@sgi.com Wed Nov 2 12:52:24 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2HqNaF189973 for ; Wed, 2 Nov 2011 12:52:23 -0500 Received: from xmail.sgi.com (pv-excas1-dc21-nlb.corp.sgi.com [137.38.102.126]) by relay1.corp.sgi.com (Postfix) with ESMTP id 686958F80E1; Wed, 2 Nov 2011 10:52:20 -0700 (PDT) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Wed, 2 Nov 2011 12:52:20 -0500 Message-ID: <1320256339.3145.30.camel@doink> Subject: Re: [PATCH] Fix possible memory corruption in xfs_readlink From: Alex Elder Reply-To: To: Ben Hutchings CC: Carlos Maiolino , Date: Wed, 2 Nov 2011 12:52:19 -0500 In-Reply-To: <1320156842.30281.28.camel@deadeye> References: <1320156842.30281.28.camel@deadeye> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I don't konw why, but I *think* the response I thought I sent earlier didn't actually make it out. Just in case, I'm trying to recreate what I had before, below. Sorry if something like this shows up twice. On Tue, 2011-11-01 at 14:14 +0000, Ben Hutchings wrote: > On Tue, 2011-10-18 at 02:18 -0200, Carlos Maiolino wrote: > > Fixes a possible memory corruption when the link is larger than > > MAXPATHLEN and XFS_DEBUG is not enabled. This also remove the > > S_ISLNK assert, since the inode mode is checked previously in > > xfs_readlink_by_handle() and via VFS. > > > > Signed-off-by: Carlos Maiolino > > --- > > fs/xfs/xfs_vnodeops.c | 11 ++++++++--- > > 1 files changed, 8 insertions(+), 3 deletions(-) A few comments inline below, followed by Ben's original message and some explanation from me. > > diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c > > index 51fc429..c3288be 100644 > > --- a/fs/xfs/xfs_vnodeops.c > > +++ b/fs/xfs/xfs_vnodeops.c > > @@ -123,13 +123,18 @@ xfs_readlink( > > > > xfs_ilock(ip, XFS_ILOCK_SHARED); > > > > - ASSERT(S_ISLNK(ip->i_d.di_mode)); > > - ASSERT(ip->i_d.di_size <= MAXPATHLEN); > > - > > pathlen = ip->i_d.di_size; > pathlen is a signed int (32-bit) and di_size has signed 64-bit type. I concur, di_size here is an xfs_fsize_t, which is defined as __int64_t (a signed 64-bit integer). pathlen is defined as a (signed) int. > So, even if di_size was verified to be non-negative earlier (is it?)... More on this question below. > > if (!pathlen) > > goto out; > > > > + if (pathlen > MAXPATHLEN) { > > ...pathlen may be negative here and will pass this check. > > Ben. You are right to call attention to this. I think defining pathlen to be an int here is a mistake in any case (the type ought to match that of id.di_size), though in practice it will not be a problem. You mention two remaining issues: - can a value held in ip->i_d.di_size result in a negative value in pathlen as a result of the assignment? - is ip->i_d.di_size guaranteed (verified) to be non-negative? On the first question, the C standard says that the result of the assignment--if id.di_size exceeds what can be represented by pathlen--is implementation defined, therefore it is not safe. So you're right, this needs to be fixed. On the second question, ip->i_d.di_size is assigned in a lot of places. I started looking at all the places where this field gets assigned. In about half of them I examined the assignment obviously left its value non-negative, or only allowing a negative assignment if the previous value was already negative. But rather than complete this research task, I think it will be better (for now) to simply check for a negative ip->i_d.di_size, and if it's seen, either return an error or initiate a forced shutdown (since it represents corruption). I'm interested in what others think. -Alex > > + xfs_alert(mp, "%s: inode (%llu) symlink length (%d) too long", > > + __func__, (unsigned long long)ip->i_ino, pathlen); > > + ASSERT(0); > > + return XFS_ERROR(EFSCORRUPTED); > > + } > > + > > + > > if (ip->i_df.if_flags & XFS_IFINLINE) { > > memcpy(link, ip->i_df.if_u1.if_data, pathlen); > > link[pathlen] = '\0'; > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From www-data@techwebex.com Wed Nov 2 13:16:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2IGerv191140 for ; Wed, 2 Nov 2011 13:16:41 -0500 X-ASG-Debug-ID: 1320257794-615f004e0000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from server.techwebex.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 272B612FE294 for ; Wed, 2 Nov 2011 11:16:35 -0700 (PDT) Received: from server.techwebex.com (static.10.106.40.188.clients.your-server.de [188.40.106.10]) by cuda.sgi.com with ESMTP id sWQfJIzodeDhkYeP for ; Wed, 02 Nov 2011 11:16:35 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by server.techwebex.com (Postfix) with ESMTP id 47D5523DC6B8 for ; Thu, 3 Nov 2011 04:16:34 +1000 (EST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: by amavisd-new at techwebex.com Received: from server.techwebex.com ([127.0.0.1]) by localhost (server.techwebex.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id whTEoSn0M0h7 for ; Thu, 3 Nov 2011 04:16:33 +1000 (EST) Received: from localhost (localhost [127.0.0.1]) by server.techwebex.com (Postfix) with ESMTP id BDFDF23DC6BD for ; Thu, 3 Nov 2011 04:16:33 +1000 (EST) Received: from server.techwebex.com (localhost [127.0.0.1]) by server.techwebex.com (Postfix) with ESMTP id A9D4723DC6B8 for ; Thu, 3 Nov 2011 04:16:33 +1000 (EST) Received: (from www-data@localhost) by server.techwebex.com (8.14.3/8.14.3/Submit) id pA2IGXWw032145; Thu, 3 Nov 2011 04:16:33 +1000 To: linux-xfs@oss.sgi.com X-ASG-Orig-Subj: world wide gaming even when you lose you win Subject: world wide gaming even when you lose you win Date: Thu, 3 Nov 2011 04:16:33 +1000 From: admin Message-ID: X-Priority: 3 X-Mailer: PHPMailer [version 1.73] X-Mailer: phplist v2.10.17 X-MessageID: 82 X-ListMember: linux-xfs@oss.sgi.com MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8" X-Barracuda-Connect: static.10.106.40.188.clients.your-server.de[188.40.106.10] X-Barracuda-Start-Time: 1320257796 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4999 1.0000 0.0000 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_SA210e X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79130 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC5_SA210e Custom Rule SA210e X-Virus-Status: Clean Fellow Seekers, I would like to share with my friends something that is Unique. If you would like to take a look at this, you will see benefit from taking up a position. I have personally taken a position within Capitol Gaming. After having numerous conversations with the International lead distributor in Europe and, who is currently now in the U.S.A,I am convinced that those who join in now, will reap the rewards from the influx of people, who will be coming in after the launch. The member’s potential income will be huge in this; I am now choosing to take what I have heard, from the top International online business builders. All the work that has gone into this venture and with the association Capitol Gaming has with Betfair.com, the unique method employed here, will maximize and bring the greatest and best potential offered to anybody seeking an online home business opportunity. One must consider that there is also massive use of online gaming these days. Taking a position at this early stage, and to be in my group near the top, has to be a winner! The cost is $49 lifetime membership and $105 for player membership (if you wish, you can upgrade later to a premium membership) I cannot say what individuals will earn, but I have been told that early positions could be paying thousands of dollars in the first few months. This industry has a massive turnover, and being part of it gives us big potential far exceeding any initial cost! More information will be coming out; I would suggest that you seriously consider taking a position within my leadership team www.capitolgaming.com/info Fellow Seekers ,timing is important as this is a once only opportunity to be placed ahead of many sporting clubs and associations that we will be benefiting from. The official launch is scheduled for November this year, be a part of my team now to maximize your potential. The best bet on the net is here now, join the top members now in soft launch, and give yourself a head start now to immediate success!!! This is a trillion dollar industry - why not grab a part of it?? • By signing now, the leaders can place you in the current best position to establish your future Income potential • Future income will roll up to you based on the activity of those below you that you bring in or WE place in via Spill over • The leaders will be placing many Clubs, Pubs and Sporting Associations into Capitol Gaming on an ongoing basis as the company expands globally. This will assist the growth of your business and YOUR income. • Your income will be generated from the points accumulated by those betting within your business. A portion of every wager placed goes into Capitol Gaming from your down line and into your account. So if they Win, Lose or Draw you make money • So getting in NOW, and being placed correctly for maximum income potential, is highly recommended. Our team will assist you in meeting the qualifications to achieve and maintain an ongoing income. As I mentioned earlier in this email regarding the International Lead Distributor, he is presently traveling throughout the U.S.A and CANADA during the next few weeks promoting Capitol Gaming. If you or anybody is interested in attending an information session, please do not hesitate contacting me for details or to see if there are any sessions planned for your area Call or email me after signing in so we can make the next moves! Best Wishes, Ian Daley 29 Baileys Rd Tallai Queensland 4213 Australia 61 421 380 302 -- If you do not want to receive any more e mails follow use this link to UNSUBSCRIBE, http://www.health-essential-oils.net/lists/?p=unsubscribe&uid=aeca879698207b3bba99098240cad5e8 -- powered by phpList, www.phplist.com -- From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 14:45:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2JjC7G196428 for ; Wed, 2 Nov 2011 14:45:14 -0500 X-ASG-Debug-ID: 1320263109-5b9201800000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A3E40FE47DB; Wed, 2 Nov 2011 12:45:09 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id HoP8bV3FlQ7h29ep; Wed, 02 Nov 2011 12:45:09 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLgkF-0004eV-JT; Wed, 02 Nov 2011 19:45:07 +0000 Date: Wed, 2 Nov 2011 15:45:07 -0400 From: Christoph Hellwig To: Alex Elder Cc: Ben Hutchings , Carlos Maiolino , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Fix possible memory corruption in xfs_readlink Subject: Re: [PATCH] Fix possible memory corruption in xfs_readlink Message-ID: <20111102194507.GA14429@infradead.org> References: <1320156842.30281.28.camel@deadeye> <1320256339.3145.30.camel@doink> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320256339.3145.30.camel@doink> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320263110 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79136 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We should validate that the value isn't negative in xfs_iformat_*, although we currently don't do that. It already verified that it fits into the XFS_DFORK_DSIZE, which should take care of fitting into 32-bits. Adding another explicit check probably won't hurt, given that XFS_DFORK_DSIZE is calculated dynamically based on the fork offset. From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 14:54:02 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2Js1ih196780 for ; Wed, 2 Nov 2011 14:54:02 -0500 X-ASG-Debug-ID: 1320263640-134d00ee0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D33E412FECAA; Wed, 2 Nov 2011 12:54:00 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id mlp2EIq0pVVoCqjE; Wed, 02 Nov 2011 12:54:00 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLgso-0005r8-0k; Wed, 02 Nov 2011 19:53:58 +0000 Date: Wed, 2 Nov 2011 15:53:58 -0400 From: Christoph Hellwig To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111102195357.GA22500@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-8-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1319849297-3506-8-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320263640 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79136 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Oct 29, 2011 at 04:48:16AM +0400, Dmitry Monakhov wrote: > During stress testing we want to cover as much code paths as possible > fsstress is very good for this purpose. But it has expandable nature > (disk usage almost continually grow). So once it goes in no ENOSPC > condition it will be where till the end. But by running 'dd' writers > in parallel we can regularly trigger ENOSPC but only for a limited > periods of time because each time it opens the same file with O_TRUNC. This fails for me on XFS because the fsstress process has already finished by the time you try to kill it: --- 266.out 2011-11-02 19:46:27.000000000 +0000 +++ 266.out.bad 2011-11-02 19:48:33.000000000 +0000 @@ -3,3 +3,4 @@ Run fsstress Run dd writers in parallel +./266: line 60: kill: (3403) - No such process From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 14:55:37 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2Jtbjh196956 for ; Wed, 2 Nov 2011 14:55:37 -0500 X-ASG-Debug-ID: 1320263736-4acc02fd0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7E7B31CDB20C; Wed, 2 Nov 2011 12:55:36 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id Rk7VnYGp0lmOjWm1; Wed, 02 Nov 2011 12:55:36 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLguM-0006j1-Ux; Wed, 02 Nov 2011 19:55:35 +0000 Date: Wed, 2 Nov 2011 15:55:34 -0400 From: Christoph Hellwig To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Message-ID: <20111102195534.GB22500@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320263736 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79137 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Oct 29, 2011 at 04:48:15AM +0400, Dmitry Monakhov wrote: > Related bug: http://patchwork.ozlabs.org/patch/118863 > > Signed-off-by: Dmitry Monakhov This causes a very weird XFS failure in test 117 for me. While it obviously is an xfs bug that you uncovered (good!) I'm a bit worried about simply enabling operations in existing tests. Alex, Eric, Dave - should we add new tests with the new operations Dmitry added, or is adding new ops to the existing tests fine? From aelder@sgi.com Wed Nov 2 15:22:40 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2KMdii198195 for ; Wed, 2 Nov 2011 15:22:40 -0500 Received: from xmail.sgi.com (pv-excas1-dc21-nlb.corp.sgi.com [137.38.102.126]) by relay2.corp.sgi.com (Postfix) with ESMTP id 770DD30408B; Wed, 2 Nov 2011 13:22:36 -0700 (PDT) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Wed, 2 Nov 2011 15:22:35 -0500 Message-ID: <1320265355.3145.53.camel@doink> Subject: Re: [PATCH] Fix possible memory corruption in xfs_readlink From: Alex Elder Reply-To: To: Christoph Hellwig CC: Ben Hutchings , Carlos Maiolino , Date: Wed, 2 Nov 2011 15:22:35 -0500 In-Reply-To: <20111102194507.GA14429@infradead.org> References: <1320156842.30281.28.camel@deadeye> <1320256339.3145.30.camel@doink> <20111102194507.GA14429@infradead.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2011-11-02 at 15:45 -0400, Christoph Hellwig wrote: > We should validate that the value isn't negative in xfs_iformat_*, > although we currently don't do that. It already verified that it > fits into the XFS_DFORK_DSIZE, which should take care of fitting > into 32-bits. Adding another explicit check probably won't hurt, > given that XFS_DFORK_DSIZE is calculated dynamically based on the > fork offset. > That's true, but there are other places where it gets updated, yet not defensively validated. For example, in xfs_dir2_shrink_inode(), if: fsbno > (INT64_MAX >> mp->m_sb.sb_blocklog) then the (signed) di_size field would be assigned a value that exceeded its max representable value, producing unreliable (implementation-defined) results. That may well be an impossible situation, but it's not obvious without really looking at the code. It's a bit of a can of worms, which is why I suggested just testing for this (unlikely) condition in xfs_readlink() for now. -Alex From rjevskiy@gmail.com Wed Nov 2 15:36:45 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_73, T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2KajjR198718 for ; Wed, 2 Nov 2011 15:36:45 -0500 X-ASG-Debug-ID: 1320266203-4b1003700000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0959421A3AC for ; Wed, 2 Nov 2011 13:36:44 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id WMMgg1hzfAZdf6XE for ; Wed, 02 Nov 2011 13:36:44 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so630856bkb.26 for ; Wed, 02 Nov 2011 13:36:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=YZN8P/gBJ+hFaEgPcx3qYVMTznt2+APK401oteAqB/A=; b=QVtBgVscyYU3NYUyevy9db4R4Etov9YR9fA/8J6t5jFkKTNldYOh7xl2A+t+Quhxc2 T8dKyb2AtPGG2gU5P56FMJhF2Vfdt/mD59pm6J8Wwv3dSJ8OGivzCJ/owySxt5ahjNFr tKsIqaHuya7fTI4IQs17T0UG2NM/h2ofBpF3w= Received: by 10.204.16.67 with SMTP id n3mr5249503bka.6.1320266203263; Wed, 02 Nov 2011 13:36:43 -0700 (PDT) Received: from smtp.gmail.com ([178.176.102.57]) by mx.google.com with ESMTPS id a27sm3416047bku.9.2011.11.02.13.36.38 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 02 Nov 2011 13:36:41 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Christoph Hellwig Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions In-Reply-To: <20111102195357.GA22500@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-8-git-send-email-dmonakhov@openvz.org> <20111102195357.GA22500@infradead.org> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Thu, 03 Nov 2011 00:36:36 +0400 Message-ID: <8739e6z2gb.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320266205 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0014 1.0000 -2.0120 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79139 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2 Nov 2011 15:53:58 -0400, Christoph Hellwig wrote: > On Sat, Oct 29, 2011 at 04:48:16AM +0400, Dmitry Monakhov wrote: > > During stress testing we want to cover as much code paths as possible > > fsstress is very good for this purpose. But it has expandable nature > > (disk usage almost continually grow). So once it goes in no ENOSPC > > condition it will be where till the end. But by running 'dd' writers > > in parallel we can regularly trigger ENOSPC but only for a limited > > periods of time because each time it opens the same file with O_TRUNC. > > This fails for me on XFS because the fsstress process has already > finished by the time you try to kill it: > > --- 266.out 2011-11-02 19:46:27.000000000 +0000 > +++ 266.out.bad 2011-11-02 19:48:33.000000000 +0000 > @@ -3,3 +3,4 @@ > Run fsstress > > Run dd writers in parallel > +./266: line 60: kill: (3403) - No such process Strange... this means that parent process died unexpectedly. Works fine fore me. In my case this happen once due to unknown options(new opt " -fsetattr=1" passed to fsstress explicitly) because i've forget to rebuilt fsstress via make. Can you please explicitly rebuild, and check one more time to be 100% sure. > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevecalll" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From BATV+572a843891b4952d1fe5+2992+infradead.org+hch@bombadil.srs.infradead.org Wed Nov 2 16:06:40 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_73 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2L6eJV200528 for ; Wed, 2 Nov 2011 16:06:40 -0500 X-ASG-Debug-ID: 1320267997-32a702980000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9F2A121A7DF; Wed, 2 Nov 2011 14:06:38 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id CBh6K4SFNn9mhyHF; Wed, 02 Nov 2011 14:06:38 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLi16-0005Um-0e; Wed, 02 Nov 2011 21:06:36 +0000 Date: Wed, 2 Nov 2011 17:06:35 -0400 From: Christoph Hellwig To: Dmitry Monakhov Cc: Christoph Hellwig , linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 7/8] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111102210635.GA20955@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-8-git-send-email-dmonakhov@openvz.org> <20111102195357.GA22500@infradead.org> <8739e6z2gb.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8739e6z2gb.fsf@dmbot.sw.ru> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320267998 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79141 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 12:36:36AM +0400, Dmitry Monakhov wrote: > Strange... this means that parent process died unexpectedly. > Works fine fore me. In my case this happen once due to unknown > options(new opt " -fsetattr=1" passed to fsstress explicitly) because i've > forget to rebuilt fsstress via make. Can you please explicitly rebuild, > and check one more time to be 100% sure. Oh, looks like it wanted the setattr option. I held back all the new features for now until we figure if we want them for existing tests. I've applied and pushed out the first three patches for now, the others should follow as soon as I've decided on what to do with new operations. From wkendall@sgi.com Wed Nov 2 16:11:07 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LB6Tr200837 for ; Wed, 2 Nov 2011 16:11:06 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id B341B304113 for ; Wed, 2 Nov 2011 14:11:03 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 94FC87001EC0 for ; Wed, 2 Nov 2011 16:11:03 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 55A8321E23D2; Wed, 2 Nov 2011 16:11:03 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Subject: [PATCH 0/8] xfsdump: enable support for multiple streams Date: Wed, 2 Nov 2011 16:10:46 -0500 Message-Id: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This series resurrects the IRIX multi-stream support for splitting a backup among several output files/tapes. This offers some nice performance improvements, particularly in xfsrestore where a single stream often cannot keep the filesystem/disks busy. I've observed a 1.7x improvement on a backup and a 5x improvement on restore. I have a couple of xfstests for this, and will submit those once a few outstanding xfsdump test patches have been reviewed. There's a bit more work to do: - Now that xfsdump has threads once again, the tape I/O ring buffer support can be enabled. This series leaves it disabled so that more testing can be done in that area. - Currently the stream split points are determined by doing an extra inode scan. This is unchanged from how it was done on IRIX. I'd like to change this so that enough info is kept from the initial inode scan to determine appropriate split points without an additional scan. - You may notice in the last patch of this series that a lot of "miniroot" references also look at whether or not the dump is to a pipe. Now that the "miniroot" checks are gone, it's possible to clean up the pipe-related code too. I've got a separate patch series for that which I'll submit later. From wkendall@sgi.com Wed Nov 2 16:11:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBBXo200871 for ; Wed, 2 Nov 2011 16:11:11 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id D2C99AC004; Wed, 2 Nov 2011 14:11:10 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 80FDF7001EC2; Wed, 2 Nov 2011 16:11:10 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 78F2421E23D2; Wed, 2 Nov 2011 16:11:10 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap Date: Wed, 2 Nov 2011 16:10:50 -0500 Message-Id: <1320268254-20583-5-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The qlock abstraction manages an array of ordinal bitmaps, one bitmap for each thread. The bitmap indicates which locks a thread is holding and is used to verify that the locks are obtained in the correct order. There is no need to use an array to store the bitmaps, and in fact adding entries to the array is broken because qlock_thrdcnt is not protected by a lock. A simpler approach is to create a per-thread bitmap using thread local storage. With this change, there is no need for each new thread to register with the qlock abstraction, so qlock_thrdinit() goes away. Remove qlock_init() since it can be statically initialized, and that means there's no need for a flag (qlock_inited) indicating that the module has been initialized. Also there's no longer a need to track or limit the number of threads that the qlock abstraction can handle. Signed-off-by: Bill Kendall --- common/cldmgr.c | 4 - common/main.c | 7 -- common/qlock.c | 195 ++++--------------------------------------------------- common/qlock.h | 11 +--- 4 files changed, 14 insertions(+), 203 deletions(-) diff --git a/common/cldmgr.c b/common/cldmgr.c index 7784a15..d327bab 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -213,12 +213,8 @@ cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; pid_t pid = getpid( ); - /* REFERENCED */ - bool_t ok; cldp->c_pid = pid; - ok = qlock_thrdinit( ); - ASSERT( ok ); if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { stream_register( pid, ( intgen_t )cldp->c_streamix ); } diff --git a/common/main.c b/common/main.c index be8a921..25c0838 100644 --- a/common/main.c +++ b/common/main.c @@ -357,13 +357,6 @@ main( int argc, char *argv[] ) miniroot = BOOL_TRUE; } - /* initialize the spinlock allocator - */ - ok = qlock_init( ); - if ( ! ok ) { - return mlog_exit(EXIT_ERROR, RV_INIT); - } - /* initialize message logging (stage 2) - allocate the message lock */ ok = mlog_init2( ); diff --git a/common/qlock.c b/common/qlock.c index 5b87b3b..67d1291 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -48,14 +48,6 @@ struct qsem { }; typedef struct qsem qsem_t; -#define QLOCK_THRDCNTMAX 256 - /* arbitrary limit on number of threads supported - */ - -static size_t qlock_thrdcnt; - /* how many threads have checked in - */ - typedef size_t ordmap_t; /* bitmap of ordinals. used to track what ordinals have * been allocated. @@ -69,12 +61,7 @@ static ordmap_t qlock_ordalloced; /* to enforce allocation of only one lock to each ordinal value */ -struct thrddesc { - pthread_t td_tid; - ordmap_t td_ordmap; -}; -typedef struct thrddesc thrddesc_t; -static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; +static __thread ordmap_t thread_ordmap; /* holds the ordmap for each thread */ @@ -94,69 +81,12 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* checks if any bits less than ord are set in the ordmap */ -/* REFERENCED */ -static bool_t qlock_inited = BOOL_FALSE; - /* to sanity check initialization - */ - -/* forward declarations - */ -static void qlock_ordmap_add( pthread_t tid ); -static ordmap_t *qlock_ordmapp_get( pthread_t tid ); -static ix_t qlock_thrdix_get( pthread_t tid ); - -bool_t -qlock_init( void ) -{ - /* sanity checks - */ - ASSERT( ! qlock_inited ); - - /* initially no threads checked in - */ - qlock_thrdcnt = 0; - - /* initially no ordinals allocated - */ - qlock_ordalloced = 0; - - /* now say we are initialized - */ - qlock_inited = BOOL_TRUE; - - /* add the parent thread to the thread list - */ - if ( ! qlock_thrdinit( )) { - qlock_inited = BOOL_FALSE; - return BOOL_FALSE; - } - - return BOOL_TRUE; -} - -bool_t -qlock_thrdinit( void ) -{ - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* add thread to ordmap list - */ - qlock_ordmap_add( pthread_self() ); - - return BOOL_TRUE; -} qlockh_t qlock_alloc( ix_t ord ) { qlock_t *qlockp; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* verify the ordinal is not already taken, and mark as taken */ ASSERT( ! QLOCK_ORDMAP_GET( qlock_ordalloced, ord )); @@ -183,48 +113,34 @@ qlock_lock( qlockh_t qlockh ) { qlock_t *qlockp = ( qlock_t * )qlockh; pthread_t tid; - ix_t thrdix; - ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* get the caller's tid and thread index + /* get the caller's tid */ tid = pthread_self(); - thrdix = qlock_thrdix_get( tid ); - - /* get the ordmap for this thread - */ - ordmapp = qlock_ordmapp_get( tid ); - /* assert that this lock not already held by this thread */ - if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) { + if ( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock already held: thrd %d tid %lu ord %d map %x\n"), - thrdix, + _("lock already held: tid %lu ord %d map %x\n"), tid, qlockp->ql_ord, - *ordmapp ); + thread_ordmap ); } - ASSERT( ! QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); + ASSERT( ! QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )); /* assert that no locks with a lesser ordinal are held by this thread */ - if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) { + if ( QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"), - thrdix, + _("lock ordinal violation: tid %lu ord %d map %x\n"), tid, qlockp->ql_ord, - *ordmapp ); + thread_ordmap ); } - ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )); + ASSERT( ! QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord )); /* acquire the lock */ @@ -233,32 +149,23 @@ qlock_lock( qlockh_t qlockh ) /* add ordinal to this threads ordmap */ - QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord ); + QLOCK_ORDMAP_SET( thread_ordmap, qlockp->ql_ord ); } void qlock_unlock( qlockh_t qlockh ) { qlock_t *qlockp = ( qlock_t * )qlockh; - ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* get the ordmap for this thread - */ - ordmapp = qlock_ordmapp_get( pthread_self() ); - /* verify lock is held by this thread */ - ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); + ASSERT( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )); /* clear lock's ord from thread's ord map */ - QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord ); + QLOCK_ORDMAP_CLR( thread_ordmap, qlockp->ql_ord ); /* release the lock */ @@ -271,10 +178,6 @@ qsem_alloc( ix_t cnt ) { qsem_t *qsemp; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* allocate a semaphore */ qsemp = ( qsem_t * )calloc( 1, sizeof( qsem_t )); @@ -297,10 +200,6 @@ qsem_free( qsemh_t qsemh ) { qsem_t *qsemp = ( qsem_t * )qsemh; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* destroy the mutex and condition */ pthread_mutex_destroy( &qsemp->qs_mutex ); @@ -317,10 +216,6 @@ qsemP( qsemh_t qsemh ) qsem_t *qsemp = ( qsem_t * )qsemh; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* acquire the lock */ rval = pthread_mutex_lock( &qsemp->qs_mutex ); @@ -347,10 +242,6 @@ qsemV( qsemh_t qsemh ) qsem_t *qsemp = ( qsem_t * )qsemh; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* acquire the lock */ rval = pthread_mutex_lock( &qsemp->qs_mutex ); @@ -375,10 +266,6 @@ qsemPwouldblock( qsemh_t qsemh ) ix_t count; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* acquire the lock */ rval = pthread_mutex_lock( &qsemp->qs_mutex ); @@ -403,10 +290,6 @@ qsemPavail( qsemh_t qsemh ) ix_t count; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* acquire the lock */ rval = pthread_mutex_lock( &qsemp->qs_mutex ); @@ -431,10 +314,6 @@ qsemPblocked( qsemh_t qsemh ) ix_t waiters; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* acquire the lock */ rval = pthread_mutex_lock( &qsemp->qs_mutex ); @@ -451,51 +330,3 @@ qsemPblocked( qsemh_t qsemh ) return waiters; } - -/* internal ordinal map abstraction - */ -static void -qlock_ordmap_add( pthread_t tid ) -{ - ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX ); - qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid; - qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0; - qlock_thrdcnt++; -} - -static thrddesc_t * -qlock_thrddesc_get( pthread_t tid ) -{ - thrddesc_t *p; - thrddesc_t *endp; - - for ( p = &qlock_thrddesc[ 0 ], - endp = &qlock_thrddesc[ qlock_thrdcnt ] - ; - p < endp - ; - p++ ) { - if ( pthread_equal( p->td_tid, tid ) ) { - return p; - } - } - - return 0; -} - -static ordmap_t * -qlock_ordmapp_get( pthread_t tid ) -{ - thrddesc_t *p; - p = qlock_thrddesc_get( tid ); - return &p->td_ordmap; -} - -static ix_t -qlock_thrdix_get( pthread_t tid ) -{ - thrddesc_t *p; - p = qlock_thrddesc_get( tid ); - ASSERT( p >= &qlock_thrddesc[ 0 ] ); - return ( ix_t )( p - &qlock_thrddesc[ 0 ] ); -} diff --git a/common/qlock.h b/common/qlock.h index 5ac51bd..da2031c 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -21,7 +21,7 @@ /* qlock - quick locks abstraction * * threads may allocate quick locks using qlock_alloc, and free them with - * qlock_free. the abstraction is initialized with qlock_init. + * qlock_free. * * deadlock detection is accomplished by giving an ordinal number to each * lock allocated, and record all locks held by each thread. locks may not @@ -48,15 +48,6 @@ typedef void *qlockh_t; /* opaque handle */ -extern bool_t qlock_init( void ); - /* called by main to initialize abstraction. returns FALSE if - * utility should abort. - */ - -extern bool_t qlock_thrdinit( void ); - /* called by each thread to prepare it for participation - */ - extern qlockh_t qlock_alloc( ix_t ord ); /* allocates a qlock with the specified ordinal. returns * NULL if lock can't be allocated. -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:07 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LB7mM200845 for ; Wed, 2 Nov 2011 16:11:07 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 0301D304115; Wed, 2 Nov 2011 14:11:06 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id D9B907001EC0; Wed, 2 Nov 2011 16:11:06 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id CE78921E23D2; Wed, 2 Nov 2011 16:11:06 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 1/8] xfsdump: link with libpthread Date: Wed, 2 Nov 2011 16:10:47 -0500 Message-Id: <1320268254-20583-2-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch changes xfsdump/xfsrestore to link with libpthread in preparation for enabling multi-stream support. The libpthread checks are the same as those used in xfsprogs. Signed-off-by: Bill Kendall --- configure.in | 3 +++ dump/Makefile | 2 +- include/builddefs.in | 1 + m4/Makefile | 1 + m4/package_pthread.m4 | 19 +++++++++++++++++++ restore/Makefile | 2 +- 6 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 m4/package_pthread.m4 diff --git a/configure.in b/configure.in index b7ea1eb..8d4a256 100644 --- a/configure.in +++ b/configure.in @@ -67,6 +67,9 @@ AC_MULTILIB($enable_lib64) AC_PACKAGE_NEED_UUID_H AC_PACKAGE_NEED_UUIDCOMPARE +AC_PACKAGE_NEED_PTHREAD_H +AC_PACKAGE_NEED_PTHREADMUTEXINIT + AC_PACKAGE_NEED_NCURSES_H AC_PACKAGE_WANT_WORKING_LIBNCURSES diff --git a/dump/Makefile b/dump/Makefile index ce964a0..d193f68 100644 --- a/dump/Makefile +++ b/dump/Makefile @@ -89,7 +89,7 @@ HFILES = $(LOCALINCL) LHFILES = $(COMMINCL) $(INVINCL) LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON) LDIRT = $(LINKS) -LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) +LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD) LTDEPENDENCIES = $(LIBRMT) LCFLAGS = -DDUMP diff --git a/include/builddefs.in b/include/builddefs.in index f69c038..d5cc8d3 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -16,6 +16,7 @@ CFLAGS = @CFLAGS@ LIBRMT = $(TOPDIR)/librmt/librmt.la LIBXFS = @libxfs@ LIBATTR = @libattr@ +LIBPTHREAD = @libpthread@ LIBUUID = @libuuid@ LIBCURSES = @libcurses@ LIBHANDLE = @libhdl@ diff --git a/m4/Makefile b/m4/Makefile index 8ce63f8..9a35056 100644 --- a/m4/Makefile +++ b/m4/Makefile @@ -17,6 +17,7 @@ LSRCFILES = \ package_attrdev.m4 \ package_globals.m4 \ package_ncurses.m4 \ + package_pthread.m4 \ package_utilies.m4 \ package_uuiddev.m4 \ package_xfslibs.m4 \ diff --git a/m4/package_pthread.m4 b/m4/package_pthread.m4 new file mode 100644 index 0000000..be21d29 --- /dev/null +++ b/m4/package_pthread.m4 @@ -0,0 +1,19 @@ +AC_DEFUN([AC_PACKAGE_NEED_PTHREAD_H], + [ AC_CHECK_HEADERS(pthread.h) + if test $ac_cv_header_pthread_h = no; then + AC_CHECK_HEADERS(pthread.h,, [ + echo + echo 'FATAL ERROR: could not find a valid pthread header.' + exit 1]) + fi + ]) + +AC_DEFUN([AC_PACKAGE_NEED_PTHREADMUTEXINIT], + [ AC_CHECK_LIB(pthread, pthread_mutex_init,, [ + echo + echo 'FATAL ERROR: could not find a valid pthread library.' + exit 1 + ]) + libpthread=-lpthread + AC_SUBST(libpthread) + ]) diff --git a/restore/Makefile b/restore/Makefile index 003ea16..ce3d6b4 100644 --- a/restore/Makefile +++ b/restore/Makefile @@ -99,7 +99,7 @@ HFILES = $(LOCALINCL) LHFILES = $(COMMINCL) $(INVINCL) LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON) LDIRT = $(LINKS) -LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) +LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD) LTDEPENDENCIES = $(LIBRMT) LCFLAGS = -DRESTORE -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBB3h200877 for ; Wed, 2 Nov 2011 16:11:12 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id 6D863AC003; Wed, 2 Nov 2011 14:11:08 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 1F8467001EC0; Wed, 2 Nov 2011 16:11:08 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 1754C21E23D2; Wed, 2 Nov 2011 16:11:08 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Date: Wed, 2 Nov 2011 16:10:48 -0500 Message-Id: <1320268254-20583-3-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When doing multi-stream dumps, the directories are dumped by each stream in lock-step fashion. A stream backs up a given directory, then waits for all others to finish that directory, then they all move on to the next one. During restore a single stream is chosen to apply the directory dump. This patch changes xfsdump so that only one stream dumps the directories. The other streams begin dumping files immediately. This results in a faster backup, which is presumably why multiple streams are being used in the first place. It also removes the barrier routines in qlock.c as they were only used by the "SYNCDIR" code. Signed-off-by: Bill Kendall --- common/qlock.c | 37 ---------- common/qlock.h | 11 --- dump/content.c | 193 ++++++++++++----------------------------------------- restore/content.c | 33 +++++++++ 4 files changed, 75 insertions(+), 199 deletions(-) diff --git a/common/qlock.c b/common/qlock.c index fd0e98d..6c9ee16 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -568,43 +568,6 @@ return 0; #endif /* HIDDEN */ } -qbarrierh_t -qbarrier_alloc( void ) -{ -#ifdef HIDDEN - barrier_t *barrierp; - - /* sanity checks - */ - ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - /* allocate a us barrier - */ - barrierp = new_barrier( qlock_usp ); - ASSERT( barrierp ); - - return ( qbarrierh_t )barrierp; -#else -return 0; -#endif /* HIDDEN */ -} - -void -qbarrier( qbarrierh_t qbarrierh, size_t thrdcnt ) -{ -#ifdef HIDDEN - barrier_t *barrierp = ( barrier_t * )qbarrierh; - - /* sanity checks - */ - ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - barrier( barrierp, thrdcnt ); -#endif /* HIDDEN */ -} - /* internal ordinal map abstraction */ #ifdef HIDDEN diff --git a/common/qlock.h b/common/qlock.h index 54469ea..1880aff 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -102,15 +102,4 @@ extern size_t qsemPblocked( qsemh_t qsemh ); /* number of threads currently blocked on this semaphore */ -typedef void *qbarrierh_t; -#define QBARRIERH_NULL 0 - /* opaque handle - */ -extern qbarrierh_t qbarrier_alloc( void ); - /* allocates a rendezvous barrier - */ -extern void qbarrier( qbarrierh_t barrierh, size_t thrdcnt ); - /* causes thrdcnt threads to rendezvous - */ - #endif /* QLOCK_H */ diff --git a/dump/content.c b/dump/content.c index 54c97d1..64bfe54 100644 --- a/dump/content.c +++ b/dump/content.c @@ -64,9 +64,6 @@ #include "getdents.h" #include "arch_xlate.h" -#undef SYNCDIR -#define SYNCDIR - /* max "unsigned long long int" */ #define ULONGLONG_MAX 18446744073709551615LLU @@ -231,7 +228,6 @@ typedef struct extent_group_context extent_group_context_t; struct pds { enum { PDS_NULL, /* per-drive activity not begun */ PDS_INOMAP, /* dumping inomap */ - PDS_DIRRENDEZVOUS, /* waiting to dump dirs */ PDS_DIRDUMP, /* dumping dirs */ PDS_NONDIR, /* dumping nondirs */ PDS_INVSYNC, /* waiting for inventory */ @@ -263,9 +259,6 @@ static rv_t dump_dirs( ix_t strmix, xfs_bstat_t *bstatbufp, size_t bstatbuflen, void *inomap_contextp ); -#ifdef SYNCDIR -static rv_t dump_dirs_rendezvous( void ); -#endif /* SYNCDIR */ static rv_t dump_dir( ix_t strmix, jdm_fshandle_t *, intgen_t, @@ -485,12 +478,6 @@ static bool_t sc_dumpextattrpr = BOOL_TRUE; static bool_t sc_dumpasoffline = BOOL_FALSE; /* dump dual-residency HSM files as offline */ -#ifdef SYNCDIR -static size_t sc_thrdsdirdumpsynccnt = 0; -static size_t sc_thrdswaitingdirdumpsync1 = 0; -static size_t sc_thrdswaitingdirdumpsync2 = 0; -static qbarrierh_t sc_barrierh; -#endif /* SYNCDIR */ static bool_t sc_savequotas = BOOL_TRUE; /* save quota information in dump @@ -1466,14 +1453,13 @@ baseuuidbypass: var_skip( &fsid, inomap_skip ); /* fill in write header template content info. always produce - * an inomap and dir dump for each media file. + * an inomap for each media file. the dirdump flag will be set + * in content_stream_dump() for streams which dump the directories. */ ASSERT( sizeof( cwhdrtemplatep->ch_specific ) >= sizeof( *scwhdrtemplatep )); scwhdrtemplatep->cih_mediafiletype = CIH_MEDIAFILETYPE_DATA; scwhdrtemplatep->cih_level = ( int32_t )sc_level; - scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP - | - CIH_DUMPATTR_DIRDUMP; + scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP; if ( subtreecnt ) { scwhdrtemplatep->cih_dumpattr |= CIH_DUMPATTR_SUBTREE; } @@ -1714,22 +1700,6 @@ baseuuidbypass: } } -#ifdef SYNCDIR - /* allocate a barrier to synchronize directory dumping - */ - if ( drivecnt > 1 ) { - sc_barrierh = qbarrier_alloc( ); - } - - /* initialize the number of players in the synchronized dir dump. - * they drop out when last media file complete. MUST be modified - * under lock( ). - */ - sc_thrdsdirdumpsynccnt = drivecnt; - -#endif /* SYNCDIR */ - - return BOOL_TRUE; } @@ -1877,10 +1847,6 @@ content_statline( char **linespp[ ] ) strcat( statline[ statlinecnt ], "dumping inomap" ); break; - case PDS_DIRRENDEZVOUS: - strcat( statline[ statlinecnt ], - "waiting for synchronized directory dump" ); - break; case PDS_DIRDUMP: sprintf( &statline[ statlinecnt ] [ strlen( statline[ statlinecnt ] ) ], @@ -2157,6 +2123,11 @@ content_stream_dump( ix_t strmix ) scwhdrp->cih_endpt.sp_flags = STARTPT_FLAGS_END; } + // the first stream dumps the directories + if ( strmix == 0 ) { + scwhdrp->cih_dumpattr |= CIH_DUMPATTR_DIRDUMP; + } + /* fill in inomap fields of write hdr */ inomap_writehdr( scwhdrp ); @@ -2322,39 +2293,41 @@ content_stream_dump( ix_t strmix ) return mlog_exit(EXIT_FAULT, rv); } - /* now dump the directories. use the bigstat iterator - * capability to call my dump_dir function - * for each directory in the bitmap. + /* now dump the directories, if this is a stream that dumps + * directories. use the bigstat iterator capability to call + * my dump_dir function for each directory in the bitmap. */ - sc_stat_pds[ strmix ].pds_dirdone = 0; - rv = dump_dirs( strmix, - bstatbufp, - bstatbuflen, - inomap_contextp ); - if ( rv == RV_INTR ) { - stop_requested = BOOL_TRUE; - goto decision_more; - } - if ( rv == RV_EOM ) { - hit_eom = BOOL_TRUE; - goto decision_more; - } - if ( rv == RV_DRIVE ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_NORMAL, rv); - } - if ( rv == RV_ERROR ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_ERROR, rv); - } - if ( rv == RV_CORE ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_FAULT, rv); - } - ASSERT( rv == RV_OK ); - if ( rv != RV_OK ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_FAULT, rv); + if ( scwhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP ) { + sc_stat_pds[ strmix ].pds_dirdone = 0; + rv = dump_dirs( strmix, + bstatbufp, + bstatbuflen, + inomap_contextp ); + if ( rv == RV_INTR ) { + stop_requested = BOOL_TRUE; + goto decision_more; + } + if ( rv == RV_EOM ) { + hit_eom = BOOL_TRUE; + goto decision_more; + } + if ( rv == RV_DRIVE ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_NORMAL, rv); + } + if ( rv == RV_ERROR ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_ERROR, rv); + } + if ( rv == RV_CORE ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_FAULT, rv); + } + ASSERT( rv == RV_OK ); + if ( rv != RV_OK ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_FAULT, rv); + } } /* finally, dump the non-directory files beginning with this @@ -2527,20 +2500,6 @@ decision_more: */ done = all_nondirs_committed; -#ifdef SYNCDIR - /* drop out of the synchronous dump game if done - */ - if ( done ) { - /* REFERENCED */ - size_t tmpthrdsdirdumpsynccnt; - lock( ); - tmpthrdsdirdumpsynccnt = sc_thrdsdirdumpsynccnt; - sc_thrdsdirdumpsynccnt--; - unlock( ); - ASSERT( tmpthrdsdirdumpsynccnt > 0 ); - } -#endif /* SYNCDIR */ - /* tell the inventory about the media file */ if ( inv_stmt != INV_TOKEN_NULL ) { @@ -2833,22 +2792,6 @@ dump_dirs( ix_t strmix, __s32 buflenout; intgen_t rval; -#ifdef SYNCDIR - /* have all threads rendezvous - */ - if ( sc_thrdsdirdumpsynccnt > 1 && stream_cnt( ) > 1 ) { - rv_t rv; - mlog( bulkstatcallcnt == 0 ? MLOG_VERBOSE : MLOG_NITTY, - _("waiting for synchronized directory dump\n") ); - sc_stat_pds[ strmix ].pds_phase = PDS_DIRRENDEZVOUS; - rv = dump_dirs_rendezvous( ); - if ( rv == RV_INTR ) { - return RV_INTR; - } - ASSERT( rv == RV_OK ); - } -#endif /* SYNCDIR */ - if ( bulkstatcallcnt == 0 ) { mlog( MLOG_VERBOSE, _( "dumping directories\n") ); @@ -2948,58 +2891,6 @@ dump_dirs( ix_t strmix, /* NOTREACHED */ } -#ifdef SYNCDIR -static rv_t -dump_dirs_rendezvous( void ) -{ - static size_t localsync1; - static size_t localsync2; - - sc_thrdswaitingdirdumpsync2 = 0; - lock( ); - sc_thrdswaitingdirdumpsync1++; - localsync1 = sc_thrdswaitingdirdumpsync1; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - while ( localsync2 == 0 - && - localsync1 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) { - sleep( 1 ); - if ( cldmgr_stop_requested( )) { - lock( ); - sc_thrdswaitingdirdumpsync1--; - unlock( ); - return RV_INTR; - } - lock( ); - localsync1 = sc_thrdswaitingdirdumpsync1; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - } - lock( ); - sc_thrdswaitingdirdumpsync1--; - sc_thrdswaitingdirdumpsync2++; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - while ( localsync2 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) { - sleep( 1 ); - if ( cldmgr_stop_requested( )) { - return RV_INTR; - } - lock( ); - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - } - if ( cldmgr_stop_requested( )) { - return RV_INTR; - } - - qbarrier( sc_barrierh, min( stream_cnt( ), sc_thrdsdirdumpsynccnt )); - - return RV_OK; -} -#endif /* SYNCDIR */ - static rv_t dump_dir( ix_t strmix, jdm_fshandle_t *fshandlep, diff --git a/restore/content.c b/restore/content.c index 34fc4a0..2228a7f 100644 --- a/restore/content.c +++ b/restore/content.c @@ -625,6 +625,9 @@ struct tran { intgen_t t_persfd; /* file descriptor of the persistent state file */ + size64_t t_dirdumps; + /* bitset of streams which contain a directory dump + */ sync_t t_sync1; /* to single-thread attempt to validate command line * selection of dump with online inventory @@ -1184,6 +1187,12 @@ content_init( intgen_t argc, char *argv[ ], size64_t vmsz ) return BOOL_FALSE; } + /* assume all streams contain a directory dump. streams will remove + * themselves from this bitset if they do not contain a directory dump. + */ + ASSERT( drivecnt <= sizeof(tranp->t_dirdumps) * NBBY ); + tranp->t_dirdumps = ( 1ULL << drivecnt ) - 1; + /* the user may specify stdin as the restore source stream, * by a single dash ('-') with no option letter. This must * appear between the last lettered argument and the destination @@ -2237,6 +2246,30 @@ content_stream_restore( ix_t thrdix ) unlock( ); continue; } + if ( !(scrhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP) ) { + /* if no streams have a directory dump, issue a + * message and exit. first set SYNC_BUSY to prevent + * other threads from coming through here and issuing + * the same message. + */ + tranp->t_dirdumps &= ~(1ULL << thrdix); + if ( !tranp->t_dirdumps ) { + tranp->t_sync3 = SYNC_BUSY; + } + unlock( ); + if ( !tranp->t_dirdumps ) { + mlog( MLOG_VERBOSE | MLOG_ERROR, _( + "no directory dump found\n") ); + Media_end( Mediap ); + return mlog_exit(EXIT_NORMAL, RV_ERROR); + } + sleep( 1 ); + if ( cldmgr_stop_requested( )) { + Media_end( Mediap ); + return mlog_exit(EXIT_NORMAL, RV_INTR); + } + continue; + } tranp->t_sync3 = SYNC_BUSY; unlock( ); if ( ! tranp->t_dirattrinitdonepr ) { -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBEhH200912 for ; Wed, 2 Nov 2011 16:11:14 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id A703EAC002; Wed, 2 Nov 2011 14:11:13 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 5850C7001EC0; Wed, 2 Nov 2011 16:11:13 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 5047021E23D2; Wed, 2 Nov 2011 16:11:13 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 7/8] xfsdump: path lookup cache must be thread specific Date: Wed, 2 Nov 2011 16:10:53 -0500 Message-Id: <1320268254-20583-8-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The xfsrestore path lookup routine (maps an ino + generation number to a pathname) contains a cache for fast lookups of consecutive files in the same parent directory. This cache either needs to be protected by a lock or a cache is needed per thread. I took the latter approach since it doesn't add lock contention and it increases the cache hit rate (it's relatively unlikely that each stream is processing files in the same directory). Signed-off-by: Bill Kendall --- restore/tree.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/restore/tree.c b/restore/tree.c index 7bddddf..9e4e83c 100644 --- a/restore/tree.c +++ b/restore/tree.c @@ -3441,7 +3441,7 @@ Node2path( nh_t nh, char *path, char *errmsg ) static intgen_t Node2path_recurse( nh_t nh, char *buf, intgen_t bufsz, intgen_t level ) { - static path_cache_t cache = { NH_NULL, 0, "" }; + static __thread path_cache_t cache = { NH_NULL, 0, "" }; node_t *np; nh_t parh; xfs_ino_t ino; -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBCVu200885 for ; Wed, 2 Nov 2011 16:11:12 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 64624304113; Wed, 2 Nov 2011 14:11:12 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 3FBC87001EC2; Wed, 2 Nov 2011 16:11:12 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 31C5B21E23D2; Wed, 2 Nov 2011 16:11:12 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 6/8] xfsdump: process thread exit status Date: Wed, 2 Nov 2011 16:10:52 -0500 Message-Id: <1320268254-20583-7-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When IRIX sprocs were in use, the main thread was notified of a thread exit just as if a child process exited -- it received SIGCHLD. The main thread would grab the pid and exit status, then call cldmgr_died() to inform it that the child was gone so the slot in the child array could be freed up for reuse. This patch implements a similar mechanism for pthreads. The "c_busy" field in struct cld has been replaced with a "c_state" field that indicates whether the array slot is free (C_AVAIL), in use (C_ALIVE), or is waiting to be joined (C_EXITED). Additionally a "c_exit_code" field has been added to store the thread's exit value. Normally this is set when the thread entry function returns, but it is initialized to EXIT_INTERRUPT in case the thread is cancelled or calls pthread_exit() rather than returning (neither of which happens in the code today). When the child thread starts, it registers a pthread cleanup handler which takes care of marking the child as C_EXITED and notifies the main thread that a child is gone. Doing this in a cleanup handler ensures that it's done regardless of how the thread exits. The main thread's loop is based around sigsuspsend(), so the notification is done by sending SIGUSR1. The main thread will then call cldmgr_join() to join all exited threads and obtain their exit status. Additional changes: * cldmgr_findbypid() has been removed, it's no longer referenced. * stream_dead() no longer grabs the lock(), because it's called only from cldmgr_join() which already holds the lock(). Signed-off-by: Bill Kendall --- common/cldmgr.c | 88 +++++++++++++++++++++++++++++++++++------------------- common/cldmgr.h | 7 +++- common/main.c | 33 +++++--------------- common/stream.c | 3 +- common/stream.h | 1 + 5 files changed, 73 insertions(+), 59 deletions(-) diff --git a/common/cldmgr.c b/common/cldmgr.c index 4574834..be7de34 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -26,6 +26,7 @@ #include #include +#include "exit.h" #include "types.h" #include "lock.h" #include "qlock.h" @@ -36,8 +37,12 @@ extern size_t pgsz; #define CLD_MAX ( STREAM_SIMMAX * 2 ) + +typedef enum { C_AVAIL, C_ALIVE, C_EXITED } state_t; + struct cld { - bool_t c_busy; + state_t c_state; + intgen_t c_exit_code; pthread_t c_tid; ix_t c_streamix; int ( * c_entry )( void *arg1 ); @@ -50,8 +55,8 @@ static cld_t cld[ CLD_MAX ]; static bool_t cldmgr_stopflag; static cld_t *cldmgr_getcld( void ); -static cld_t * cldmgr_findbytid( pthread_t ); static void *cldmgr_entry( void * ); +static void cldmgr_cleanup( void * ); /* REFERENCED */ static pthread_t cldmgr_parenttid; @@ -87,6 +92,7 @@ cldmgr_create( int ( * entry )( void *arg1 ), return BOOL_FALSE; } + cldp->c_exit_code = EXIT_INTERRUPT; cldp->c_streamix = streamix; cldp->c_entry = entry; cldp->c_arg1 = arg1; @@ -117,18 +123,37 @@ cldmgr_stop( void ) cldmgr_stopflag = BOOL_TRUE; } -void -cldmgr_died( pthread_t tid ) +intgen_t +cldmgr_join( void ) { - cld_t *cldp = cldmgr_findbytid( tid ); + cld_t *p = cld; + cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); + intgen_t xc = EXIT_NORMAL; - if ( ! cldp ) { - return; - } - cldp->c_busy = BOOL_FALSE; - if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_dead( tid ); + lock(); + for ( ; p < ep ; p++ ) { + if ( p->c_state == C_EXITED ) { + if ( ( intgen_t )( p->c_streamix ) >= 0 ) { + stream_dead( p->c_tid ); + } + pthread_join( p->c_tid, NULL ); + if ( p->c_exit_code != EXIT_NORMAL && xc != EXIT_FAULT ) + xc = p->c_exit_code; + if ( p->c_exit_code != EXIT_NORMAL ) { + mlog( MLOG_DEBUG | MLOG_PROC | MLOG_NOLOCK, + "child (thread %lu) requested stop: " + "exit code %d (%s)\n", + p->c_tid, p->c_exit_code, + exit_codestring( p->c_exit_code )); + } + + // reinit this child for reuse + memset( ( void * )p, 0, sizeof( cld_t )); + } } + unlock(); + + return xc; } bool_t @@ -147,7 +172,7 @@ cldmgr_remainingcnt( void ) cnt = 0; lock( ); for ( ; p < ep ; p++ ) { - if ( p->c_busy ) { + if ( p->c_state == C_ALIVE ) { cnt++; } } @@ -164,7 +189,7 @@ cldmgr_otherstreamsremain( ix_t streamix ) lock( ); for ( ; p < ep ; p++ ) { - if ( p->c_busy && p->c_streamix != streamix ) { + if ( p->c_state == C_ALIVE && p->c_streamix != streamix ) { unlock( ); return BOOL_TRUE; } @@ -182,8 +207,8 @@ cldmgr_getcld( void ) lock(); for ( ; p < ep ; p++ ) { - if ( ! p->c_busy ) { - p->c_busy = BOOL_TRUE; + if ( p->c_state == C_AVAIL ) { + p->c_state = C_ALIVE; break; } } @@ -192,27 +217,14 @@ cldmgr_getcld( void ) return ( p < ep ) ? p : 0; } -static cld_t * -cldmgr_findbytid( pthread_t tid ) -{ - cld_t *p = cld; - cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); - - for ( ; p < ep ; p++ ) { - if ( p->c_busy && pthread_equal( p->c_tid, tid )) { - break; - } - } - - return ( p < ep ) ? p : 0; -} - static void * cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; pthread_t tid = pthread_self( ); + pthread_cleanup_push( cldmgr_cleanup, arg1 ); + if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { stream_register( tid, ( intgen_t )cldp->c_streamix ); } @@ -220,7 +232,21 @@ cldmgr_entry( void *arg1 ) "thread %lu created for stream %d\n", tid, cldp->c_streamix ); + cldp->c_exit_code = ( * cldp->c_entry )( cldp->c_arg1 ); + + pthread_cleanup_pop( 1 ); - ( * cldp->c_entry )( cldp->c_arg1 ); return NULL; } + +static void +cldmgr_cleanup( void *arg1 ) +{ + cld_t *cldp = ( cld_t * )arg1; + + lock(); + cldp->c_state = C_EXITED; + // signal the main thread to look for exited threads + kill( getpid( ), SIGUSR1 ); + unlock(); +} diff --git a/common/cldmgr.h b/common/cldmgr.h index e393b82..1df0c0c 100644 --- a/common/cldmgr.h +++ b/common/cldmgr.h @@ -39,9 +39,12 @@ extern bool_t cldmgr_create( int ( * entry )( void *arg1 ), */ extern void cldmgr_stop( void ); -/* cldmgr_died - tells the child manager that the child died +/* cldmgr_join - join child threads that have exited. + * returns EXIT_NORMAL if all exited normally (or no threads have exited), + * EXIT_FAULT if any threads requested a core dump, or another EXIT_* + * value if any threads exited abnormally. */ -extern void cldmgr_died( pthread_t tid ); +extern intgen_t cldmgr_join( void ); /* cldmgr_stop_requested - returns TRUE if the child should gracefully * terminate. diff --git a/common/main.c b/common/main.c index d4dbe28..38b3889 100644 --- a/common/main.c +++ b/common/main.c @@ -137,10 +137,6 @@ static bool_t sighup_received; static bool_t sigterm_received; static bool_t sigquit_received; static bool_t sigint_received; -static size_t prbcld_cnt; -static pid_t prbcld_pid; -static intgen_t prbcld_xc; -static intgen_t prbcld_signo; /* REFERENCED */ static intgen_t sigstray_received; static bool_t progrpt_enabledpr; @@ -168,6 +164,8 @@ main( int argc, char *argv[] ) intgen_t exitcode; rlim64_t tmpstacksz; struct sigaction sa; + intgen_t prbcld_xc = EXIT_NORMAL; + intgen_t xc; bool_t ok; /* REFERENCED */ int rval; @@ -563,7 +561,6 @@ main( int argc, char *argv[] ) sigint_received = BOOL_FALSE; sigquit_received = BOOL_FALSE; sigstray_received = BOOL_FALSE; - prbcld_cnt = 0; alarm( 0 ); @@ -573,6 +570,7 @@ main( int argc, char *argv[] ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); + sigaddset( &blocked_set, SIGUSR1 ); pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); sa.sa_handler = sighandler; @@ -581,6 +579,7 @@ main( int argc, char *argv[] ) sigaction( SIGTERM, &sa, NULL ); sigaction( SIGQUIT, &sa, NULL ); sigaction( SIGALRM, &sa, NULL ); + sigaction( SIGUSR1, &sa, NULL ); } /* do content initialization. @@ -710,31 +709,16 @@ main( int argc, char *argv[] ) * stop. furthermore, note that core should be dumped if * the child explicitly exited with EXIT_FAULT. */ - if ( prbcld_cnt ) { - if ( prbcld_xc == EXIT_FAULT || prbcld_signo != 0 ) { + xc = cldmgr_join( ); + if ( xc ) { + if ( xc == EXIT_FAULT ) { coredump_requested = BOOL_TRUE; stop_timeout = ABORT_TIMEOUT; } else { stop_timeout = STOP_TIMEOUT; } + prbcld_xc = xc; stop_requested = BOOL_TRUE; - if ( prbcld_xc != EXIT_NORMAL ) { - mlog( MLOG_DEBUG | MLOG_PROC, - "child (pid %d) requested stop: " - "exit code %d (%s)\n", - prbcld_pid, - prbcld_xc, - exit_codestring( prbcld_xc )); - } else if ( prbcld_signo ) { - ASSERT( prbcld_signo ); - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC, - _("child (pid %d) faulted: " - "signal number %d (%s)\n"), - prbcld_pid, - prbcld_signo, - sig_numstring( prbcld_signo )); - } - prbcld_cnt = 0; } /* all children died normally. break out. @@ -1528,6 +1512,7 @@ sighandler( int signo ) sigquit_received = BOOL_TRUE; break; case SIGALRM: + case SIGUSR1: break; default: sigstray_received = signo; diff --git a/common/stream.c b/common/stream.c index 48e25ee..6704661 100644 --- a/common/stream.c +++ b/common/stream.c @@ -86,19 +86,18 @@ stream_register( pthread_t tid, intgen_t streamix ) p->s_exit_hint = RV_NONE; } +/* NOTE: lock() must be held when calling stream_dead() */ void stream_dead( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); - lock(); for ( ; p < ep ; p++ ) if ( pthread_equal( p->s_tid, tid ) ) { p->s_state = S_ZOMBIE; break; } - unlock(); ASSERT( p < ep ); } diff --git a/common/stream.h b/common/stream.h index 292792e..4b3799f 100644 --- a/common/stream.h +++ b/common/stream.h @@ -43,6 +43,7 @@ typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t; extern void stream_init( void ); extern void stream_register( pthread_t tid, intgen_t streamix ); +/* NOTE: lock() must be held when calling stream_dead */ extern void stream_dead( pthread_t tid ); extern void stream_free( pthread_t tid ); extern int stream_find_all( stream_state_t states[], -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBC7b200884 for ; Wed, 2 Nov 2011 16:11:12 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5FB3F8F80D1; Wed, 2 Nov 2011 14:11:09 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 4A0A97001EC1; Wed, 2 Nov 2011 16:11:09 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 4001421E23D2; Wed, 2 Nov 2011 16:11:09 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 3/8] xfsdump: implement lock abstraction with pthreads Date: Wed, 2 Nov 2011 16:10:49 -0500 Message-Id: <1320268254-20583-4-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch implements xfsdump's locking abstraction (qlock_t and qsem_t) using pthread locking primitives. The obsolete IRIX locking code has been removed. A few debug-level log calls had to be changed to use the "no lock" flag to prevent locks from being acquired out of order. This is used in various debug messages throughout xfsdump. Signed-off-by: Bill Kendall --- common/main.c | 2 +- common/qlock.c | 407 +++++++++++++++++++---------------------------------- common/qlock.h | 10 +- restore/content.c | 17 ++- 4 files changed, 158 insertions(+), 278 deletions(-) diff --git a/common/main.c b/common/main.c index 08be16e..be8a921 100644 --- a/common/main.c +++ b/common/main.c @@ -359,7 +359,7 @@ main( int argc, char *argv[] ) /* initialize the spinlock allocator */ - ok = qlock_init( miniroot ); + ok = qlock_init( ); if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } diff --git a/common/qlock.c b/common/qlock.c index 6c9ee16..5b87b3b 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -19,6 +19,8 @@ #include #include +#include + #include "types.h" #include "qlock.h" #include "mlog.h" @@ -27,23 +29,24 @@ struct qlock { ix_t ql_ord; /* ordinal position of this lock */ - pid_t ql_owner; - /* who owns this lock - */ -#ifdef HIDDEN - ulock_t ql_uslockh; - /* us lock handle - */ -#endif /* HIDDEN */ + pthread_mutex_t ql_mutex; }; typedef struct qlock qlock_t; /* internal qlock */ -#define QLOCK_SPINS 0x1000 - /* how many times to spin on lock before sleeping for it - */ +struct qsem { + ix_t qs_count; + /* current count of this semaphore + */ + ix_t qs_waiters; + /* number of threads waiting on semaphore + */ + pthread_mutex_t qs_mutex; + pthread_cond_t qs_cond; +}; +typedef struct qsem qsem_t; #define QLOCK_THRDCNTMAX 256 /* arbitrary limit on number of threads supported @@ -67,15 +70,13 @@ static ordmap_t qlock_ordalloced; */ struct thrddesc { - pid_t td_pid; + pthread_t td_tid; ordmap_t td_ordmap; }; typedef struct thrddesc thrddesc_t; -#ifdef HIDDEN static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* holds the ordmap for each thread */ -#endif #define QLOCK_ORDMAP_SET( ordmap, ord ) ( ordmap |= 1U << ord ) /* sets the ordinal bit in an ordmap @@ -93,21 +94,6 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* checks if any bits less than ord are set in the ordmap */ -#ifdef HIDDEN -static usptr_t *qlock_usp; -#else -static void *qlock_usp; -#endif /* HIDDEN */ - - /* pointer to shared arena from which locks are allocated - */ - -#ifdef HIDDEN -static char *qlock_arenaroot = "xfsrestoreqlockarena"; - /* shared arena file name root - */ -#endif - /* REFERENCED */ static bool_t qlock_inited = BOOL_FALSE; /* to sanity check initialization @@ -115,22 +101,13 @@ static bool_t qlock_inited = BOOL_FALSE; /* forward declarations */ -#ifdef HIDDEN -static void qlock_ordmap_add( pid_t pid ); -static ordmap_t *qlock_ordmapp_get( pid_t pid ); -static ix_t qlock_thrdix_get( pid_t pid ); -#endif +static void qlock_ordmap_add( pthread_t tid ); +static ordmap_t *qlock_ordmapp_get( pthread_t tid ); +static ix_t qlock_thrdix_get( pthread_t tid ); bool_t -qlock_init( bool_t miniroot ) +qlock_init( void ) { -#ifdef HIDDEN - char arenaname[ 100 ]; - /* REFERENCED */ - intgen_t nwritten; - intgen_t rval; -#endif - /* sanity checks */ ASSERT( ! qlock_inited ); @@ -143,44 +120,6 @@ qlock_init( bool_t miniroot ) */ qlock_ordalloced = 0; - /* if miniroot, fake it - */ - if ( miniroot ) { - qlock_inited = BOOL_TRUE; - qlock_usp = 0; - return BOOL_TRUE; - } -#ifdef HIDDEN - - /* generate the arena name - */ - nwritten = sprintf( arenaname, - "/tmp/%s.%d", - qlock_arenaroot, - get_pid() ); - ASSERT( nwritten > 0 ); - ASSERT( ( size_t )nwritten < sizeof( arenaname )); - - /* configure shared arenas to automatically unlink on last close - */ - rval = usconfig( CONF_ARENATYPE, ( u_intgen_t )US_SHAREDONLY ); - if ( rval ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to configure shared arena for auto unlink: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } - - /* allocate a shared arena for the locks - */ - qlock_usp = usinit( arenaname ); - if ( ! qlock_usp ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to allocate shared arena for thread locks: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } - /* now say we are initialized */ qlock_inited = BOOL_TRUE; @@ -191,7 +130,6 @@ qlock_init( bool_t miniroot ) qlock_inited = BOOL_FALSE; return BOOL_FALSE; } -#endif /* HIDDEN */ return BOOL_TRUE; } @@ -199,28 +137,13 @@ qlock_init( bool_t miniroot ) bool_t qlock_thrdinit( void ) { -#ifdef HIDDEN - intgen_t rval; - /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - /* add thread to shared arena - */ - rval = usadd( qlock_usp ); - if ( rval ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to add thread to shared arena: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } /* add thread to ordmap list */ - qlock_ordmap_add( get_pid() ); -#endif /* HIDDEN */ + qlock_ordmap_add( pthread_self() ); return BOOL_TRUE; } @@ -244,14 +167,9 @@ qlock_alloc( ix_t ord ) qlockp = ( qlock_t * )calloc( 1, sizeof( qlock_t )); ASSERT( qlockp ); -#ifdef HIDDEN - /* allocate a us lock: bypass if miniroot + /* initialize the mutex */ - if ( qlock_usp ) { - qlockp->ql_uslockh = usnewlock( qlock_usp ); - ASSERT( qlockp->ql_uslockh ); - } -#endif /* HIDDEN */ + pthread_mutex_init( &qlockp->ql_mutex, NULL ); /* assign the ordinal position */ @@ -263,44 +181,34 @@ qlock_alloc( ix_t ord ) void qlock_lock( qlockh_t qlockh ) { -#ifdef HIDDEN qlock_t *qlockp = ( qlock_t * )qlockh; - pid_t pid; + pthread_t tid; ix_t thrdix; ordmap_t *ordmapp; /* REFERENCED */ - bool_t lockacquired; -#endif + intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - /* bypass if miniroot - */ - if ( ! qlock_usp ) { - return; - } - -#ifdef HIDDEN - - /* get the caller's pid and thread index + /* get the caller's tid and thread index */ - pid = get_pid(); + tid = pthread_self(); - thrdix = qlock_thrdix_get( pid ); + thrdix = qlock_thrdix_get( tid ); /* get the ordmap for this thread */ - ordmapp = qlock_ordmapp_get( pid ); + ordmapp = qlock_ordmapp_get( tid ); - /* assert that this lock not already held + /* assert that this lock not already held by this thread */ if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock already held: thrd %d pid %d ord %d map %x\n"), + _("lock already held: thrd %d tid %lu ord %d map %x\n"), thrdix, - pid, + tid, qlockp->ql_ord, *ordmapp ); } @@ -310,278 +218,253 @@ qlock_lock( qlockh_t qlockh ) */ if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock ordinal violation: thrd %d pid %d ord %d map %x\n"), + _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"), thrdix, - pid, + tid, qlockp->ql_ord, *ordmapp ); } ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )); - /* acquire the us lock + /* acquire the lock */ - lockacquired = uswsetlock( qlockp->ql_uslockh, QLOCK_SPINS ); - ASSERT( lockacquired ); - - /* verify lock is not already held - */ - ASSERT( ! qlockp->ql_owner ); + rval = pthread_mutex_lock( &qlockp->ql_mutex ); + ASSERT( !rval ); /* add ordinal to this threads ordmap */ QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord ); - - /* indicate the lock's owner - */ - qlockp->ql_owner = pid; -#endif /* HIDDEN */ } void qlock_unlock( qlockh_t qlockh ) { -#ifdef HIDDEN qlock_t *qlockp = ( qlock_t * )qlockh; - pid_t pid; ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; -#endif /* sanity checks */ ASSERT( qlock_inited ); - /* bypass if miniroot - */ - if ( ! qlock_usp ) { - return; - } - -#ifdef HIDDEN - /* get the caller's pid - */ - pid = get_pid(); - /* get the ordmap for this thread */ - ordmapp = qlock_ordmapp_get( pid ); + ordmapp = qlock_ordmapp_get( pthread_self() ); /* verify lock is held by this thread */ ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); - ASSERT( qlockp->ql_owner == pid ); - - /* clear lock owner - */ - qlockp->ql_owner = 0; /* clear lock's ord from thread's ord map */ QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord ); - /* release the us lock + /* release the lock */ - rval = usunsetlock( qlockp->ql_uslockh ); + rval = pthread_mutex_unlock( &qlockp->ql_mutex ); ASSERT( ! rval ); -#endif /* HIDDEN */ } qsemh_t qsem_alloc( ix_t cnt ) { -#ifdef HIDDEN - usema_t *usemap; + qsem_t *qsemp; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* allocate a us semaphore + /* allocate a semaphore */ - usemap = usnewsema( qlock_usp, ( intgen_t )cnt ); - ASSERT( usemap ); + qsemp = ( qsem_t * )calloc( 1, sizeof( qsem_t )); + ASSERT( qsemp ); - return ( qsemh_t )usemap; -#else - return 0; -#endif /* HIDDEN */ + /* initialize the mutex and condition + */ + pthread_mutex_init(&qsemp->qs_mutex, NULL); + pthread_cond_init(&qsemp->qs_cond, NULL); + + /* assign the initial count + */ + qsemp->qs_count = cnt; + + return ( qsemh_t )qsemp; } void qsem_free( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* free the us semaphore + /* destroy the mutex and condition + */ + pthread_mutex_destroy( &qsemp->qs_mutex ); + pthread_cond_destroy( &qsemp->qs_cond ); + + /* free the semaphore */ - usfreesema( usemap, qlock_usp ); -#endif /* HIDDEN */ + free( qsemp ); } void qsemP( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); + + /* acquire the lock + */ + rval = pthread_mutex_lock( &qsemp->qs_mutex ); + ASSERT( !rval ); /* "P" the semaphore */ - rval = uspsema( usemap ); - if ( rval != 1 ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to \"P\" semaphore: " - "rval == %d, errno == %d (%s)\n"), - rval, - errno, - strerror( errno )); + qsemp->qs_waiters++; + while ( qsemp->qs_count == 0 ) { + pthread_cond_wait( &qsemp->qs_cond, &qsemp->qs_mutex ); } - ASSERT( rval == 1 ); -#endif /* HIDDEN */ + qsemp->qs_waiters--; + qsemp->qs_count--; + + /* release the lock + */ + rval = pthread_mutex_unlock( &qsemp->qs_mutex ); + ASSERT( !rval ); } void qsemV( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); + + /* acquire the lock + */ + rval = pthread_mutex_lock( &qsemp->qs_mutex ); + ASSERT( !rval ); /* "V" the semaphore */ - rval = usvsema( usemap ); - if ( rval != 0 ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to \"V\" semaphore: " - "rval == %d, errno == %d (%s)\n"), - rval, - errno, - strerror( errno )); - } - ASSERT( rval == 0 ); -#endif /* HIDDEN */ + qsemp->qs_count++; + if ( qsemp->qs_waiters ) + pthread_cond_signal( &qsemp->qs_cond ); + + /* release the lock + */ + rval = pthread_mutex_unlock( &qsemp->qs_mutex ); + ASSERT( !rval ); } bool_t qsemPwouldblock( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; + ix_t count; intgen_t rval; - + /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* check the semaphore + /* acquire the lock */ - rval = ustestsema( usemap ); + rval = pthread_mutex_lock( &qsemp->qs_mutex ); + ASSERT( !rval ); - /* if equal to zero, no tokens left. if less than zero, other thread(s) - * currently waiting. + /* check the count */ - if ( rval <= 0 ) { - return BOOL_TRUE; - } else { - return BOOL_FALSE; - } -#else -return BOOL_FALSE; -#endif /* HIDDEN */ + count = qsemp->qs_count; + + /* release the lock + */ + rval = pthread_mutex_unlock( &qsemp->qs_mutex ); + ASSERT( !rval ); + + return count == 0 ? BOOL_TRUE : BOOL_FALSE; } size_t qsemPavail( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; + ix_t count; intgen_t rval; - + /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* check the semaphore + /* acquire the lock */ - rval = ustestsema( usemap ); + rval = pthread_mutex_lock( &qsemp->qs_mutex ); + ASSERT( !rval ); - /* if greater or equal to zero, no one is blocked and that is the number - * of resources available. if less than zero, absolute value is the - * number of blocked threads. + /* check the count */ - if ( rval < 0 ) { - return 0; - } else { - return ( size_t )rval; - } -#else -return 0; -#endif /* HIDDEN */ + count = qsemp->qs_count; + + /* release the lock + */ + rval = pthread_mutex_unlock( &qsemp->qs_mutex ); + ASSERT( !rval ); + + return count; } size_t qsemPblocked( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + qsem_t *qsemp = ( qsem_t * )qsemh; + ix_t waiters; intgen_t rval; - + /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* check the semaphore + /* acquire the lock */ - rval = ustestsema( usemap ); + rval = pthread_mutex_lock( &qsemp->qs_mutex ); + ASSERT( !rval ); - /* if greater or equal to zero, no one is blocked. if less than zero, - * absolute value is the number of blocked threads. + /* check the waiters */ - if ( rval < 0 ) { - return ( size_t )( 0 - rval ); - } else { - return 0; - } -#else -return 0; -#endif /* HIDDEN */ + waiters = qsemp->qs_waiters; + + /* release the lock + */ + rval = pthread_mutex_unlock( &qsemp->qs_mutex ); + ASSERT( !rval ); + + return waiters; } /* internal ordinal map abstraction */ -#ifdef HIDDEN static void -qlock_ordmap_add( pid_t pid ) +qlock_ordmap_add( pthread_t tid ) { ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX ); - qlock_thrddesc[ qlock_thrdcnt ].td_pid = pid; + qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid; qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0; qlock_thrdcnt++; } static thrddesc_t * -qlock_thrddesc_get( pid_t pid ) +qlock_thrddesc_get( pthread_t tid ) { thrddesc_t *p; thrddesc_t *endp; @@ -592,29 +475,27 @@ qlock_thrddesc_get( pid_t pid ) p < endp ; p++ ) { - if ( p->td_pid == pid ) { + if ( pthread_equal( p->td_tid, tid ) ) { return p; } } - ASSERT( 0 ); return 0; } static ordmap_t * -qlock_ordmapp_get( pid_t pid ) +qlock_ordmapp_get( pthread_t tid ) { thrddesc_t *p; - p = qlock_thrddesc_get( pid ); + p = qlock_thrddesc_get( tid ); return &p->td_ordmap; } static ix_t -qlock_thrdix_get( pid_t pid ) +qlock_thrdix_get( pthread_t tid ) { thrddesc_t *p; - p = qlock_thrddesc_get( pid ); + p = qlock_thrddesc_get( tid ); ASSERT( p >= &qlock_thrddesc[ 0 ] ); return ( ix_t )( p - &qlock_thrddesc[ 0 ] ); } -#endif diff --git a/common/qlock.h b/common/qlock.h index 1880aff..5ac51bd 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -21,19 +21,13 @@ /* qlock - quick locks abstraction * * threads may allocate quick locks using qlock_alloc, and free them with - * qlock_free. the abstraction is initialized with qlock_init. the underlying - * mechanism is the IRIX us lock primitive. in order to use this, a temporary - * shared arena is created in /tmp. this will be automatically unlinked - * when the last thread exits. + * qlock_free. the abstraction is initialized with qlock_init. * * deadlock detection is accomplished by giving an ordinal number to each * lock allocated, and record all locks held by each thread. locks may not * be acquired out of order. that is, subsequently acquired locks must have * a lower ordinal than all locks currently held. for convenience, the ordinals * of all locks to be allocated will be defined in this file. - * - * ADDITION: added counting semaphores. simpler to do here since same - * shared arena can be used. */ #define QLOCK_ORD_CRIT 0 @@ -54,7 +48,7 @@ typedef void *qlockh_t; /* opaque handle */ -extern bool_t qlock_init( bool_t miniroot ); +extern bool_t qlock_init( void ); /* called by main to initialize abstraction. returns FALSE if * utility should abort. */ diff --git a/restore/content.c b/restore/content.c index 2228a7f..8dfa456 100644 --- a/restore/content.c +++ b/restore/content.c @@ -8911,7 +8911,8 @@ partial_reg( ix_t d_index, /* If not found, find a free one, fill it in and return */ if ( ! isptr ) { - mlog(MLOG_NITTY, "partial_reg: no entry found for %llu\n", ino); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: no entry found for %llu\n", ino); /* find a free one */ for (i=0; i < partialmax; i++ ) { if (persp->a.parrest[i].is_ino == 0) { @@ -8950,8 +8951,10 @@ found: /* no existing entry for this drive, fill in the values */ bsptr->offset = offset; bsptr->endoffset = endoffset; - mlog(MLOG_NITTY, "partial_reg: update entry [%d]: " - "\n", d_index, offset, endoffset); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: update entry [%d]: " + "\n", + d_index, offset, endoffset); } else { bool_t ret; @@ -8960,9 +8963,11 @@ found: */ bsptr->endoffset = endoffset; ret = partial_check2(isptr, fsize); - mlog(MLOG_NITTY, "partial_reg: extend entry [%d]: " - "\n", d_index, endoffset); - mlog(MLOG_NITTY, "partial_reg: partial_check returns: %d\n", ret); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: extend entry [%d]: \n", + d_index, endoffset); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: partial_check returns: %d\n", ret); } pi_unlock(); -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:16 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBFE0200925 for ; Wed, 2 Nov 2011 16:11:16 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id 268F8AC002; Wed, 2 Nov 2011 14:11:15 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id CE5377001EC0; Wed, 2 Nov 2011 16:11:14 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id C63DF21E23D2; Wed, 2 Nov 2011 16:11:14 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 8/8] xfsdump: enable multiple streams Date: Wed, 2 Nov 2011 16:10:54 -0500 Message-Id: <1320268254-20583-9-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean IRIX contained an environment referred to as "miniroot" where sproc threads were either not available, or at least not used in xfsdump. Throughout xfsdump there's a "miniroot" variable which indicates whether or not thread support is enabled. On Linux this variable has always been false in order to disable support for multiple streams. Now that the threading infracstructure has been converted over to pthreads, this patch removes the "miniroot" variable and enables the option of using multiple streams. Note that another feature in xfsdump, using a ring buffer for I/O to tapes, also depends on thread support. I'm leaving that disabled for now until more testing has been done. Signed-off-by: Bill Kendall --- common/drive.c | 20 +------ common/drive.h | 8 +-- common/drive_minrmt.c | 15 +++--- common/drive_scsitape.c | 15 +++--- common/drive_simple.c | 8 ++-- common/main.c | 124 +++++----------------------------------------- dump/content.c | 35 ++++++------- dump/getopt.h | 4 +- man/man8/xfsdump.8 | 8 +-- man/man8/xfsrestore.8 | 4 +- restore/content.c | 23 ++++----- restore/getopt.h | 4 +- 12 files changed, 74 insertions(+), 194 deletions(-) diff --git a/common/drive.c b/common/drive.c index 227fecd..32a7191 100644 --- a/common/drive.c +++ b/common/drive.c @@ -81,7 +81,7 @@ static drive_strategy_t *strategypp[] = { * specified on the command line. */ bool_t -drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) +drive_init1( int argc, char *argv[ ] ) { intgen_t c; ix_t driveix; @@ -103,18 +103,6 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) } } - /* validate drive count - */ - if ( singlethreaded && drivecnt > 1 ) { - mlog( MLOG_NORMAL, _( - "too many -%c arguments: " - "maximum is %d when running in miniroot\n"), - GETOPT_DUMPDEST, - 1 ); - usage( ); - return BOOL_FALSE; - } - /* allocate an array to hold ptrs to drive descriptors */ if (drivecnt > 0) { @@ -221,8 +209,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) intgen_t score; score = ( * sp->ds_match )( argc, argv, - drivep, - singlethreaded ); + drivep ); if ( ! bestsp || score > bestscore ) { bestsp = sp; bestscore = score; @@ -237,8 +224,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) bestsp->ds_description ); ok = ( * bestsp->ds_instantiate )( argc, argv, - drivep, - singlethreaded ); + drivep ); if ( ! ok ) { return BOOL_FALSE; } diff --git a/common/drive.h b/common/drive.h index 4dc4270..f693976 100644 --- a/common/drive.h +++ b/common/drive.h @@ -140,15 +140,13 @@ struct drive_strategy { */ intgen_t ( * ds_match )( intgen_t argc, char *argv[ ], - struct drive *drivep, - bool_t singlethreaded ); + struct drive *drivep ); /* returns degree of match. drivep has been pre-allocated * and initialized with generic info. */ bool_t ( * ds_instantiate )( intgen_t argc, char *argv[ ], - struct drive *drivep, - bool_t singlethreaded ); + struct drive *drivep ); /* creates a drive manager instance, by filling in the * blanks of the pre-allocated drive descriptor * returns FALSE on failure. @@ -648,7 +646,7 @@ extern size_t partialmax; * * Returns FALSE if utility should be aborted. */ -extern bool_t drive_init1( int argc, char *argv[], bool_t singlethreaded ); +extern bool_t drive_init1( int argc, char *argv[] ); /* drive_init2 - allocate and initialize read and write hdr buffers, diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c index 3ff4d0f..5f5f96c 100644 --- a/common/drive_minrmt.c +++ b/common/drive_minrmt.c @@ -257,8 +257,8 @@ extern int rmtwrite( int, const void *, uint); /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* manager operations */ @@ -404,7 +404,7 @@ static u_int32_t cmdlineblksize = 0; */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { intgen_t fd; intgen_t c; @@ -464,7 +464,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; intgen_t c; @@ -487,9 +487,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) ASSERT( contextp ); memset( ( void * )contextp, 0, sizeof( *contextp )); - /* transfer indication of singlethreadedness to context + /* do not enable a separate I/O thread, + * more testing to be done first... */ - contextp->dc_singlethreadedpr = singlethreaded; + contextp->dc_singlethreadedpr = BOOL_TRUE; /* scan the command line for the I/O buffer ring length * and record checksum request @@ -580,7 +581,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ - if ( singlethreaded ) { + if ( contextp->dc_singlethreadedpr ) { contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ ); ASSERT( contextp->dc_bufp ); } else { diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c index f24d604..71b809f 100644 --- a/common/drive_scsitape.c +++ b/common/drive_scsitape.c @@ -299,8 +299,8 @@ extern int rmtwrite( int, const void *, uint); /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* manager operations */ @@ -506,7 +506,7 @@ is_scsi_driver(char *pathname) */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { struct mtget mt_stat; intgen_t fd; @@ -544,7 +544,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; intgen_t c; @@ -567,9 +567,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) ASSERT( contextp ); memset( ( void * )contextp, 0, sizeof( *contextp )); - /* transfer indication of singlethreadedness to context + /* do not enable a separate I/O thread, + * more testing to be done first... */ - contextp->dc_singlethreadedpr = singlethreaded; + contextp->dc_singlethreadedpr = BOOL_TRUE; /* scan the command line for the I/O buffer ring length * and record checksum request @@ -670,7 +671,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ - if ( singlethreaded ) { + if ( contextp->dc_singlethreadedpr ) { contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ ); ASSERT( contextp->dc_bufp ); } else { diff --git a/common/drive_simple.c b/common/drive_simple.c index cb88aa8..086d162 100644 --- a/common/drive_simple.c +++ b/common/drive_simple.c @@ -98,8 +98,8 @@ extern size_t pgsz; /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* declare manager operators */ @@ -178,7 +178,7 @@ static drive_ops_t drive_ops = { */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { bool_t isrmtpr; struct stat64 statbuf; @@ -221,7 +221,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; diff --git a/common/main.c b/common/main.c index 38b3889..5880723 100644 --- a/common/main.c +++ b/common/main.c @@ -87,10 +87,6 @@ bool_t preemptchk( int ); static bool_t loadoptfile( int *argcp, char ***argvp ); static char * stripquotes( char *p ); static void shiftleftby1( char *p, char *endp ); -static bool_t in_miniroot_heuristic( void ); -#ifdef HIDDEN -static void mrh_sighandler( int ); -#endif static void sighandler( int ); static int childmain( void * ); static bool_t sigint_dialog( void ); @@ -111,11 +107,6 @@ intgen_t version = 3; intgen_t subversion = 0; char *progname = 0; /* used in all error output */ char *homedir = 0; /* directory invoked from */ -#ifdef HIDDEN -bool_t miniroot = BOOL_FALSE; -#else -bool_t miniroot = BOOL_TRUE; -#endif /* HIDDEN */ bool_t pipeline = BOOL_FALSE; bool_t stdoutpiped = BOOL_FALSE; pthread_t parenttid; @@ -215,16 +206,11 @@ main( int argc, char *argv[] ) if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } - /* scan the command line for the miniroot, info, progress + /* scan the command line for the info, progress * report options, and stacksz. */ minstacksz = MINSTACKSZ; maxstacksz = MAXSTACKSZ; -#ifdef HIDDEN - miniroot = BOOL_FALSE; -#else - miniroot = BOOL_TRUE; -#endif /* HIDDEN */ infoonly = BOOL_FALSE; progrpt_enabledpr = BOOL_FALSE; optind = 1; @@ -275,9 +261,6 @@ main( int argc, char *argv[] ) } maxstacksz = tmpstacksz; break; - case GETOPT_MINIROOT: - miniroot = BOOL_TRUE; - break; case GETOPT_HELP: infoonly = BOOL_TRUE; mlog_exit_hint(RV_USAGE); @@ -349,13 +332,6 @@ main( int argc, char *argv[] ) return mlog_exit(EXIT_ERROR, RV_INIT); } - /* perform an experiment to determine if we are in the miniroot. - * various features will be disallowed if in miniroot. - */ - if ( ! miniroot && in_miniroot_heuristic( )) { - miniroot = BOOL_TRUE; - } - /* initialize message logging (stage 2) - allocate the message lock */ ok = mlog_init2( ); @@ -473,7 +449,7 @@ main( int argc, char *argv[] ) * terribly time-consuming here. A second initialization pass * will be done shortly. */ - ok = drive_init1( argc, argv, miniroot ); + ok = drive_init1( argc, argv ); if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } @@ -503,11 +479,7 @@ main( int argc, char *argv[] ) VERSION, version, subversion ); - if ( miniroot ) { - mlog( MLOG_VERBOSE | MLOG_BARE, _( - " - " - "Running single-threaded\n") ); - } else if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) { + if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) { mlog( MLOG_VERBOSE | MLOG_BARE, _( " - " "type %s for status and control\n"), @@ -531,11 +503,10 @@ main( int argc, char *argv[] ) */ mlog_tell_streamcnt( drivecnt ); - /* initialize the state of signal processing. if miniroot or - * pipeline, just want to exit when a signal is received. otherwise, - * hold signals so they don't interfere with sys calls; they will - * be released at pre-emption points and upon pausing in the main - * loop. + /* initialize the state of signal processing. if in a pipeline, just + * want to exit when a signal is received. otherwise, hold signals so + * they don't interfere with sys calls; they will be released at + * pre-emption points and upon pausing in the main loop. */ sigfillset(&sa.sa_mask); @@ -551,7 +522,7 @@ main( int argc, char *argv[] ) sigaction( SIGPIPE, &sa, NULL ); sigaction( SIGCHLD, &sa, NULL ); - if ( ! miniroot && ! pipeline ) { + if ( ! pipeline ) { sigset_t blocked_set; stop_in_progress = BOOL_FALSE; @@ -594,10 +565,9 @@ main( int argc, char *argv[] ) return mlog_exit(EXIT_ERROR, RV_INIT); } - /* if miniroot or a pipeline, go single-threaded - * with just one stream. + /* if in a pipeline, go single-threaded with just one stream. */ - if ( miniroot || pipeline ) { + if ( pipeline ) { intgen_t exitcode; sa.sa_handler = sighandler; @@ -977,9 +947,6 @@ usage( void ) ULO(_("(show verbosity in messages)"), GETOPT_SHOWLOGLEVEL ); #endif /* REVEAL */ ULO(_(""), GETOPT_RINGLEN ); -#ifdef REVEAL - ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT ); -#endif /* REVEAL */ ULN(_("- (stdout)") ); ULN(_("") ); #endif /* DUMP */ @@ -1030,9 +997,6 @@ usage( void ) #endif /* REVEAL */ ULO(_(" ..."), GETOPT_NOSUBTREE ); ULO(_(""), GETOPT_RINGLEN ); -#ifdef REVEAL - ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT ); -#endif /* REVEAL */ ULN(_("- (stdin)") ); ULN(_("") ); #endif /* RESTORE */ @@ -1081,9 +1045,9 @@ preemptchk( int flg ) return BOOL_FALSE; } - /* signals not caught in these cases + /* signals not caught if in a pipeline */ - if ( miniroot || pipeline ) { + if ( pipeline ) { return BOOL_FALSE; } @@ -1400,66 +1364,6 @@ loadoptfile( intgen_t *argcp, char ***argvp ) return BOOL_TRUE; } -#ifdef HIDDEN -static pid_t mrh_cid; -#endif - -static bool_t -in_miniroot_heuristic( void ) -{ - return BOOL_TRUE; - -#ifdef HIDDEN - SIG_PF prev_handler_hup; - SIG_PF prev_handler_term; - SIG_PF prev_handler_int; - SIG_PF prev_handler_quit; - SIG_PF prev_handler_cld; - bool_t in_miniroot; - - /* attempt to call sproc. - */ - prev_handler_hup = sigset( SIGHUP, SIG_IGN ); - prev_handler_term = sigset( SIGTERM, SIG_IGN ); - prev_handler_int = sigset( SIGINT, SIG_IGN ); - prev_handler_quit = sigset( SIGQUIT, SIG_IGN ); - prev_handler_cld = sigset( SIGCLD, mrh_sighandler ); - ( void )sighold( SIGCLD ); - mrh_cid = ( pid_t )sproc( ( void ( * )( void * ))exit, PR_SALL, 0 ); - if ( mrh_cid < 0 ) { - in_miniroot = BOOL_TRUE; - } else { - while ( mrh_cid >= 0 ) { - ( void )sigpause( SIGCLD ); - } - in_miniroot = BOOL_FALSE; - } - ( void )sigset( SIGHUP, prev_handler_hup ); - ( void )sigset( SIGTERM, prev_handler_term ); - ( void )sigset( SIGINT, prev_handler_int ); - ( void )sigset( SIGQUIT, prev_handler_quit ); - ( void )sigset( SIGCLD, prev_handler_cld ); - - return in_miniroot; -#endif /* HIDDEN */ -} - -#ifdef HIDDEN -static void -mrh_sighandler( int signo ) -{ - if ( signo == SIGCLD ) { - pid_t cid; - intgen_t stat; - - cid = wait( &stat ); - if ( cid == mrh_cid ) { - mrh_cid = -1; - } - } -} -#endif - /* parent and children share this handler. */ static void @@ -1470,9 +1374,9 @@ sighandler( int signo ) if ( dlog_sighandler( signo ) ) return; - /* if in miniroot, don't do anything risky. just quit. + /* if in pipeline, don't do anything risky. just quit. */ - if ( miniroot || pipeline ) { + if ( pipeline ) { intgen_t rval; mlog( MLOG_TRACE | MLOG_NOTE | MLOG_NOLOCK | MLOG_PROC, diff --git a/dump/content.c b/dump/content.c index 33f1166..3a7f508 100644 --- a/dump/content.c +++ b/dump/content.c @@ -244,7 +244,6 @@ typedef struct pds pds_t; extern void usage( void ); extern bool_t preemptchk( int ); extern char *homedir; -extern bool_t miniroot; extern bool_t pipeline; extern bool_t stdoutpiped; extern char *sistr; @@ -2352,7 +2351,7 @@ content_stream_dump( ix_t strmix ) inomap_next_nondir, inomap_contextp, ( intgen_t * )&rv, - ( miniroot || pipeline ) ? + pipeline ? (bool_t (*)(int))preemptchk : 0, bstatbufp, bstatbuflen ); @@ -2586,24 +2585,22 @@ decision_more: * from all streams have been registered. */ if ( drivep->d_capabilities & DRIVE_CAP_FILES ) { - if ( ! miniroot ) { - if ( stream_cnt( ) > 1 ) { - mlog( MLOG_VERBOSE, _( - "waiting for synchronized " - "session inventory dump\n") ); - sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC; - } + if ( stream_cnt( ) > 1 ) { + mlog( MLOG_VERBOSE, _( + "waiting for synchronized " + "session inventory dump\n") ); + sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC; + } - /* first be sure all threads have begun - */ - while ( sc_thrdsarrivedcnt < drivecnt ) { - sleep( 1 ); - } - /* now wait for survivors to checkin - */ - while ( sc_thrdsdonecnt < stream_cnt( )) { - sleep( 1 ); - } + /* first be sure all threads have begun + */ + while ( sc_thrdsarrivedcnt < drivecnt ) { + sleep( 1 ); + } + /* now wait for survivors to checkin + */ + while ( sc_thrdsdonecnt < stream_cnt( )) { + sleep( 1 ); } /* proceeed */ diff --git a/dump/getopt.h b/dump/getopt.h index 8a55631..ba26c93 100644 --- a/dump/getopt.h +++ b/dump/getopt.h @@ -27,7 +27,7 @@ * facilitating easy changes. */ -#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:Z" +#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:" #define GETOPT_DUMPASOFFLINE 'a' /* dump DMF dualstate files as offline */ #define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */ @@ -80,6 +80,6 @@ #define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */ /* 'X' */ #define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */ -#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */ +/* 'Z' used on irix for miniroot */ #endif /* GETOPT_H */ diff --git a/man/man8/xfsdump.8 b/man/man8/xfsdump.8 index 46e6bd5..5360378 100644 --- a/man/man8/xfsdump.8 +++ b/man/man8/xfsdump.8 @@ -66,10 +66,6 @@ option. Other techniques, such as making a second copy of the dump image, provide more protection against media failures than multiple media files will. .P -However, the current implementation in Linux only supports one destination -and running single threaded. Therefore, the above comments -regarding multiple streams describe the possible future capabilities. -.P .I xfsdump maintains an online dump inventory in \f2/var/lib/xfsdump/inventory\f1. The @@ -391,8 +387,8 @@ Specify I/O buffer ring length. uses a ring of output buffers to achieve maximum throughput when dumping to tape drives. The default ring length is 3. -However, this is only supported when running multi-threaded -which has not been done for Linux yet - making this option benign. +However, this is not currently enabled on +Linux yet, making this option benign. .TP 5 .B \- A lone diff --git a/man/man8/xfsrestore.8 b/man/man8/xfsrestore.8 index 8e33ee9..aad97fa 100644 --- a/man/man8/xfsrestore.8 +++ b/man/man8/xfsrestore.8 @@ -426,8 +426,8 @@ Specify I/O buffer ring length. uses a ring of input buffers to achieve maximum throughput when restoring from tape drives. The default ring length is 3. -However, this is only supported when running multi-threaded -which has not been done for Linux yet - making this option benign. +However, this is not currently enabled on +Linux yet, making this option benign. .TP 5 .B \- A lone diff --git a/restore/content.c b/restore/content.c index 0108a40..a9e0b20 100644 --- a/restore/content.c +++ b/restore/content.c @@ -660,7 +660,6 @@ typedef struct tran tran_t; extern void usage( void ); extern bool_t preemptchk( void ); extern char *homedir; -extern bool_t miniroot; extern bool_t pipeline; extern bool_t stdoutpiped; extern char *sistr; @@ -2544,14 +2543,12 @@ content_stream_restore( ix_t thrdix ) } tranp->t_sync5 = SYNC_BUSY; unlock( ); - if ( ! miniroot ) { - if ( drivecnt > 1 ) { - mlog( MLOG_TRACE, - "waiting for other streams to exit\n" ); - } - while ( cldmgr_otherstreamsremain( thrdix )) { - sleep( 1 ); - } + if ( drivecnt > 1 ) { + mlog( MLOG_TRACE, + "waiting for other streams to exit\n" ); + } + while ( cldmgr_otherstreamsremain( thrdix )) { + sleep( 1 ); } mlog( MLOG_DEBUG, @@ -3006,10 +3003,10 @@ applydirdump( drive_t *drivep, return RV_INTR; } - /* if miniroot or pipeline , call preemptchk( ) to + /* if in a pipeline , call preemptchk( ) to * print status reports */ - if ( miniroot || pipeline ) + if ( pipeline ) { mlog( MLOG_DEBUG , "preemptchk( )\n"); @@ -3543,10 +3540,10 @@ applynondirdump( drive_t *drivep, fhdrp->fh_offset ); } - /* if miniroot or pipeline , call preemptchk( ) to + /* if in a pipeline , call preemptchk( ) to * print status reports */ - if ( miniroot || pipeline ) + if ( pipeline ) { mlog( MLOG_DEBUG , "preemptchk( )\n"); diff --git a/restore/getopt.h b/restore/getopt.h index 4ee865d..63568de 100644 --- a/restore/getopt.h +++ b/restore/getopt.h @@ -26,7 +26,7 @@ * purpose is to contain that command string. */ -#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:Z" +#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:" #define GETOPT_WORKSPACE 'a' /* workspace dir (content.c) */ #define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */ @@ -79,6 +79,6 @@ #define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */ #define GETOPT_NOSUBTREE 'X' /* subtree to exclude */ #define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */ -#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */ +/* 'Z' used on irix for miniroot */ #endif /* GETOPT_H */ -- 1.7.0.4 From wkendall@sgi.com Wed Nov 2 16:11:13 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64, LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA2LBC4Z200881 for ; Wed, 2 Nov 2011 16:11:12 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id E61D2AC002; Wed, 2 Nov 2011 14:11:11 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 992CD7001EC2; Wed, 2 Nov 2011 16:11:11 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id 8D48621E23D2; Wed, 2 Nov 2011 16:11:11 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads Date: Wed, 2 Nov 2011 16:10:51 -0500 Message-Id: <1320268254-20583-6-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The existing (disabled) threading code in xfsdump is based on IRIX sprocs. This patch converts the code to use pthreads. The threading code remains disabled at this point. Changes: - pid_t to pthread_t - getpid() to pthread_self() - "pid1 == pid2" to pthread_equal(tid1, tid2) - sigprocmask() to pthread_sigmask() - sproc() to pthread_create() Also the following are not referenced and have been removed: - cldmgr_pid2streamix() - PROCMAX - r_slavepid - sproc.c and sproc.h Signed-off-by: Bill Kendall --- common/Makefile | 2 +- common/cldmgr.c | 56 ++++++++++++++-------------- common/cldmgr.h | 8 +--- common/dlog.c | 4 +- common/drive_minrmt.c | 2 +- common/drive_scsitape.c | 2 +- common/main.c | 20 +++++----- common/mlog.c | 44 +++++++++++----------- common/ring.c | 9 +--- common/ring.h | 1 - common/sproc.c | 42 --------------------- common/sproc.h | 23 ----------- common/stream.c | 94 +++++++++++++++++++++-------------------------- common/stream.h | 22 +++++----- dump/Makefile | 4 +- dump/content.c | 4 +- restore/Makefile | 2 - restore/content.c | 5 +- 18 files changed, 127 insertions(+), 217 deletions(-) delete mode 100644 common/sproc.c delete mode 100644 common/sproc.h diff --git a/common/Makefile b/common/Makefile index 8d9d868..ad3d61a 100644 --- a/common/Makefile +++ b/common/Makefile @@ -13,7 +13,7 @@ LSRCFILES = arch_xlate.c arch_xlate.h \ hsmapi.c hsmapi.h inventory.c inventory.h lock.c lock.h \ main.c media.c media.h media_rmvtape.h mlog.c mlog.h \ openutil.c openutil.h path.c path.h qlock.c qlock.h \ - rec_hdr.h ring.c ring.h sproc.c sproc.h stream.c \ + rec_hdr.h ring.c ring.h stream.c \ stream.h timeutil.c timeutil.h ts_mtio.h types.h util.c util.h default install install-dev : diff --git a/common/cldmgr.c b/common/cldmgr.c index d327bab..4574834 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -24,6 +24,7 @@ #include #include #include +#include #include "types.h" #include "lock.h" @@ -31,14 +32,13 @@ #include "stream.h" #include "mlog.h" #include "cldmgr.h" -#include "sproc.h" extern size_t pgsz; #define CLD_MAX ( STREAM_SIMMAX * 2 ) struct cld { bool_t c_busy; - pid_t c_pid; + pthread_t c_tid; ix_t c_streamix; int ( * c_entry )( void *arg1 ); void * c_arg1; @@ -50,32 +50,31 @@ static cld_t cld[ CLD_MAX ]; static bool_t cldmgr_stopflag; static cld_t *cldmgr_getcld( void ); -static cld_t * cldmgr_findbypid( pid_t ); -static int cldmgr_entry( void * ); +static cld_t * cldmgr_findbytid( pthread_t ); +static void *cldmgr_entry( void * ); /* REFERENCED */ -static pid_t cldmgr_parentpid; +static pthread_t cldmgr_parenttid; bool_t cldmgr_init( void ) { ( void )memset( ( void * )cld, 0, sizeof( cld )); cldmgr_stopflag = BOOL_FALSE; - cldmgr_parentpid = getpid( ); + cldmgr_parenttid = pthread_self( ); return BOOL_TRUE; } bool_t cldmgr_create( int ( * entry )( void *arg1 ), - u_intgen_t inh, ix_t streamix, char *descstr, void *arg1 ) { cld_t *cldp; - pid_t cldpid; + intgen_t rval; - ASSERT( getpid( ) == cldmgr_parentpid ); + ASSERT( pthread_equal( pthread_self( ), cldmgr_parenttid ) ); cldp = cldmgr_getcld( ); if ( ! cldp ) { @@ -91,22 +90,22 @@ cldmgr_create( int ( * entry )( void *arg1 ), cldp->c_streamix = streamix; cldp->c_entry = entry; cldp->c_arg1 = arg1; - cldpid = ( pid_t )sproc( cldmgr_entry, inh, ( void * )cldp ); - if ( cldpid < 0 ) { + rval = pthread_create( &cldp->c_tid, NULL, cldmgr_entry, cldp ); + if ( rval ) { mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC, _( - "sproc failed creating %s thread for stream %u: %s\n"), + "failed creating %s thread for stream %u: %s\n"), descstr, streamix, - strerror( errno )); + strerror( rval )); } else { mlog( MLOG_NITTY | MLOG_PROC, - "%s thread created for stream %u: pid %d\n", + "%s thread created for stream %u: tid %lu\n", descstr, streamix, - cldpid ); + cldp->c_tid ); } - return cldpid < 0 ? BOOL_FALSE : BOOL_TRUE; + return rval ? BOOL_FALSE : BOOL_TRUE; } void @@ -119,16 +118,16 @@ cldmgr_stop( void ) } void -cldmgr_died( pid_t pid ) +cldmgr_died( pthread_t tid ) { - cld_t *cldp = cldmgr_findbypid( pid ); + cld_t *cldp = cldmgr_findbytid( tid ); if ( ! cldp ) { return; } cldp->c_busy = BOOL_FALSE; if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_dead( pid ); + stream_dead( tid ); } } @@ -194,13 +193,13 @@ cldmgr_getcld( void ) } static cld_t * -cldmgr_findbypid( pid_t pid ) +cldmgr_findbytid( pthread_t tid ) { cld_t *p = cld; cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); for ( ; p < ep ; p++ ) { - if ( p->c_busy && p->c_pid == pid ) { + if ( p->c_busy && pthread_equal( p->c_tid, tid )) { break; } } @@ -208,19 +207,20 @@ cldmgr_findbypid( pid_t pid ) return ( p < ep ) ? p : 0; } -static int +static void * cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; - pid_t pid = getpid( ); + pthread_t tid = pthread_self( ); - cldp->c_pid = pid; if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_register( pid, ( intgen_t )cldp->c_streamix ); + stream_register( tid, ( intgen_t )cldp->c_streamix ); } mlog( MLOG_DEBUG | MLOG_PROC, - "child %d created for stream %d\n", - pid, + "thread %lu created for stream %d\n", + tid, cldp->c_streamix ); - return ( * cldp->c_entry )( cldp->c_arg1 ); + + ( * cldp->c_entry )( cldp->c_arg1 ); + return NULL; } diff --git a/common/cldmgr.h b/common/cldmgr.h index bb3f612..e393b82 100644 --- a/common/cldmgr.h +++ b/common/cldmgr.h @@ -30,7 +30,6 @@ extern bool_t cldmgr_init( void ); * encountered */ extern bool_t cldmgr_create( int ( * entry )( void *arg1 ), - u_intgen_t inh, ix_t streamix, char *descstr, void *arg1 ); @@ -42,18 +41,13 @@ extern void cldmgr_stop( void ); /* cldmgr_died - tells the child manager that the child died */ -extern void cldmgr_died( pid_t pid ); +extern void cldmgr_died( pthread_t tid ); /* cldmgr_stop_requested - returns TRUE if the child should gracefully * terminate. */ extern bool_t cldmgr_stop_requested( void ); -/* cldmgr_pid2streamix - retrieves the stream index. returns -1 if - * not associated with any stream. - */ -extern intgen_t cldmgr_pid2streamix( pid_t pid ); - /* cldmgr_remainingcnt - returns number of children remaining */ extern size_t cldmgr_remainingcnt( void ); diff --git a/common/dlog.c b/common/dlog.c index 8cf9a65..51666cf 100644 --- a/common/dlog.c +++ b/common/dlog.c @@ -429,7 +429,7 @@ promptinput( char *buf, sigaddset( &dlog_registered_sigs, SIGQUIT ); } - sigprocmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set ); + pthread_sigmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set ); /* wait for input, timeout, or interrupt. * note we come out of the select() frequently in order to @@ -455,7 +455,7 @@ promptinput( char *buf, /* restore signal handling */ - sigprocmask( SIG_SETMASK, &orig_set, NULL ); + pthread_sigmask( SIG_SETMASK, &orig_set, NULL ); sigemptyset( &dlog_registered_sigs ); /* check for timeout or interrupt diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c index 836b663..3ff4d0f 100644 --- a/common/drive_minrmt.c +++ b/common/drive_minrmt.c @@ -577,7 +577,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) drivep->d_cap_est = -1; drivep->d_rate_est = -1; - /* if sproc not allowed, allocate a record buffer. otherwise + /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ if ( singlethreaded ) { diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c index 8c1bd49..f24d604 100644 --- a/common/drive_scsitape.c +++ b/common/drive_scsitape.c @@ -667,7 +667,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) drivep->d_cap_est = -1; drivep->d_rate_est = -1; - /* if sproc not allowed, allocate a record buffer. otherwise + /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ if ( singlethreaded ) { diff --git a/common/main.c b/common/main.c index 25c0838..d4dbe28 100644 --- a/common/main.c +++ b/common/main.c @@ -32,6 +32,7 @@ #include #include #include +#include #include "exit.h" #include "types.h" @@ -117,7 +118,7 @@ bool_t miniroot = BOOL_TRUE; #endif /* HIDDEN */ bool_t pipeline = BOOL_FALSE; bool_t stdoutpiped = BOOL_FALSE; -pid_t parentpid; +pthread_t parenttid; char *sistr; size_t pgsz; size_t pgmask; @@ -195,10 +196,10 @@ main( int argc, char *argv[] ) */ mlog_init0(); - /* Get the parent's pid. will be used in signal handling + /* Get the parent's pthread id. will be used * to differentiate parent from children. */ - parentpid = getpid( ); + parenttid = pthread_self( ); rval = atexit(mlog_exit_flush); assert(rval == 0); @@ -395,11 +396,11 @@ main( int argc, char *argv[] ) ASSERT( ( intgen_t )pgsz > 0 ); pgmask = pgsz - 1; - /* report parent pid + /* report parent tid */ mlog( MLOG_DEBUG | MLOG_PROC, - "parent pid is %d\n", - parentpid ); + "parent tid is %lu\n", + parenttid ); /* get the current working directory: this is where we will dump * core, if necessary. some tmp files may be placed here as well. @@ -572,7 +573,7 @@ main( int argc, char *argv[] ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); - sigprocmask( SIG_SETMASK, &blocked_set, NULL ); + pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); sa.sa_handler = sighandler; sigaction( SIGINT, &sa, NULL ); @@ -676,7 +677,6 @@ main( int argc, char *argv[] ) if ( ! init_error ) { for ( stix = 0 ; stix < drivecnt ; stix++ ) { ok = cldmgr_create( childmain, - CLONE_VM, stix, "child", ( void * )stix ); @@ -895,7 +895,7 @@ main( int argc, char *argv[] ) if ( coredump_requested ) { mlog( MLOG_DEBUG | MLOG_PROC, "core dump requested, aborting (pid %d)\n", - parentpid ); + getpid() ); abort(); } @@ -1560,7 +1560,7 @@ childmain( void *arg1 ) drivep = drivepp[ stix ]; ( * drivep->d_opsp->do_quit )( drivep ); - exit( exitcode ); + return exitcode; } diff --git a/common/mlog.c b/common/mlog.c index 2265895..b0135b9 100644 --- a/common/mlog.c +++ b/common/mlog.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "types.h" #include "qlock.h" @@ -40,7 +41,7 @@ extern char *progname; extern void usage( void ); -extern pid_t parentpid; +extern pthread_t parenttid; #ifdef DUMP static FILE *mlog_fp = NULL; /* stderr */; @@ -385,7 +386,7 @@ mlog_va( intgen_t levelarg, char *fmt, va_list args ) if ( ! ( levelarg & MLOG_BARE )) { intgen_t streamix; - streamix = stream_getix( getpid() ); + streamix = stream_getix( pthread_self( ) ); if ( mlog_showss ) { sprintf( mlog_ssstr, ":%s", mlog_ss_names[ ss ] ); @@ -568,10 +569,10 @@ rv_getdesc(rv_t rv) int _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) { - pid_t pid; + pthread_t tid; const struct rv_map *rvp; - pid = getpid(); + tid = pthread_self(); rvp = rv_getdesc(rv); @@ -595,7 +596,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) * most accurate information about the termination condition. */ - if (pid == parentpid) { + if ( pthread_equal( tid, parenttid ) ) { if (mlog_main_exit_code == -1) { mlog_main_exit_code = exit_code; mlog_main_exit_return = rv; @@ -608,7 +609,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) int exit_code; rv_t exit_return, exit_hint; - if (stream_get_exit_status(pid, + if (stream_get_exit_status(tid, states, N(states), &state, @@ -618,8 +619,8 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) &exit_hint)) { if (exit_code == -1) { - stream_set_code(pid, exit_code); - stream_set_return(pid, rv); + stream_set_code(tid, exit_code); + stream_set_return(tid, rv); } } } @@ -630,10 +631,10 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) void _mlog_exit_hint( const char *file, int line, rv_t rv ) { - pid_t pid; + pthread_t tid; const struct rv_map *rvp; - pid = getpid(); + tid = pthread_self(); rvp = rv_getdesc(rv); mlog( MLOG_DEBUG | MLOG_NOLOCK, @@ -655,10 +656,10 @@ _mlog_exit_hint( const char *file, int line, rv_t rv ) * information about the termination condition. */ - if (pid == parentpid) + if ( pthread_equal( tid, parenttid ) ) mlog_main_exit_hint = rv; else - stream_set_hint( pid, rv ); + stream_set_hint( tid, rv ); } @@ -670,10 +671,10 @@ mlog_get_hint( void ) bool_t ok; rv_t hint; - if (getpid() == parentpid) + if ( pthread_equal( pthread_self(), parenttid ) ) return mlog_main_exit_hint; - ok = stream_get_exit_status(getpid(), states, N(states), + ok = stream_get_exit_status(pthread_self(), states, N(states), NULL, NULL, NULL, NULL, &hint); ASSERT(ok); return hint; @@ -697,8 +698,8 @@ mlog_get_hint( void ) void mlog_exit_flush(void) { - pid_t pids[STREAM_SIMMAX]; - int i, npids; + pthread_t tids[STREAM_SIMMAX]; + int i, ntids; const struct rv_map *rvp; stream_state_t states[] = { S_RUNNING, S_ZOMBIE }; bool_t incomplete = BOOL_FALSE; @@ -713,13 +714,13 @@ mlog_exit_flush(void) if (mlog_main_exit_hint == RV_USAGE) return; - npids = stream_find_all(states, N(states), pids, STREAM_SIMMAX); - if (npids > 0) { + ntids = stream_find_all(states, N(states), tids, STREAM_SIMMAX); + if (ntids > 0) { /* print the state of all the streams */ fprintf(mlog_fp, _("%s: %s Summary:\n"), progname, PROGSTR_CAPS ); - for (i = 0; i < npids; i++) { + for (i = 0; i < ntids; i++) { stream_state_t state; intgen_t streamix; int exit_code; @@ -727,7 +728,7 @@ mlog_exit_flush(void) /* REFERENCED */ bool_t ok; - ok = stream_get_exit_status(pids[i], + ok = stream_get_exit_status(tids[i], states, N(states), &state, @@ -743,11 +744,10 @@ mlog_exit_flush(void) /* print status of this stream */ rvp = rv_getdesc(rv); fprintf(mlog_fp, - _("%s: stream %d (pid %d) %s " + _("%s: stream %d %s " "%s (%s)\n"), progname, streamix, - pids[i], drivepp[streamix]->d_pathname, rvp->rv_string, rvp->rv_desc); diff --git a/common/ring.c b/common/ring.c index e1dddba..42bd7d7 100644 --- a/common/ring.c +++ b/common/ring.c @@ -123,7 +123,6 @@ ring_create( size_t ringlen, /* kick off the slave thread */ ok = cldmgr_create( ring_slave_entry, - CLONE_VM, drive_index, _("slave"), ringp ); @@ -421,11 +420,7 @@ ring_slave_entry( void *ringctxp ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); - sigprocmask( SIG_SETMASK, &blocked_set, NULL ); - - /* record slave pid to be used to kill slave - */ - ringp->r_slavepid = getpid( ); + pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); /* loop reading and precessing messages until told to die */ @@ -498,5 +493,5 @@ ring_slave_entry( void *ringctxp ) ring_slave_put( ringp, msgp ); } - exit( 0 ); + return 0; } diff --git a/common/ring.h b/common/ring.h index 56e3924..caa505c 100644 --- a/common/ring.h +++ b/common/ring.h @@ -134,7 +134,6 @@ struct ring { time32_t r_first_io_time; off64_t r_all_io_cnt; /* ALL BELOW PRIVATE!!! */ - pid_t r_slavepid; size_t r_len; ring_msg_t *r_msgp; size_t r_ready_in_ix; diff --git a/common/sproc.c b/common/sproc.c deleted file mode 100644 index 3b1425a..0000000 --- a/common/sproc.c +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright (c) 2000-2001 Silicon Graphics, Inc. - * All Rights Reserved. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it would be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#include -#include -#include - -#define STACKSIZE 65536 - -int -sproc (int (*entry) (void *), int flags, void *arg) -{ - int retval = -1; -#ifdef HIDDEN - void *newstack; - - if ( (newstack = calloc (1, STACKSIZE)) != NULL ) { - void ** stackp = ((void **)newstack)+(STACKSIZE -1)/sizeof(void*); - - flags |= SIGCHLD; - - retval = clone (entry, stackp, flags, arg); - } -#endif - - return retval; -} diff --git a/common/sproc.h b/common/sproc.h deleted file mode 100644 index 46111ff..0000000 --- a/common/sproc.h +++ /dev/null @@ -1,23 +0,0 @@ -/* - * Copyright (c) 2000-2001 Silicon Graphics, Inc. - * All Rights Reserved. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it would be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - */ -#ifndef SPROC_H -#define SPROC_H - -int sproc (int (*) (void *), int, void *); - -#endif /* SPROC_H */ diff --git a/common/stream.c b/common/stream.c index adaf7c0..48e25ee 100644 --- a/common/stream.c +++ b/common/stream.c @@ -19,18 +19,19 @@ #include #include +#include + #include "types.h" #include "exit.h" #include "stream.h" #include "lock.h" #include "mlog.h" -#define PROCMAX ( STREAM_SIMMAX * 2 + 1 ) #define N(a) (sizeof((a)) / sizeof((a)[0])) struct spm { stream_state_t s_state; - pid_t s_pid; + pthread_t s_tid; intgen_t s_ix; int s_exit_code; rv_t s_exit_return; @@ -38,37 +39,28 @@ struct spm { }; typedef struct spm spm_t; -extern pid_t parentpid; static spm_t spm[ STREAM_SIMMAX * 3 ]; static bool_t initialized = BOOL_FALSE; void stream_init( void ) { -#ifdef HIDDEN - /* REFERENCED */ - intgen_t rval; - - rval = ( intgen_t )usconfig( CONF_INITUSERS, PROCMAX ); - ASSERT( rval >= 0 ); -#endif /* HIDDEN */ - ( void )memset( ( void * )spm, 0, sizeof( spm )); initialized = BOOL_TRUE; } /* * Note that the stream list structure (updated via the stream_* functions) - * is indexed by pid. Multiple processes can be registered against the same - * stream index, typically: the primary content process that does the work; - * and the drive slave process, which just processes stuff off the ring buffer. - * In general having multiple pids registered per stream is not an issue for - * termination status reporting, as the mlog_exit* logging functions only + * is indexed by pthread_t (tid). Multiple processes can be registered against + * the same stream index, typically: the primary content process that does the + * work; and the drive slave process, which just processes stuff off the ring + * buffer. In general having multiple tids registered per stream is not an issue + * for termination status reporting, as the mlog_exit* logging functions only * ever get called out of the primary content process. */ void -stream_register( pid_t pid, intgen_t streamix ) +stream_register( pthread_t tid, intgen_t streamix ) { spm_t *p = spm; spm_t *ep = spm + N(spm); @@ -87,7 +79,7 @@ stream_register( pid_t pid, intgen_t streamix ) if ( p >= ep ) return; - p->s_pid = pid; + p->s_tid = tid; p->s_ix = streamix; p->s_exit_code = -1; p->s_exit_return = RV_NONE; @@ -95,14 +87,14 @@ stream_register( pid_t pid, intgen_t streamix ) } void -stream_dead( pid_t pid ) +stream_dead( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); lock(); for ( ; p < ep ; p++ ) - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { p->s_state = S_ZOMBIE; break; } @@ -111,14 +103,14 @@ stream_dead( pid_t pid ) } void -stream_free( pid_t pid ) +stream_free( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); lock(); for ( ; p < ep ; p++ ) { - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { (void) memset( (void *) p, 0, sizeof(spm_t) ); p->s_state = S_FREE; break; @@ -130,22 +122,22 @@ stream_free( pid_t pid ) int stream_find_all( stream_state_t states[], int nstates, - pid_t pids[], int npids ) + pthread_t tids[], int ntids ) { int i, count = 0; spm_t *p = spm; spm_t *ep = spm + N(spm); - ASSERT(nstates > 0 && npids > 0); + ASSERT(nstates > 0 && ntids > 0); if (!initialized) return 0; /* lock - make sure we get a consistent snapshot of the stream status */ lock(); - for ( ; p < ep && count < npids; p++ ) + for ( ; p < ep && count < ntids; p++ ) for (i = 0; i < nstates; i++) if (p->s_state == states[i]) { - pids[count++] = p->s_pid; + tids[count++] = p->s_tid; break; } unlock(); @@ -153,7 +145,7 @@ stream_find_all( stream_state_t states[], int nstates, } static spm_t * -stream_find( pid_t pid, stream_state_t s[], int nstates ) +stream_find( pthread_t tid, stream_state_t s[], int nstates ) { int i; spm_t *p = spm; @@ -163,7 +155,7 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) /* note we don't lock the stream array in this function */ for ( ; p < ep ; p++ ) - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { /* check state */ for (i = 0; i < nstates; i++) if (p->s_state == s[i]) @@ -174,8 +166,8 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) { static const char *state_strings[] = { "S_FREE", "S_RUNNING", "S_ZOMBIE" }; mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE, - "stream_find(): no stream with pid: %d and state%s:", - pid, nstates == 1 ? "" : "s" ); + "stream_find(): no stream with tid: %lu and state%s:", + tid, nstates == 1 ? "" : "s" ); for (i = 0; i < nstates; i++) mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE, " %s", state_strings[s[i]]); @@ -187,20 +179,18 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) } /* - * Note, the following function is called from two places: - * main.c:sighandler(), and mlog.c:mlog_va() in the first case we - * aren't allowed to take locks, and in the second locking may be - * disabled and we are already protected by another lock. So no - * locking is done in this function. + * Note, the following function is called from mlog.c:mlog_va(), + * where locking may be disabled and we are already protected by + * another lock. So no locking is done in this function. */ intgen_t -stream_getix( pid_t pid ) +stream_getix( pthread_t tid ) { stream_state_t states[] = { S_RUNNING }; spm_t *p; intgen_t ix; - p = stream_find( pid, states, N(states) ); + p = stream_find( tid, states, N(states) ); ix = p ? p->s_ix : -1; return ix; } @@ -213,43 +203,43 @@ stream_getix( pid_t pid ) * streams. */ -#define stream_set(field_name, pid, value) \ +#define stream_set(field_name, tid, value) \ stream_state_t states[] = { S_RUNNING }; \ spm_t *p; \ - pid_t mypid = getpid(); \ + pthread_t mytid = pthread_self(); \ \ - if (mypid != (pid)) { \ + if ( !pthread_equal(mytid, (tid))) { \ mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK, \ "stream_set_" #field_name "(): " \ - "foreign stream (pid %d) " \ - "not permitted to update this stream (pid %d)\n", \ - mypid, (pid)); \ + "foreign stream (tid %lu) " \ + "not permitted to update this stream (tid %lu)\n",\ + mytid, (tid)); \ return; \ } \ \ lock(); \ - p = stream_find( (pid), states, N(states) ); \ + p = stream_find( (tid), states, N(states) ); \ if (p) p->s_exit_ ## field_name = (value); \ unlock(); -void stream_set_code( pid_t pid, int exit_code ) +void stream_set_code( pthread_t tid, int exit_code ) { - stream_set( code, pid, exit_code ); + stream_set( code, tid, exit_code ); } -void stream_set_return( pid_t pid, rv_t rv ) +void stream_set_return( pthread_t tid, rv_t rv ) { - stream_set( return, pid, rv ); + stream_set( return, tid, rv ); } -void stream_set_hint( pid_t pid, rv_t rv ) +void stream_set_hint( pthread_t tid, rv_t rv ) { - stream_set( hint, pid, rv ); + stream_set( hint, tid, rv ); } bool_t -stream_get_exit_status( pid_t pid, +stream_get_exit_status( pthread_t tid, stream_state_t states[], int nstates, stream_state_t *state, @@ -262,7 +252,7 @@ stream_get_exit_status( pid_t pid, spm_t *p; lock(); - p = stream_find( pid, states, nstates ); + p = stream_find( tid, states, nstates ); if (! p) goto unlock; if (state) *state = p->s_state; diff --git a/common/stream.h b/common/stream.h index 984634c..292792e 100644 --- a/common/stream.h +++ b/common/stream.h @@ -42,19 +42,19 @@ typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t; extern void stream_init( void ); -extern void stream_register( pid_t pid, intgen_t streamix ); -extern void stream_dead( pid_t pid ); -extern void stream_free( pid_t pid ); +extern void stream_register( pthread_t tid, intgen_t streamix ); +extern void stream_dead( pthread_t tid ); +extern void stream_free( pthread_t tid ); extern int stream_find_all( stream_state_t states[], int nstates, - pid_t pids[], - int npids ); -extern intgen_t stream_getix( pid_t pid ); -extern void stream_set_code( pid_t pid, int code ); -extern void stream_set_return( pid_t pid, rv_t rv ); -extern void stream_set_hint( pid_t pid, rv_t rv ); -extern bool_t stream_exists( pid_t pid ); -extern bool_t stream_get_exit_status( pid_t pid, + pthread_t tids[], + int ntids ); +extern intgen_t stream_getix( pthread_t tid ); +extern void stream_set_code( pthread_t tid, int code ); +extern void stream_set_return( pthread_t tid, rv_t rv ); +extern void stream_set_hint( pthread_t tid, rv_t rv ); +extern bool_t stream_exists( pthread_t tid ); +extern bool_t stream_get_exit_status( pthread_t tid, stream_state_t states[], int nstates, stream_state_t *state, diff --git a/dump/Makefile b/dump/Makefile index d193f68..97879fa 100644 --- a/dump/Makefile +++ b/dump/Makefile @@ -30,7 +30,6 @@ COMMINCL = \ ts_mtio.h \ types.h \ util.h \ - sproc.h \ rec_hdr.h INVINCL = \ @@ -68,8 +67,7 @@ COMMON = \ ring.c \ stream.c \ timeutil.c \ - util.c \ - sproc.c + util.c LOCALS = \ content.c \ diff --git a/dump/content.c b/dump/content.c index 64bfe54..33f1166 100644 --- a/dump/content.c +++ b/dump/content.c @@ -1666,12 +1666,12 @@ baseuuidbypass: sigaddset( &tty_set, SIGINT ); sigaddset( &tty_set, SIGQUIT ); sigaddset( &tty_set, SIGHUP ); - sigprocmask( SIG_BLOCK, &tty_set, &orig_set ); + pthread_sigmask( SIG_BLOCK, &tty_set, &orig_set ); result = create_inv_session( gwhdrtemplatep, &fsid, mntpnt, fsdevice, subtreecnt, strmix ); - sigprocmask( SIG_SETMASK, &orig_set, NULL ); + pthread_sigmask( SIG_SETMASK, &orig_set, NULL ); if ( !result ) { return BOOL_FALSE; diff --git a/restore/Makefile b/restore/Makefile index ce3d6b4..c6f3f25 100644 --- a/restore/Makefile +++ b/restore/Makefile @@ -25,7 +25,6 @@ COMMINCL = \ qlock.h \ rec_hdr.h \ ring.h \ - sproc.h \ stream.h \ timeutil.h \ ts_mtio.h \ @@ -64,7 +63,6 @@ COMMON = \ path.c \ qlock.c \ ring.c \ - sproc.c \ stream.c \ timeutil.c \ util.c diff --git a/restore/content.c b/restore/content.c index 8dfa456..0108a40 100644 --- a/restore/content.c +++ b/restore/content.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "types.h" #include "timeutil.h" @@ -2227,7 +2228,7 @@ content_stream_restore( ix_t thrdix ) #if DEBUG_DUMPSTREAMS { static int count[STREAM_MAX] = {0}; - intgen_t streamix = stream_getix( getpid() ); + intgen_t streamix = stream_getix( pthread_self() ); if (++(count[streamix]) == 30) { mlog( MLOG_TRACE, "still waiting for dirs to be restored\n"); @@ -2390,7 +2391,7 @@ content_stream_restore( ix_t thrdix ) #if DEBUG_DUMPSTREAMS { static int count[STREAM_MAX] = {0}; - intgen_t streamix = stream_getix( getpid() ); + intgen_t streamix = stream_getix( pthread_self() ); if (++(count[streamix]) == 30) { mlog( MLOG_NORMAL, "still waiting for dirs post-processing\n"); -- 1.7.0.4 From wu.bo@cn.fujitsu.com Wed Nov 2 22:06:34 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA336Xkv223464 for ; Wed, 2 Nov 2011 22:06:34 -0500 X-ASG-Debug-ID: 1320289591-2f5700ce0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3EF2E1000B26 for ; Wed, 2 Nov 2011 20:06:31 -0700 (PDT) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id 5czU2twXUzVRxwOF for ; Wed, 02 Nov 2011 20:06:31 -0700 (PDT) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id 361EF170135; Thu, 3 Nov 2011 11:06:30 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA336Smr008254; Thu, 3 Nov 2011 11:06:29 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110311063345-40403 ; Thu, 3 Nov 2011 11:06:33 +0800 Message-ID: <4EB20505.7070005@cn.fujitsu.com> Date: Thu, 03 Nov 2011 11:05:41 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 0/3] xfstests: Add a group of reservation space test Subject: [PATCH 0/3] xfstests: Add a group of reservation space test X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:06:33, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:06:35, Serialize complete at 2011-11-03 11:06:35 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320289592 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0404 1.0000 -1.7607 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.76 X-Barracuda-Spam-Status: No, SCORE=-1.76 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79166 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch set add a group of reservation space test. Especailly for btrfs. It includes three parts: copy workload, prealloc and write posix. For test 264, I hope it's usefull for Josef's reserve improve work. For test 265 and 266, the current btrfs is not pass yet. TO avoid fill the huge disk, I just resize the filesystem size which NOT all filesystems supportted. Here I just make these tests support btrfs only, but it's can ream to ext3/4 if needed. Wu Bo (3): xfstests 264: add a copy and reserve test xfstests 265: add a prealloc and reserve test xfstests 266: add a write and reserve test 264 | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 264.out | 6 ++ 265 | 107 ++++++++++++++++++++++++++++++++++++++++++ 265.out | 5 ++ 266 | 104 +++++++++++++++++++++++++++++++++++++++++ 266.out | 5 ++ group | 3 + 7 files changed, 388 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out create mode 100755 265 create mode 100644 265.out create mode 100755 266 create mode 100644 266.out -- 1.7.3.1 From wu.bo@cn.fujitsu.com Wed Nov 2 22:09:53 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA339qr8223641 for ; Wed, 2 Nov 2011 22:09:52 -0500 X-ASG-Debug-ID: 1320289790-2f4400e40000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BFA3F1000B40 for ; Wed, 2 Nov 2011 20:09:50 -0700 (PDT) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id kFEWbfJ8WUzAQZ9E for ; Wed, 02 Nov 2011 20:09:50 -0700 (PDT) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id AB18F17011B; Thu, 3 Nov 2011 11:09:44 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA339gLD008429; Thu, 3 Nov 2011 11:09:43 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110311094747-40448 ; Thu, 3 Nov 2011 11:09:47 +0800 Message-ID: <4EB205C7.6010906@cn.fujitsu.com> Date: Thu, 03 Nov 2011 11:08:55 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 1/3] xfstests 264: add a copy and reserve test Subject: [PATCH 1/3] xfstests 264: add a copy and reserve test References: <4EB20505.7070005@cn.fujitsu.com> In-Reply-To: <4EB20505.7070005@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:47, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:49, Serialize complete at 2011-11-03 11:09:49 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320289791 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79166 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is a stress test. It creates a set of threads for coping small files into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is not full under kenerl 3.0 with intel64. Signed-off-by: Wu Bo --- 264 | 158 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 264.out | 6 ++ group | 1 + 3 files changed, 165 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out diff --git a/264 b/264 new file mode 100755 index 0000000..8df56c2 --- /dev/null +++ b/264 @@ -0,0 +1,158 @@ +#! /bin/bash +# FS QA Test No. 264 +# +# reservation test with heavy cp workload +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +threads=0 # will be set later +count=1 + +_threads_set() +{ + _cpu_num=`cat /proc/cpuinfo | grep "processor" | wc -l` + threads=$(( $_cpu_num * 50 )) + if [ $threads -gt 200 ] + then + threads=200 + fi + + echo "create $threads threads to run..." +} + +_file_create() +{ + _i=0 + + if ! mkdir $TEST_DIR/origin + then + echo "mkdir origin err" + status=1 + exit + fi + + cd $TEST_DIR/origin + + _disksize=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` + _disksize=$(( $_disksize / 3 )) + _num=$(( $_disksize / $count / $threads / 4096 )) + _count=$count + while [ $_i -lt $_num ] + do + dd if=/dev/zero of=file_$_i bs=4096 count=$_count > /dev/null 2>&1 + _i=$(( $_i + 1 )) + done + + cd $here +} + +_porter() +{ + _suffix=$1 + + if ! mkdir $TEST_DIR/sub_$_suffix + then + echo "mkdir sub_xxx err" + status=1 + exit + fi + + cp -r $TEST_DIR/origin $TEST_DIR/sub_$_suffix + + sync +} + +_do_workload() +{ + _pid=1 + + rm -rf $TEST_DIR/* + + _threads_set + _file_create + + _threads=$threads + + while [ $_pid -lt $_threads ] + do + _porter $_pid & + + _pid=$(( $_pid + 1 )) + done + + wait +} + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "start the workload" +echo "------------------------------" + +_resize=0 +_new=$(( 2 * 1024 * 1024 * 1024 )) # 2G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +_do_workload + +echo "done" +exit diff --git a/264.out b/264.out new file mode 100644 index 0000000..b218578 --- /dev/null +++ b/264.out @@ -0,0 +1,6 @@ +QA output created by 264 +------------------------------ +start the workload +------------------------------ +create 100 threads to run... +done diff --git a/group b/group index 2a8970c..301aaec 100644 --- a/group +++ b/group @@ -377,3 +377,4 @@ deprecated 261 auto quick quota 262 auto quick quota 263 rw auto quick +264 auto rw -- 1.7.3.1 From wu.bo@cn.fujitsu.com Wed Nov 2 22:09:56 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA339tWM223655 for ; Wed, 2 Nov 2011 22:09:56 -0500 X-ASG-Debug-ID: 1320289793-196701d80000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EB0421000B42 for ; Wed, 2 Nov 2011 20:09:53 -0700 (PDT) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id 1JFxKAODt7bsjt8F for ; Wed, 02 Nov 2011 20:09:53 -0700 (PDT) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id C3C66170135; Thu, 3 Nov 2011 11:09:52 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA339lF0008451; Thu, 3 Nov 2011 11:09:50 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110311095548-40459 ; Thu, 3 Nov 2011 11:09:55 +0800 Message-ID: <4EB205CF.8040202@cn.fujitsu.com> Date: Thu, 03 Nov 2011 11:09:03 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 3/3] xfstests 266: add a write and reserve test Subject: [PATCH 3/3] xfstests 266: add a write and reserve test References: <4EB20505.7070005@cn.fujitsu.com> In-Reply-To: <4EB20505.7070005@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:55, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:57, Serialize complete at 2011-11-03 11:09:57 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320289794 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79166 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is for write-posix test. If writing a file when the disk is almost full, the posix wants the call to write as much as possible but not none. quote the POSIX: If a write() requests that more bytes be written than there is room for (for example, [XSI] [Option Start] the process' file size limit or [Option End] the physical end of a medium), only as many bytes as there is room for shall be written. For example, suppose there is space for 20 bytes more in a file before reaching a limit. A write of 512 bytes will return 20. The next write of a non-zero number of bytes would give a failure return (except as noted below). Signed-off-by: Wu Bo --- 266 | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 266.out | 5 +++ group | 1 + 3 files changed, 110 insertions(+), 0 deletions(-) create mode 100755 266 create mode 100644 266.out diff --git a/266 b/266 new file mode 100755 index 0000000..6331ff8 --- /dev/null +++ b/266 @@ -0,0 +1,104 @@ +#! /bin/bash +# FS QA Test No. 266 +# +# The posix write test. when write size is larger than disk free size, +# should write as more as possible +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "write lack test" +echo "------------------------------" + +_resize=0 +_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +rm -rf $TEST_DIR/* +cd $TEST_DIR +dd if=/dev/zero of=tmp1 bs=4K count=1 > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp2 bs=1M > /dev/null 2>&1 +dd if=/dev/zero of=tmp3 bs=4K > /dev/null 2>&1 +sync + +rm -f tmp1 +sync + +dd if=/dev/zero of=tmp1 bs=8K count=1 > /dev/null 2>&1 +_filesize=`du tmp1 | awk '{print $1}'` +if [ $_filesize -ne 4 ] +then + echo "write file err" + status=1 + exit +fi + +echo "done" +exit diff --git a/266.out b/266.out new file mode 100644 index 0000000..ce3340c --- /dev/null +++ b/266.out @@ -0,0 +1,5 @@ +QA output created by 266 +------------------------------ +write lack test +------------------------------ +done diff --git a/group b/group index f76ab8c..cf93ef9 100644 --- a/group +++ b/group @@ -379,3 +379,4 @@ deprecated 263 rw auto quick 264 auto rw 265 auto rw +266 auto rw -- 1.7.3.1 From wu.bo@cn.fujitsu.com Wed Nov 2 22:09:58 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_24, J_CHICKENPOX_47 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA339vNl223666 for ; Wed, 2 Nov 2011 22:09:57 -0500 X-ASG-Debug-ID: 1320289795-2f5600d00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A148C1000B45 for ; Wed, 2 Nov 2011 20:09:55 -0700 (PDT) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id jgS0xS33c1hTzPIA for ; Wed, 02 Nov 2011 20:09:55 -0700 (PDT) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id 005C3170134; Thu, 3 Nov 2011 11:09:49 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA339lEx008451; Thu, 3 Nov 2011 11:09:48 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110311095266-40456 ; Thu, 3 Nov 2011 11:09:52 +0800 Message-ID: <4EB205CC.6010509@cn.fujitsu.com> Date: Thu, 03 Nov 2011 11:09:00 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 2/3] xfstests 265: add a prealloc and reserve test Subject: [PATCH 2/3] xfstests 265: add a prealloc and reserve test References: <4EB20505.7070005@cn.fujitsu.com> In-Reply-To: <4EB20505.7070005@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:52, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 11:09:55, Serialize complete at 2011-11-03 11:09:55 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320289796 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79166 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not. Signed-off-by: Wu Bo --- 265 | 107 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 265.out | 5 +++ group | 1 + 3 files changed, 113 insertions(+), 0 deletions(-) create mode 100755 265 create mode 100644 265.out diff --git a/265 b/265 new file mode 100755 index 0000000..13c3520 --- /dev/null +++ b/265 @@ -0,0 +1,107 @@ +#! /bin/bash +# FS QA Test No. 265 +# +# preallocation test +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + rm -f $TEST_DIR/* $tmp.* + + if [ $_resize -eq 1 ] + then + btrfs filesystem resize ${_old}B $TEST_DIR > /dev/null 2>&1 + fi + + _cleanup_testdir +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs btrfs +_supported_os IRIX Linux + +_setup_testdir +_check_test_fs + +echo "------------------------------" +echo "preallocation test" +echo "------------------------------" + +_resize=0 +_new=$(( 1 * 1024 * 1024 * 1024 )) # 1G +_old=`df --block-size=1 | grep $TEST_DEV | awk '{print $2}'` +if [ $_new -lt $_old ] +then + btrfs filesystem resize ${_new}B $TEST_DIR > /dev/null 2>&1 + if [ $? -ne 0 ] + then + echo "btrfs filesystem resize err" + status=1 + exit + fi + _resize=1 +fi + +rm -rf $TEST_DIR/* +cd $TEST_DIR +dd if=/dev/zero of=test bs=4K count=1 > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +fallocate -n -o 4K -l 1M test > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fallocate file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp1 bs=1M > /dev/null 2>&1 +dd if=/dev/zero of=tmp2 bs=4K > /dev/null 2>&1 +sync + +dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc > /dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fill prealloc range err" + status=1 + exit +fi + +echo "done" +exit diff --git a/265.out b/265.out new file mode 100644 index 0000000..202ca6a --- /dev/null +++ b/265.out @@ -0,0 +1,5 @@ +QA output created by 265 +------------------------------ +preallocation test +------------------------------ +done diff --git a/group b/group index 301aaec..f76ab8c 100644 --- a/group +++ b/group @@ -378,3 +378,4 @@ deprecated 262 auto quick quota 263 rw auto quick 264 auto rw +265 auto rw -- 1.7.3.1 From amit.sahrawat83@gmail.com Thu Nov 3 01:07:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA36777C239405 for ; Thu, 3 Nov 2011 01:07:08 -0500 X-ASG-Debug-ID: 1320300425-4cc3007c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B8586FDF726 for ; Wed, 2 Nov 2011 23:07:05 -0700 (PDT) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id e1lraA5qKhpdMJMO for ; Wed, 02 Nov 2011 23:07:05 -0700 (PDT) Received: by faai28 with SMTP id i28so1297626faa.26 for ; Wed, 02 Nov 2011 23:07:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=QugE0D6fBpobddBtCAoGKcNeVtiN2FphR74kEwGh1is=; b=fT/Myr33+TMH+5iiUgMsDpcFnsOty9gQQhc1n2wGNQ3rMgBMQxTFIgSanX05BOIxlP e6NMMe0XIoFCNnjP0zPkuOKeK5dtpbWcAaLg6s1C9SqT21jBk1BtQkaEH1qpR8rCzYaP u2L0HBc0Sc3H9woUk0PwEcao6v/FUllz5babk= MIME-Version: 1.0 Received: by 10.223.76.27 with SMTP id a27mr14005973fak.12.1320300423454; Wed, 02 Nov 2011 23:07:03 -0700 (PDT) Received: by 10.152.10.35 with HTTP; Wed, 2 Nov 2011 23:07:03 -0700 (PDT) Date: Thu, 3 Nov 2011 11:37:03 +0530 Message-ID: X-ASG-Orig-Subj: [Patch] xfs: serialise unaligned direct IOs Subject: [Patch] xfs: serialise unaligned direct IOs From: Amit Sahrawat To: Christoph Hellwig , Dave Chinner , xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320300426 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79178 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is needed for long term kernel 2.6.35.14. Please let me know for any changes/suggestions. Thanks & Regards, Amit Sahrawat xfs: serialise unaligned direct IOs This patch published in 2.6.38 kernel(Original reference http://oss.sgi.com/archives/xfs/2011-01/msg00013.html), but can not be applied to 2.6.35 kernel directly, because of the absence of required function, its reimplmented to resolve xfstest test 240 fail. When two concurrent unaligned, non-overlapping direct IOs are issued to the same block, the direct Io layer will race to zero the block. The result is that one of the concurrent IOs will overwrite data written by the other IO with zeros. This is demonstrated by the xfsqa test 240. To avoid this problem, serialise all unaligned direct IOs to an inode with a big hammer. We need a big hammer approach as we need to serialise AIO as well, so we can't just block writes on locks. Hence, the big hammer is calling xfs_ioend_wait() while holding out other unaligned direct IOs from starting. We don't bother trying to serialised aligned vs unaligned IOs as they are overlapping IO and the result of concurrent overlapping IOs is undefined - the result of either IO is a valid result so we let them race. Hence we only penalise unaligned IO, which already has a major overhead compared to aligned IO so this isn't a major problem. diff -Nurp linux-Orig/fs/xfs/linux-2.6/xfs_file.c linux-Updated/fs/xfs/linux-2.6/xfs_file.c --- linux-Orig/fs/xfs/linux-2.6/xfs_file.c 2011-10-28 12:10:52.000000000 +0530 +++ linux-Updated/fs/xfs/linux-2.6/xfs_file.c 2011-10-29 12:34:45.000000000 +0530 @@ -587,6 +587,7 @@ xfs_file_aio_write( xfs_fsize_t isize, new_size; int iolock; size_t ocount = 0, count; + int unaligned_io = 0; int need_i_mutex; XFS_STATS_INC(xs_write_calls); @@ -641,7 +642,10 @@ start: return XFS_ERROR(-EINVAL); } - if (!need_i_mutex && (mapping->nrpages || pos > ip->i_size)) { + if ((pos & mp->m_blockmask) || ((pos + count) & mp->m_blockmask)) + unaligned_io = 1; + + if (!need_i_mutex && ( unaligned_io || mapping->nrpages || pos > ip->i_size)) { xfs_iunlock(ip, XFS_ILOCK_EXCL|iolock); iolock = XFS_IOLOCK_EXCL; need_i_mutex = 1; @@ -700,11 +704,15 @@ start: } if (need_i_mutex) { - /* demote the lock now the cached pages are gone */ - xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); + if (unaligned_io) + xfs_ioend_wait(ip); + /* demote the lock now the cached pages are gone if we can */ + else { + xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); + iolock = XFS_IOLOCK_SHARED; + } mutex_unlock(&inode->i_mutex); - iolock = XFS_IOLOCK_SHARED; need_i_mutex = 0; } From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 01:55:29 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA36tT43242929 for ; Thu, 3 Nov 2011 01:55:29 -0500 X-ASG-Debug-ID: 1320303328-721501ee0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id ACF361CDD227 for ; Wed, 2 Nov 2011 23:55:28 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id PHJCrHHco6G74ofu for ; Wed, 02 Nov 2011 23:55:28 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLrCv-0002kQ-4A; Thu, 03 Nov 2011 06:55:25 +0000 Date: Thu, 3 Nov 2011 02:55:25 -0400 From: Christoph Hellwig To: WuBo Cc: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 1/3] xfstests 264: add a copy and reserve test Subject: Re: [PATCH 1/3] xfstests 264: add a copy and reserve test Message-ID: <20111103065525.GA6800@infradead.org> References: <4EB20505.7070005@cn.fujitsu.com> <4EB205C7.6010906@cn.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EB205C7.6010906@cn.fujitsu.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320303328 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79181 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 11:08:55AM +0800, WuBo wrote: > This test is a stress test. It creates a set of threads for coping small files > into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is > not full under kenerl 3.0 with intel64. It seems like you really want to use _scratch_mkfs_sized instead of the btrfs-specific resize option and make the test generic. Also we already have used up these test numbers in the xfstests-dev repository, and I will commit another batch of tests today. From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 02:02:52 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA372qBX243522 for ; Thu, 3 Nov 2011 02:02:52 -0500 X-ASG-Debug-ID: 1320303769-3ad900010000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6E2E31CDD393 for ; Thu, 3 Nov 2011 00:02:49 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id lOIZQyT3Kk4GWXMC for ; Thu, 03 Nov 2011 00:02:49 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLrK2-0003m6-Qu; Thu, 03 Nov 2011 07:02:46 +0000 Date: Thu, 3 Nov 2011 03:02:46 -0400 From: Christoph Hellwig To: Amit Sahrawat Cc: Christoph Hellwig , Dave Chinner , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [Patch] xfs: serialise unaligned direct IOs Subject: Re: [Patch] xfs: serialise unaligned direct IOs Message-ID: <20111103070246.GA10579@infradead.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320303771 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79181 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 11:37:03AM +0530, Amit Sahrawat wrote: > This is needed for long term kernel 2.6.35.14. > Please let me know for any changes/suggestions. It sounds like a fine candidate to backport, although the context differs a lot from the actual changes commited to mainline. A few comments below: > they are overlapping IO and the result of concurrent overlapping IOs > is undefined - the result of either IO is a valid result so we let > them race. Hence we only penalise unaligned IO, which already has a > major overhead compared to aligned IO so this isn't a major problem. > You probably should keep the original Signoff and reviewed-by tags, and add your editor note on the top into [ ] brackets. > + if (!need_i_mutex && ( unaligned_io || mapping->nrpages || pos > > ip->i_size)) { no space after the opening brace please, and split overly-long lines into two: if (!need_i_mutex && (unaligned_io || mapping->nrpages || pos > ip->i_size)) { > if (need_i_mutex) { > - /* demote the lock now the cached pages are gone */ > - xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); > + if (unaligned_io) > + xfs_ioend_wait(ip); > + /* demote the lock now the cached pages are gone if we can */ > + else { > + xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); > + iolock = XFS_IOLOCK_SHARED; > + } Please use the comment that was used upstream here: /* * If we are doing unaligned IO, wait for all other IO * to drain, otherwise demote the lock if we had to * flush cached pages. */ > mutex_unlock(&inode->i_mutex); > > - iolock = XFS_IOLOCK_SHARED; > need_i_mutex = 0; You also need to make the i_mutex unlock and need_i_mutex update conditional here, otherwise you still serialize all O_DIRECT writes. From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 02:10:57 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA37AvZp244154 for ; Thu, 3 Nov 2011 02:10:57 -0500 X-ASG-Debug-ID: 1320304255-096e01460000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 29BDA21BC0A; Thu, 3 Nov 2011 00:10:55 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 5h9yRudUaqbHFPXg; Thu, 03 Nov 2011 00:10:55 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLrRu-0005jv-VT; Thu, 03 Nov 2011 07:10:55 +0000 Date: Thu, 3 Nov 2011 03:10:54 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 3/8] xfsdump: implement lock abstraction with pthreads Subject: Re: [PATCH 3/8] xfsdump: implement lock abstraction with pthreads Message-ID: <20111103071054.GB10579@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-4-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-4-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320304256 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79181 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean > +struct qsem { > + ix_t qs_count; > + /* current count of this semaphore > + */ > + ix_t qs_waiters; > + /* number of threads waiting on semaphore > + */ > + pthread_mutex_t qs_mutex; > + pthread_cond_t qs_cond; > +}; > +typedef struct qsem qsem_t; Can't you use Posix Semaphores directly? See http://www.csc.villanova.edu/~mdamian/threads/posixsem.html for a quick introduction. From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:21:37 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38LbGL249663 for ; Thu, 3 Nov 2011 03:21:37 -0500 X-ASG-Debug-ID: 1320308495-3ad501020000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 82F671CDD597; Thu, 3 Nov 2011 01:21:36 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id BoFCOYRfnhQxhfgK; Thu, 03 Nov 2011 01:21:36 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsYJ-0003DR-KJ; Thu, 03 Nov 2011 08:21:35 +0000 Date: Thu, 3 Nov 2011 04:21:35 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 1/8] xfsdump: link with libpthread Subject: Re: [PATCH 1/8] xfsdump: link with libpthread Message-ID: <20111103082135.GA11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-2-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-2-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320308496 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79187 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 02, 2011 at 04:10:47PM -0500, Bill Kendall wrote: > This patch changes xfsdump/xfsrestore to link with libpthread in > preparation for enabling multi-stream support. The libpthread checks > are the same as those used in xfsprogs. Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:22:50 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38MomO249750 for ; Thu, 3 Nov 2011 03:22:50 -0500 X-ASG-Debug-ID: 1320308568-2d04012f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E6F71CDD5B9; Thu, 3 Nov 2011 01:22:49 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id t5BKXm2LkMGHPVS1; Thu, 03 Nov 2011 01:22:49 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsZU-0003Sq-Ni; Thu, 03 Nov 2011 08:22:48 +0000 Date: Thu, 3 Nov 2011 04:22:48 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Subject: Re: [PATCH 2/8] xfsdump: remove multi-stream synchronous dir dump Message-ID: <20111103082248.GB11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-3-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-3-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320308569 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79187 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 02, 2011 at 04:10:48PM -0500, Bill Kendall wrote: > When doing multi-stream dumps, the directories are dumped by each > stream in lock-step fashion. A stream backs up a given directory, > then waits for all others to finish that directory, then they all > move on to the next one. During restore a single stream is chosen to > apply the directory dump. > > This patch changes xfsdump so that only one stream dumps the > directories. The other streams begin dumping files immediately. > This results in a faster backup, which is presumably why multiple > streams are being used in the first place. > > It also removes the barrier routines in qlock.c as they were only > used by the "SYNCDIR" code. > > Signed-off-by: Bill Kendall Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:30:15 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38UErv250445 for ; Thu, 3 Nov 2011 03:30:14 -0500 X-ASG-Debug-ID: 1320309012-4cd102a80000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 05C221001FA2; Thu, 3 Nov 2011 01:30:12 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id Z8RHnDHWuggIWl9o; Thu, 03 Nov 2011 01:30:12 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsge-0005cy-7n; Thu, 03 Nov 2011 08:30:12 +0000 Date: Thu, 3 Nov 2011 04:30:12 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap Subject: Re: [PATCH 4/8] xfsdump: simplify qlock ordinal bitmap Message-ID: <20111103083012.GC11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-5-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-5-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320309014 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79186 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:31:13 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38VCsd250508 for ; Thu, 3 Nov 2011 03:31:13 -0500 X-ASG-Debug-ID: 1320309071-0969026d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3EA2021C033; Thu, 3 Nov 2011 01:31:11 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id UHiEQ2uqz13cM1vA; Thu, 03 Nov 2011 01:31:11 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLshb-0005fT-Jv; Thu, 03 Nov 2011 08:31:11 +0000 Date: Thu, 3 Nov 2011 04:31:11 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads Subject: Re: [PATCH 5/8] xfsdump: convert IRIX sproc threads to pthreads Message-ID: <20111103083111.GD11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-6-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-6-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320309072 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79187 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:32:48 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38Wmc9250663 for ; Thu, 3 Nov 2011 03:32:48 -0500 X-ASG-Debug-ID: 1320309167-6f04037f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BF0DA1CDD948; Thu, 3 Nov 2011 01:32:47 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 3xTJ8LXxf3LcMfz0; Thu, 03 Nov 2011 01:32:47 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsj9-0005hg-1S; Thu, 03 Nov 2011 08:32:47 +0000 Date: Thu, 3 Nov 2011 04:32:47 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfsdump: process thread exit status Subject: Re: [PATCH 6/8] xfsdump: process thread exit status Message-ID: <20111103083246.GE11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-7-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-7-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320309167 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79187 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:33:10 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38XA7u250718 for ; Thu, 3 Nov 2011 03:33:10 -0500 X-ASG-Debug-ID: 1320309189-2d1001810000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CE7221CDD94C; Thu, 3 Nov 2011 01:33:09 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id pOyR4VjmDC4f6ExF; Thu, 03 Nov 2011 01:33:09 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsjV-0005iA-81; Thu, 03 Nov 2011 08:33:09 +0000 Date: Thu, 3 Nov 2011 04:33:09 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/8] xfsdump: path lookup cache must be thread specific Subject: Re: [PATCH 7/8] xfsdump: path lookup cache must be thread specific Message-ID: <20111103083309.GF11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-8-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-8-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320309189 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0017 1.0000 -2.0097 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.91 X-Barracuda-Spam-Status: No, SCORE=-1.91 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79187 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 02, 2011 at 04:10:53PM -0500, Bill Kendall wrote: > The xfsrestore path lookup routine (maps an ino + generation number to a > pathname) contains a cache for fast lookups of consecutive files in the > same parent directory. This cache either needs to be protected by a lock > or a cache is needed per thread. I took the latter approach since it > doesn't add lock contention and it increases the cache hit rate (it's > relatively unlikely that each stream is processing files in the same > directory). > > Signed-off-by: Bill Kendall Looks good, Reviewed-by: Christoph Hellwig From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 03:36:15 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA38aEFj250977 for ; Thu, 3 Nov 2011 03:36:15 -0500 X-ASG-Debug-ID: 1320309373-076b019c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 12D65F420AF; Thu, 3 Nov 2011 01:36:13 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id tjGSRWuF37HHfYmX; Thu, 03 Nov 2011 01:36:13 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLsmS-0006br-WA; Thu, 03 Nov 2011 08:36:13 +0000 Date: Thu, 3 Nov 2011 04:36:12 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 8/8] xfsdump: enable multiple streams Subject: Re: [PATCH 8/8] xfsdump: enable multiple streams Message-ID: <20111103083612.GG11854@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <1320268254-20583-9-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-9-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320309374 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79188 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good, Reviewed-by: Christoph Hellwig From wu.bo@cn.fujitsu.com Thu Nov 3 04:06:05 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3965PU253331 for ; Thu, 3 Nov 2011 04:06:05 -0500 X-ASG-Debug-ID: 1320311161-2d04020f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 37ED31CDCBCD for ; Thu, 3 Nov 2011 02:06:02 -0700 (PDT) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id sRKMe4YLRaBMTvFu for ; Thu, 03 Nov 2011 02:06:02 -0700 (PDT) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id 42359170168; Thu, 3 Nov 2011 17:06:00 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA395s4K014533; Thu, 3 Nov 2011 17:05:59 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110316054504-43202 ; Thu, 3 Nov 2011 16:05:45 +0800 Message-ID: <4EB24B1B.2030304@cn.fujitsu.com> Date: Thu, 03 Nov 2011 16:04:43 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: Christoph Hellwig CC: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 1/3] xfstests 264: add a copy and reserve test Subject: Re: [PATCH 1/3] xfstests 264: add a copy and reserve test References: <4EB20505.7070005@cn.fujitsu.com> <4EB205C7.6010906@cn.fujitsu.com> <20111103065525.GA6800@infradead.org> In-Reply-To: <20111103065525.GA6800@infradead.org> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 16:05:45, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-03 17:06:05, Serialize complete at 2011-11-03 17:06:05 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320311164 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0039 1.0000 -1.9954 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.00 X-Barracuda-Spam-Status: No, SCORE=-2.00 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79189 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 11/03/2011 02:55 PM, Christoph Hellwig wrote: > On Thu, Nov 03, 2011 at 11:08:55AM +0800, WuBo wrote: >> This test is a stress test. It creates a set of threads for coping small files >> into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is >> not full under kenerl 3.0 with intel64. > > It seems like you really want to use _scratch_mkfs_sized instead of the > btrfs-specific resize option and make the test generic. Also we already OK, thanks for _scratch_mkfs_sized, It's will make these tests generic. > have used up these test numbers in the xfstests-dev repository, and I > will commit another batch of tests today. I will change my test numbers in next version. thanks, wubo > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > From rjevskiy@gmail.com Thu Nov 3 04:34:49 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA39YnJ5255286 for ; Thu, 3 Nov 2011 04:34:49 -0500 X-ASG-Debug-ID: 1320312887-500a026d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4F98821C649 for ; Thu, 3 Nov 2011 02:34:47 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id vUgnUAMuzMNeQQdR for ; Thu, 03 Nov 2011 02:34:47 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1117298bkb.26 for ; Thu, 03 Nov 2011 02:34:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=dLwsTAGlNORnbrjoR6n6sTsxLFTO7B9Pl+rQ0rbdw9Q=; b=mMHg6PAfZdA3wVQT+qmg/T9KENicR/LLGRjvXX5jv8igNf9FRyf7OPlb1jrQKfOnTz BAJOitSnVy5ABcseCxuak9YN9QGAgcDZC4NXjeX/vKB/y6yqxPkaaECDMslB9zThCkVB C7WHIlzhCcsrUMLnkPQM41X0Vl+ZBeUPvdu64= Received: by 10.204.140.129 with SMTP id i1mr7249763bku.19.1320312886902; Thu, 03 Nov 2011 02:34:46 -0700 (PDT) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id b20sm949411bkr.11.2011.11.03.02.34.43 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 02:34:44 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Christoph Hellwig Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress In-Reply-To: <20111102195534.GB22500@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Thu, 03 Nov 2011 13:34:41 +0400 Message-ID: <87zkgdy2fi.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320312888 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79191 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2 Nov 2011 15:55:34 -0400, Christoph Hellwig wrote: > On Sat, Oct 29, 2011 at 04:48:15AM +0400, Dmitry Monakhov wrote: > > Related bug: http://patchwork.ozlabs.org/patch/118863 > > > > Signed-off-by: Dmitry Monakhov > > This causes a very weird XFS failure in test 117 for me. While it > obviously is an xfs bug that you uncovered (good!) I'm a bit worried > about simply enabling operations in existing tests. Ohh. i've got what your are talking about. We can not add new ops for tests there seed is passed explicitly. And yes i've braked this, but this is because such frozen tests was written in not determined way :). Good determined test should has not just seed opt, but also explicit set of operations. All others (non determined) tests which use fsstress may benefit from new ops. So I'll redo my patch queue like this: 1) Add explicit option set for all frozen tests 2) Add new features to fssstress with non zero probability. > > Alex, Eric, Dave - should we add new tests with the new operations > Dmitry added, or is adding new ops to the existing tests fine? > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From amit.sahrawat83@gmail.com Thu Nov 3 04:59:20 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA39xKtZ257406 for ; Thu, 3 Nov 2011 04:59:20 -0500 X-ASG-Debug-ID: 1320314359-501d03580000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CFC7221C2A4 for ; Thu, 3 Nov 2011 02:59:19 -0700 (PDT) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id tAslLzgOCSTU73F5 for ; Thu, 03 Nov 2011 02:59:19 -0700 (PDT) Received: by faai28 with SMTP id i28so1465719faa.26 for ; Thu, 03 Nov 2011 02:59:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=jj7NAS5gyURGpD3/Xooa8doRRkKaUDMIFGTzN2N+Qpk=; b=rRv7A40SV0Cy+s/2IGxoHam5l0spJ/1+x+/wp3md1wGUHsqU1kKx8xj5NCnSZrNlmh leznTEeIBqNd5l4aOcVpCz9STXfN5xeE8ZIZAEcx2lYAml0OPwOBsgvlIXPQ8MSMV1m5 Jh6E031BIOJBvJMEEDFmaL0VPxB3WZlUhGmD4= MIME-Version: 1.0 Received: by 10.223.16.82 with SMTP id n18mr15331885faa.2.1320314358758; Thu, 03 Nov 2011 02:59:18 -0700 (PDT) Received: by 10.152.10.35 with HTTP; Thu, 3 Nov 2011 02:59:18 -0700 (PDT) In-Reply-To: <20111103070246.GA10579@infradead.org> References: <20111103070246.GA10579@infradead.org> Date: Thu, 3 Nov 2011 15:29:18 +0530 Message-ID: X-ASG-Orig-Subj: Re: [Patch] xfs: serialise unaligned direct IOs Subject: Re: [Patch] xfs: serialise unaligned direct IOs From: Amit Sahrawat To: Christoph Hellwig Cc: Dave Chinner , xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320314359 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79193 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Thanks Chrisoph for the suggestions. On Thu, Nov 3, 2011 at 12:32 PM, Christoph Hellwig wrot= e: > On Thu, Nov 03, 2011 at 11:37:03AM +0530, Amit Sahrawat wrote: >> This is needed for long term kernel 2.6.35.14. >> Please let me know for any changes/suggestions. > > It sounds like a fine candidate to backport, although the context > differs a lot from the actual changes commited to mainline. =A0A few > comments below: > >> they are overlapping IO and the result of concurrent overlapping IOs >> is undefined - the result of either IO is a valid result so we let >> them race. Hence we only penalise unaligned IO, which already has a >> major overhead compared to aligned IO so this isn't a major problem. >> > > You probably should keep the original Signoff and reviewed-by tags, > and add your editor note on the top into [ ] =A0brackets. Ok, will do so in the final patch. Actually was unaware of information to keep in backported patches? > >> + =A0 =A0 =A0 =A0 =A0 =A0 if (!need_i_mutex && ( unaligned_io || mapping= ->nrpages || pos > >> ip->i_size)) { > > no space after the opening brace please, and split overly-long lines > into two: Probably running checkpatch.pl will help. > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (!need_i_mutex && > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0(unaligned_io || mapping->nrpages = || pos > ip->i_size)) { > >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (need_i_mutex) { >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 /* demote the lock now the cac= hed pages are gone */ >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 xfs_ilock_demote(ip, XFS_IOLOC= K_EXCL); > >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (unaligned_io) >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 xfs_ioend_wait= (ip); >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* demote the lock now the cach= ed pages are gone if we can */ >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 else { >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 xfs_ilock_demo= te(ip, XFS_IOLOCK_EXCL); >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iolock =3D XFS= _IOLOCK_SHARED; >> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 } > > Please use the comment that was used upstream here: > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * If we are doing unalign= ed IO, wait for all other IO > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * to drain, otherwise dem= ote the lock if we had to > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 * flush cached pages. > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 */ > >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 mutex_unlock(&inode->i_mutex= ); >> >> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 iolock =3D XFS_IOLOCK_SHARED; > > >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 need_i_mutex =3D 0; > > You also need to make the i_mutex unlock and need_i_mutex update > conditional here, otherwise you still serialize all O_DIRECT writes. > you mean, keeping need_i_mutex=3D0 and mutex_unlock as part of 'else' state= ment. Thanks & Regards, Amit Sahrawat From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 05:06:54 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3A6rid258113 for ; Thu, 3 Nov 2011 05:06:53 -0500 X-ASG-Debug-ID: 1320314812-2cfa03450000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8C86E1CDD60A for ; Thu, 3 Nov 2011 03:06:52 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id EIOvYUexDh2CzGLP for ; Thu, 03 Nov 2011 03:06:52 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLuC7-0000M5-Lo; Thu, 03 Nov 2011 10:06:47 +0000 Date: Thu, 3 Nov 2011 06:06:47 -0400 From: Christoph Hellwig To: Amit Sahrawat Cc: Christoph Hellwig , Dave Chinner , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [Patch] xfs: serialise unaligned direct IOs Subject: Re: [Patch] xfs: serialise unaligned direct IOs Message-ID: <20111103100647.GA1002@infradead.org> References: <20111103070246.GA10579@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320314812 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0001 1.0000 -2.0202 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79193 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 03:29:18PM +0530, Amit Sahrawat wrote: > > You probably should keep the original Signoff and reviewed-by tags, > > and add your editor note on the top into [ ] ?brackets. > Ok, will do so in the final patch. Actually was unaware of information > to keep in backported patches? The standard procedure is to keep patches basically as-is. This doesn't quite apply for your case, so I think just adding a comment in [ ] brackets on the top is the best you can do. > > You also need to make the i_mutex unlock and need_i_mutex update > > conditional here, otherwise you still serialize all O_DIRECT writes. > > > you mean, keeping need_i_mutex=0 and mutex_unlock as part of 'else' statement. Yes. From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 05:19:10 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3AJAO7259820 for ; Thu, 3 Nov 2011 05:19:10 -0500 X-ASG-Debug-ID: 1320315549-625d00360000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A2CB321C3BA; Thu, 3 Nov 2011 03:19:09 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 83cK9V38IojuyNKp; Thu, 03 Nov 2011 03:19:09 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLuO4-0002Hh-Ky; Thu, 03 Nov 2011 10:19:08 +0000 Date: Thu, 3 Nov 2011 06:19:08 -0400 From: Christoph Hellwig To: Bill Kendall Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 0/8] xfsdump: enable support for multiple streams Subject: Re: [PATCH 0/8] xfsdump: enable support for multiple streams Message-ID: <20111103101908.GA8572@infradead.org> References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320268254-20583-1-git-send-email-wkendall@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320315549 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0001 1.0000 -2.0206 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79193 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Nov 02, 2011 at 04:10:46PM -0500, Bill Kendall wrote: > This series resurrects the IRIX multi-stream support for splitting a > backup among several output files/tapes. This offers some nice > performance improvements, particularly in xfsrestore where a single > stream often cannot keep the filesystem/disks busy. I've observed > a 1.7x improvement on a backup and a 5x improvement on restore. > > I have a couple of xfstests for this, and will submit those once > a few outstanding xfsdump test patches have been reviewed. I've put both your two previous xfsdump patches and the all your xfstests changes into the kernel.org repositories. They all look fine to me. Btw, one thing I'd love to see is a script that uses the userspace iscsi target (tgt) infrastructure to emulate a tape, and thus allow to run the tape tests in any VM without requiring a real tape device. Maybe there is someone on the list feels challenged by this? From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 05:26:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_45 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3AQDJf260478 for ; Thu, 3 Nov 2011 05:26:14 -0500 X-ASG-Debug-ID: 1320315970-2cff03d90000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A69311CDDE42 for ; Thu, 3 Nov 2011 03:26:10 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id XIBmLVhk9E9nReUY for ; Thu, 03 Nov 2011 03:26:10 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLuUr-0003zr-Ae; Thu, 03 Nov 2011 10:26:09 +0000 Date: Thu, 3 Nov 2011 06:26:09 -0400 From: Christoph Hellwig To: Arkadiusz Mi??kiewicz Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Message-ID: <20111103102609.GA12066@infradead.org> References: <201110311156.20421.arekm@maven.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201110311156.20421.arekm@maven.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320315970 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79195 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Oct 31, 2011 at 11:56:20AM +0100, Arkadiusz Mi??kiewicz wrote: > > xfs_repair version 3.1.6 > > disconnected inode 17491441754, moving to lost+found > disconnected inode 17491441755, moving to lost+found > disconnected inode 17491441756, moving to lost+found > disconnected inode 17491441757, moving to lost+found > corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. > Please capture the filesystem metadata with xfs_metadump and > report it to xfs@oss.sgi.com. > cache_node_purge: refcount was 1, not zero (node=0x21450c90) > > fatal error -- 117 - couldn't iget disconnected inode > > 30GB metadump image, 6.1GB compressed of ~7TB real partition > http://ixion.pld-linux.org/~arekm/lv_storage1.metadump.xz > > You need ~8-12GB of memory for xfs_repair on this. > > I can also provide ssh access to the system with this image and all needed > stuff, so you don't need to download it or waste own resources. I think I understand the problem - we found a disconnected inode, which we try to move to lost + found. For some reason the inode is found to be incorrect by xfs_iformat, so iget bailds out. The fix will be to do a pass over the the inodes we want to move to correct such inconsistencies and/or junk them. I'll try to prepare a fix as soon as I get some time, but I'm fairly busy at the moment. Btw, what did you to to the fs? Having the total blocks out of sync with the numbers in the data and attribute forks seems like an extremly unusal error case. From arekm@maven.pl Thu Nov 3 05:40:54 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Aesm1261199 for ; Thu, 3 Nov 2011 05:40:54 -0500 X-ASG-Debug-ID: 1320316851-1cd600190000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp-relay.maven.pl (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5557B1CDE0BF for ; Thu, 3 Nov 2011 03:40:51 -0700 (PDT) Received: from smtp-relay.maven.pl (smtp-relay.maven.pl [193.239.45.138]) by cuda.sgi.com with ESMTP id 2ekc5H7VpU2RwVFX for ; Thu, 03 Nov 2011 03:40:51 -0700 (PDT) Received: from [83.238.65.58] (port=52448 helo=t400.localnet ident=matrix157) by smtp-relay.maven.pl with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1RLuj3-00086F-Sw; Thu, 03 Nov 2011 11:40:49 +0100 From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: Christoph Hellwig X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Date: Thu, 3 Nov 2011 11:40:46 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-final-dirty; KDE/4.7.3; x86_64; ; ) Cc: xfs@oss.sgi.com References: <201110311156.20421.arekm@maven.pl> <20111103102609.GA12066@infradead.org> In-Reply-To: <20111103102609.GA12066@infradead.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201111031140.46783.arekm@maven.pl> X-Barracuda-Connect: smtp-relay.maven.pl[193.239.45.138] X-Barracuda-Start-Time: 1320316852 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79195 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thursday 03 of November 2011, Christoph Hellwig wrote: > On Mon, Oct 31, 2011 at 11:56:20AM +0100, Arkadiusz Mi??kiewicz wrote: > > xfs_repair version 3.1.6 > >=20 > > disconnected inode 17491441754, moving to lost+found > > disconnected inode 17491441755, moving to lost+found > > disconnected inode 17491441756, moving to lost+found > > disconnected inode 17491441757, moving to lost+found > > corrupt dinode 17491441757, extent total =3D 1, nblocks =3D 0. This is= a > > bug. Please capture the filesystem metadata with xfs_metadump and > > report it to xfs@oss.sgi.com. > > cache_node_purge: refcount was 1, not zero (node=3D0x21450c90) > >=20 > > fatal error -- 117 - couldn't iget disconnected inode > >=20 > > 30GB metadump image, 6.1GB compressed of ~7TB real partition > > http://ixion.pld-linux.org/~arekm/lv_storage1.metadump.xz > >=20 > > You need ~8-12GB of memory for xfs_repair on this. > >=20 > > I can also provide ssh access to the system with this image and all > > needed stuff, so you don't need to download it or waste own resources. >=20 > I think I understand the problem - we found a disconnected inode, > which we try to move to lost + found. For some reason the inode > is found to be incorrect by xfs_iformat, so iget bailds out. >=20 > The fix will be to do a pass over the the inodes we want to move > to correct such inconsistencies and/or junk them. I'll try to prepare > a fix as soon as I get some time, but I'm fairly busy at the moment. >=20 > Btw, what did you to to the fs? Having the total blocks out of sync > with the numbers in the data and attribute forks seems like an extremly > unusal error case. Well, This serwer has 16 various SATA disk connected to art-of-crap controller -= =20 Promise SuperTrak EX16350. The system exhibits funny issues with intel_idle driver=20 (https://lkml.org/lkml/2011/10/28/270). It has only 8GB of ram which xfs_repair eats for breakfast causing watchdog= to=20 reboot machine while xfs_repair was in progress (would be nice if repair co= uld=20 estimate needed ram before it is too late). All these issues combined caused few reboots in which some were in middle o= f=20 xfs_repair. Most likely all that caused such corruption. =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 05:48:52 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Amq1w261717 for ; Thu, 3 Nov 2011 05:48:52 -0500 X-ASG-Debug-ID: 1320317330-5f3b01310000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7501C21C576 for ; Thu, 3 Nov 2011 03:48:51 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id Ov2wxFFmG90No41V for ; Thu, 03 Nov 2011 03:48:51 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLuqo-0007j2-Gn; Thu, 03 Nov 2011 10:48:50 +0000 Date: Thu, 3 Nov 2011 06:48:50 -0400 From: Christoph Hellwig To: Arkadiusz Mi??kiewicz Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Message-ID: <20111103104850.GA26259@infradead.org> References: <201110311156.20421.arekm@maven.pl> <20111103102609.GA12066@infradead.org> <201111031140.46783.arekm@maven.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201111031140.46783.arekm@maven.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320317331 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79195 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 11:40:46AM +0100, Arkadiusz Mi??kiewicz wrote: > This serwer has 16 various SATA disk connected to art-of-crap controller - > Promise SuperTrak EX16350. > > The system exhibits funny issues with intel_idle driver > (https://lkml.org/lkml/2011/10/28/270). > > It has only 8GB of ram which xfs_repair eats for breakfast causing watchdog to > reboot machine while xfs_repair was in progress (would be nice if repair could > estimate needed ram before it is too late). Yes. So far most of the issues are with the internal buffercache and I suspect we could do better sizing decisions there. If you can provide some testing (xfs_repair -n should be enough) I'll happily send you some RFC patches as soon as I get time for it. In the meantime is there any chance you could send the output of xfs_repair -n -vv -m 1 for this filesystem? From tytso@mit.edu Thu Nov 3 05:54:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3AsNZp262072 for ; Thu, 3 Nov 2011 05:54:23 -0500 X-ASG-Debug-ID: 1320317662-625d01500000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from dmz-mailsec-scanner-6.mit.edu (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7675721C21B for ; Thu, 3 Nov 2011 03:54:22 -0700 (PDT) Received: from dmz-mailsec-scanner-6.mit.edu (DMZ-MAILSEC-SCANNER-6.MIT.EDU [18.7.68.35]) by cuda.sgi.com with ESMTP id BE8CFXxk6zrnAw2D for ; Thu, 03 Nov 2011 03:54:22 -0700 (PDT) X-AuditID: 12074423-b7f756d0000008d0-85-4eb272ddde25 Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) by dmz-mailsec-scanner-6.mit.edu (Symantec Messaging Gateway) with SMTP id 34.5C.02256.DD272BE4; Thu, 3 Nov 2011 06:54:21 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id pA3AsKbK004127; Thu, 3 Nov 2011 06:54:20 -0400 Received: from [10.0.42.109] (c-98-216-98-217.hsd1.ma.comcast.net [98.216.98.217]) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id pA3AsFKj029384 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Thu, 3 Nov 2011 06:54:17 -0400 (EDT) X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=windows-1252 From: Theodore Tso In-Reply-To: <20111102195534.GB22500@infradead.org> Date: Thu, 3 Nov 2011 06:54:16 -0400 Cc: Theodore Tso , Dmitry Monakhov , linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com Content-Transfer-Encoding: quoted-printable Message-Id: References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> To: Christoph Hellwig X-Mailer: Apple Mail (2.1251.1) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHKsWRmVeSWpSXmKPExsUixG6nrnu3aJOfwbEDNhaH2x6yW5yY6Wlx esIiJouVq48yWezZe5LFYlHfLUYHNo/NK7Q8dt9sYPOYdPgzk8fqC1sZPe5ef8nk8XmTXABb FJdNSmpOZllqkb5dAlfGk1Pv2QpmslRMam9la2Ccz9zFyMkhIWAisWTvbDYIW0ziwr31QDYX h5DAPkaJF+v/QTnrGSU+vPzAAuGcYpJYd+0OK0iLsICzxJ3l65lAbF4BY4k1t96xgNjMAnoS O67/AqthE1CSuPNpP1Ccg4MTqObzASGQMIuAisTjDedZQWYyCyxjlJh0YwEjRK+2xLKFr5kh ZlpJvHpyE2y+kMAiRolbp7NAbBEBTYlby9uhXpCXaPl6h20Co+AsJGfMQnLGLCRjFzAyr2KU Tcmt0s1NzMwpTk3WLU5OzMtLLdI108vNLNFLTSndxAiOBhflHYx/DiodYhTgYFTi4U0s2ugn xJpYVlyZe4hRkoNJSZTXs3CTnxBfUn5KZUZicUZ8UWlOavEhRgkOZiURXkkvoBxvSmJlVWpR PkxKmoNFSZxXZqeDn5BAemJJanZqakFqEUxWhoNDSYL3P8hQwaLU9NSKtMycEoQ0EwcnyHAe oOGcRSDDiwsSc4sz0yHypxgVpcR5z4M0C4AkMkrz4HphyeoVozjQK8K8z9OBqniAiQ6u+xXQ YCagwTMvrQcZXJKIkJJqYOzh3STnoaQbfOPszGBlpUS7bKcn383f2zW9y6/nillc7myZZnJA f/oV/zkr5yzbs+z+Bsmbm5dsnP9EtnflbN2upoDHlyy4qvQMrinv192ck6XlM3/D9lKH/3s+ PlC2cuhhEhOsv5skxbD7WsZk5qj+Keoc88OvzqgU2S7EYdZi/kChuecLtxJLcUaioRZzUXEi AKQvDdoxAwAA X-Barracuda-Connect: DMZ-MAILSEC-SCANNER-6.MIT.EDU[18.7.68.35] X-Barracuda-Start-Time: 1320317662 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79197 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Nov 2, 2011, at 3:55 PM, Christoph Hellwig wrote: > Alex, Eric, Dave - should we add new tests with the new operations > Dmitry added, or is adding new ops to the existing tests fine? One argument for adding new ops to existing tests is that it makes the = run time of the entire test suite take longer. A QA pass is already = taking quite a while, and it would be nice if we could keep xfstests as = efficient as possible in terms of the maximum testing coverage per time = spent running the test suite=85. -- Ted From arekm@maven.pl Thu Nov 3 05:57:45 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3AvjuB000407 for ; Thu, 3 Nov 2011 05:57:45 -0500 X-ASG-Debug-ID: 1320317861-3975010d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp-relay.maven.pl (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 716E5154182B for ; Thu, 3 Nov 2011 03:57:41 -0700 (PDT) Received: from smtp-relay.maven.pl (smtp-relay.maven.pl [193.239.45.138]) by cuda.sgi.com with ESMTP id xH22a61IRaZMybZD for ; Thu, 03 Nov 2011 03:57:41 -0700 (PDT) Received: from [83.238.65.58] (port=52995 helo=t400.localnet ident=matrix157) by smtp-relay.maven.pl with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1RLuzM-0008Ps-Al; Thu, 03 Nov 2011 11:57:40 +0100 From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: Christoph Hellwig X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Date: Thu, 3 Nov 2011 11:57:38 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-final-dirty; KDE/4.7.3; x86_64; ; ) Cc: xfs@oss.sgi.com References: <201110311156.20421.arekm@maven.pl> <201111031140.46783.arekm@maven.pl> <20111103104850.GA26259@infradead.org> In-Reply-To: <20111103104850.GA26259@infradead.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201111031157.39179.arekm@maven.pl> X-Barracuda-Connect: smtp-relay.maven.pl[193.239.45.138] X-Barracuda-Start-Time: 1320317862 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79196 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thursday 03 of November 2011, Christoph Hellwig wrote: > On Thu, Nov 03, 2011 at 11:40:46AM +0100, Arkadiusz Mi??kiewicz wrote: > > This serwer has 16 various SATA disk connected to art-of-crap controller > > - Promise SuperTrak EX16350. > >=20 > > The system exhibits funny issues with intel_idle driver > > (https://lkml.org/lkml/2011/10/28/270). > >=20 > > It has only 8GB of ram which xfs_repair eats for breakfast causing > > watchdog to reboot machine while xfs_repair was in progress (would be > > nice if repair could estimate needed ram before it is too late). >=20 > Yes. So far most of the issues are with the internal buffercache and I > suspect we could do better sizing decisions there. If you can provide > some testing (xfs_repair -n should be enough) I'll happily send you > some RFC patches as soon as I get time for it. I can do some testing (on image though but that shouldn't matter). >=20 > In the meantime is there any chance you could send the output of >=20 > xfs_repair -n -vv -m 1 >=20 > for this filesystem? Will such repair done on metadumped & restored image be enough for you? =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 06:03:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3B38pA000884 for ; Thu, 3 Nov 2011 06:03:08 -0500 X-ASG-Debug-ID: 1320318186-4918005f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6F12B1541861 for ; Thu, 3 Nov 2011 04:03:07 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 1x9uhMSsUzFB3f75 for ; Thu, 03 Nov 2011 04:03:07 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLv4c-0002CV-Ju; Thu, 03 Nov 2011 11:03:06 +0000 Date: Thu, 3 Nov 2011 07:03:06 -0400 From: Christoph Hellwig To: Arkadiusz Mi??kiewicz Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Message-ID: <20111103110306.GA8382@infradead.org> References: <201110311156.20421.arekm@maven.pl> <201111031140.46783.arekm@maven.pl> <20111103104850.GA26259@infradead.org> <201111031157.39179.arekm@maven.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201111031157.39179.arekm@maven.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320318187 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ3078, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79196 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ3078 Custom Rule MJ3078 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 11:57:38AM +0100, Arkadiusz Mi??kiewicz wrote: > > In the meantime is there any chance you could send the output of > > > > xfs_repair -n -vv -m 1 > > > > for this filesystem? > > Will such repair done on metadumped & restored image be enough for you? Yes. I also looked at the memory heuristics and found some fairly obvious flaws. I'll have some test patches for you ASAP. From rjevskiy@gmail.com Thu Nov 3 06:04:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3B4NS4001014 for ; Thu, 3 Nov 2011 06:04:23 -0500 X-ASG-Debug-ID: 1320318262-6a2f011c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DFA1421C769 for ; Thu, 3 Nov 2011 04:04:22 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id LrUVFaMjdpTmTes4 for ; Thu, 03 Nov 2011 04:04:22 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1192735bkb.26 for ; Thu, 03 Nov 2011 04:04:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type:content-transfer-encoding; bh=bNHOksAqZbqwzWp6bM196NcZfTdqBLvR/eKNxDh4OSM=; b=x+dE80T9BhHeDEDkE1JIWd7hBhQosILLjLM86RjtHtmNPG14IHYJ9USENzBkhiYe6w pny1sHS9iImffcV3ri/I2l0JK2RJmEGhJUB/cy0JFxCz+4JyNxy/A/3CZTfkeNUYGkiV 6pgF9XeTz7ZVz+74yT7ScWgB0CFBEmbgpuRoY= Received: by 10.204.157.142 with SMTP id b14mr7644184bkx.44.1320318261738; Thu, 03 Nov 2011 04:04:21 -0700 (PDT) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5239337bkd.2.2011.11.03.04.04.19 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 04:04:20 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Theodore Tso , Christoph Hellwig Cc: Theodore Tso , linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress In-Reply-To: References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Thu, 03 Nov 2011 15:04:17 +0400 Message-ID: <87wrbhxya6.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320318262 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79197 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, 3 Nov 2011 06:54:16 -0400, Theodore Tso wrote: >=20 > On Nov 2, 2011, at 3:55 PM, Christoph Hellwig wrote: >=20 > > Alex, Eric, Dave - should we add new tests with the new operations > > Dmitry added, or is adding new ops to the existing tests fine? >=20 > One argument for adding new ops to existing tests is that it makes the ru= n time of the entire test suite take longer. A QA pass is already taking = quite a while, and it would be nice if we could keep xfstests as efficient = as possible in terms of the maximum testing coverage per time spent running= the test suite=E2=80=A6. Yes, but regression test with explicit seed option should be preserved. Number of such test is not too big, so it is reasonable to hardcode set of operations in such tests and let all others use new feature= s. >=20 > -- Ted >=20 From amit.sahrawat83@gmail.com Thu Nov 3 06:19:59 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,FREEMAIL_FROM, J_CHICKENPOX_72,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3BJxqi002182 for ; Thu, 3 Nov 2011 06:19:59 -0500 X-ASG-Debug-ID: 1320319193-1cc001530000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D25351CDDC9B for ; Thu, 3 Nov 2011 04:19:53 -0700 (PDT) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id C65DZ4AXMSbjyKh3 for ; Thu, 03 Nov 2011 04:19:53 -0700 (PDT) Received: by faai28 with SMTP id i28so1534881faa.26 for ; Thu, 03 Nov 2011 04:19:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=4B9tdeovyeaB1XHk5U/wA1gqWhVAm0M2ieUDAAF4eiw=; b=A7JkTZmQ0eFoamBMoKjlBhU9tK3z+wUWnphefOmQ6mtq3oHnaoa5FsZgGepRHdmRzj 9dKzA7b3j8k85XQkBbSrzkjIvG3V5drOLXbz15b3B0XlWdELo98LgsZBh+r7nn8A6eZW V7E8+HA6kHON1A7Lpm/8DAGyGz9WSKqMeFnv8= MIME-Version: 1.0 Received: by 10.223.39.20 with SMTP id d20mr15524821fae.37.1320319193323; Thu, 03 Nov 2011 04:19:53 -0700 (PDT) Received: by 10.152.10.35 with HTTP; Thu, 3 Nov 2011 04:19:53 -0700 (PDT) In-Reply-To: <20111103100647.GA1002@infradead.org> References: <20111103070246.GA10579@infradead.org> <20111103100647.GA1002@infradead.org> Date: Thu, 3 Nov 2011 16:49:53 +0530 Message-ID: X-ASG-Orig-Subj: Re: [Patch] xfs: serialise unaligned direct IOs Subject: Re: [Patch] xfs: serialise unaligned direct IOs From: Amit Sahrawat To: Christoph Hellwig Cc: Dave Chinner , xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320319194 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79199 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean After all modifications and running checkpatch.pl. Thanks & Regards, Amit Sahrawat xfs: serialise unaligned direct IOs [ This patch published in 2.6.38 kernel(Original reference http://oss.sgi.com/archives/xfs/2011-01/msg00013.html), but can not be applied to 2.6.35 kernel directly, because of the absence of required function, its reimplmented to resolve xfstest test 240 fail.] When two concurrent unaligned, non-overlapping direct IOs are issued to the same block, the direct Io layer will race to zero the block. The result is that one of the concurrent IOs will overwrite data written by the other IO with zeros. This is demonstrated by the xfsqa test 240. To avoid this problem, serialise all unaligned direct IOs to an inode with a big hammer. We need a big hammer approach as we need to serialise AIO as well, so we can't just block writes on locks. Hence, the big hammer is calling xfs_ioend_wait() while holding out other unaligned direct IOs from starting. We don't bother trying to serialised aligned vs unaligned IOs as they are overlapping IO and the result of concurrent overlapping IOs is undefined - the result of either IO is a valid result so we let them race. Hence we only penalise unaligned IO, which already has a major overhead compared to aligned IO so this isn't a major problem. Signed-off-by: Dave Chinner Signed-off-by: Amit Sahrawat Signed-off-by: Ajeet Yadav diff -Nurp linux-Orig/fs/xfs/linux-2.6/xfs_file.c linux-Updated/fs/xfs/linux-2.6/xfs_file.c --- linux-Orig/fs/xfs/linux-2.6/xfs_file.c 2011-11-02 12:10:52.000000000 +0= 530 +++ linux-Updated/fs/xfs/linux-2.6/xfs_file.c 2011-11-03 16:39:08.000000000 +0530 @@ -587,6 +587,7 @@ xfs_file_aio_write( xfs_fsize_t isize, new_size; int iolock; size_t ocount =3D 0, count; + int unaligned_io =3D 0; int need_i_mutex; XFS_STATS_INC(xs_write_calls); @@ -640,8 +641,26 @@ start: xfs_iunlock(ip, XFS_ILOCK_EXCL|iolock); return XFS_ERROR(-EINVAL); } + /* + * In most cases the direct IO writes will be done with IOLOCK_SHARED + * allowing them to be done in parallel with reads and other direct IO + * writes. However,if the IO is not aligned to filesystem blocks, the + * direct IO layer needs to do sub-block zeroing and that requires + * serialisation against other direct IOs to the same block. In this + * case we need to serialise the submission of the unaligned IOs so + * that we don't get racing block zeroing in the dio layer. + * To avoid the problem with aio, we also need to wait for outstanding + * IOs to complete so that unwritten extent conversion is completed + * before we try to map the overlapping block. This is currently + * implemented by hitting it with a big hammer (i.e. xfs_ioend_wait()). + */ + + if ((pos & mp->m_blockmask) || + ((pos + count) & mp->m_blockmask)) + unaligned_io =3D 1; - if (!need_i_mutex && (mapping->nrpages || pos > ip->i_size)) { + if (!need_i_mutex && + (unaligned_io || mapping->nrpages || pos > ip->i_size)) { xfs_iunlock(ip, XFS_ILOCK_EXCL|iolock); iolock =3D XFS_IOLOCK_EXCL; need_i_mutex =3D 1; @@ -700,12 +719,18 @@ start: } if (need_i_mutex) { - /* demote the lock now the cached pages are gone */ - xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); - mutex_unlock(&inode->i_mutex); - - iolock =3D XFS_IOLOCK_SHARED; - need_i_mutex =3D 0; + if (unaligned_io) + xfs_ioend_wait(ip); + else { + /* + * demote the lock now the cached pages + * are gone if we can + */ + xfs_ilock_demote(ip, XFS_IOLOCK_EXCL); + iolock =3D XFS_IOLOCK_SHARED; + mutex_unlock(&inode->i_mutex); + need_i_mutex =3D 0; + } } trace_xfs_file_direct_write(ip, count, iocb->ki_pos, ioflags); On Thu, Nov 3, 2011 at 3:36 PM, Christoph Hellwig wrote= : > On Thu, Nov 03, 2011 at 03:29:18PM +0530, Amit Sahrawat wrote: >> > You probably should keep the original Signoff and reviewed-by tags, >> > and add your editor note on the top into [ ] ?brackets. >> Ok, will do so in the final patch. Actually was unaware of information >> to keep in backported patches? > > The standard procedure is to keep patches basically as-is. =A0This doesn'= t > quite apply for your case, so I think just adding a comment in > [ ] brackets on the top is the best you can do. > >> > You also need to make the i_mutex unlock and need_i_mutex update >> > conditional here, otherwise you still serialize all O_DIRECT writes. >> > >> you mean, keeping need_i_mutex=3D0 and mutex_unlock as part of 'else' st= atement. > > Yes. > > From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 06:54:33 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_28, J_CHICKENPOX_43 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3BsW9A005568 for ; Thu, 3 Nov 2011 06:54:33 -0500 X-ASG-Debug-ID: 1320321270-6cab00730000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4EED816CD9F2 for ; Thu, 3 Nov 2011 04:54:30 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id KEWrsN0AVU4T9hil for ; Thu, 03 Nov 2011 04:54:30 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLvsL-00045l-Ud; Thu, 03 Nov 2011 11:54:29 +0000 Date: Thu, 3 Nov 2011 07:54:29 -0400 From: Christoph Hellwig To: Arkadiusz Mi??kiewicz Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Message-ID: <20111103115429.GA15726@infradead.org> References: <201110311156.20421.arekm@maven.pl> <201111031140.46783.arekm@maven.pl> <20111103104850.GA26259@infradead.org> <201111031157.39179.arekm@maven.pl> <20111103110306.GA8382@infradead.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="zYM0uCDKw75PZbzx" Content-Disposition: inline In-Reply-To: <20111103110306.GA8382@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320321271 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0063 1.0000 -1.9798 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.88 X-Barracuda-Spam-Status: No, SCORE=-1.88 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79200 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --zYM0uCDKw75PZbzx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Can you give this patch a quick try on the image in that constrained setup? --zYM0uCDKw75PZbzx Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=libxfs-estimate-free-memory-better Index: xfsprogs-dev/libxfs/linux.c =================================================================== --- xfsprogs-dev.orig/libxfs/linux.c 2011-11-03 12:01:14.213689743 +0100 +++ xfsprogs-dev/libxfs/linux.c 2011-11-03 12:46:56.928191650 +0100 @@ -207,8 +207,13 @@ platform_nproc(void) return sysconf(_SC_NPROCESSORS_ONLN); } +/* + * Return the memory that we can use freely. + * + * The return value is in kilobytes. + */ unsigned long -platform_physmem(void) +platform_freemem(void) { struct sysinfo si; @@ -217,5 +222,14 @@ platform_physmem(void) progname); exit(1); } - return (si.totalram >> 10) * si.mem_unit; /* kilobytes */ + + /* + * Assume we can use memory that is marked free. This is a very + * conservative approximation given that there might be a lot of + * pagecache that is easily reclaimable, but the only way to figure + * out pagecache size is by parsing /proc/meminfo, and the format + * of that file keeps changing. This approach is still better than + * guessing based on si.totalram which might be highly overestimated. + */ + return (si.freeram >> 10) * si.mem_unit; } Index: xfsprogs-dev/include/libxfs.h =================================================================== --- xfsprogs-dev.orig/include/libxfs.h 2011-11-03 12:29:02.001691672 +0100 +++ xfsprogs-dev/include/libxfs.h 2011-11-03 12:31:49.780691838 +0100 @@ -475,7 +475,7 @@ enum ce { CE_DEBUG, CE_CONT, CE_NOTE, CE #define LIBXFS_BBTOOFF64(bbs) (((xfs_off_t)(bbs)) << BBSHIFT) extern int libxfs_nproc(void); -extern unsigned long libxfs_physmem(void); /* in kilobytes */ +extern unsigned long libxfs_freemem(void); /* in kilobytes */ #include #include Index: xfsprogs-dev/libxfs/darwin.c =================================================================== --- xfsprogs-dev.orig/libxfs/darwin.c 2011-11-03 12:29:02.017691106 +0100 +++ xfsprogs-dev/libxfs/darwin.c 2011-11-03 12:40:10.720691752 +0100 @@ -128,18 +128,27 @@ platform_nproc(void) return ncpu; } +/* + * Return the memory that we can use freely. + * + * The return value is in kilobytes. + */ unsigned long -platform_physmem(void) +platform_freemem(void) { - unsigned long physmem; - size_t len = sizeof(physmem); + unsigned long freemem; + size_t len = sizeof(freemem); static int mib[2] = {CTL_HW, HW_PHYSMEM}; - if (sysctl(mib, 2, &physmem, &len, NULL, 0) < 0) { + if (sysctl(mib, 2, &freemem, &len, NULL, 0) < 0) { fprintf(stderr, _("%s: can't determine memory size\n"), progname); exit(1); } - return physmem >> 10; + + /* + * Assume we can use approximately 3/4 of the physical memory. + */ + return (freemem >> (10 + 2)) * 3; } Index: xfsprogs-dev/libxfs/freebsd.c =================================================================== --- xfsprogs-dev.orig/libxfs/freebsd.c 2011-11-03 12:29:02.037693919 +0100 +++ xfsprogs-dev/libxfs/freebsd.c 2011-11-03 12:40:19.000190942 +0100 @@ -187,17 +187,26 @@ platform_nproc(void) return ncpu; } +/* + * Return the memory that we can use freely. + * + * The return value is in kilobytes. + */ unsigned long -platform_physmem(void) +platform_freemem(void) { - unsigned long physmem; - size_t len = sizeof(physmem); + unsigned long freemem; + size_t len = sizeof(freemem); static int mib[2] = {CTL_HW, HW_PHYSMEM}; - if (sysctl(mib, 2, &physmem, &len, NULL, 0) < 0) { + if (sysctl(mib, 2, &freemem, &len, NULL, 0) < 0) { fprintf(stderr, _("%s: can't determine memory size\n"), progname); exit(1); } - return physmem >> 10; + + /* + * Assume we can use approximately 3/4 of the physical memory. + */ + return (freemem >> (10 + 2)) * 3; } Index: xfsprogs-dev/libxfs/init.c =================================================================== --- xfsprogs-dev.orig/libxfs/init.c 2011-11-03 12:29:02.057690208 +0100 +++ xfsprogs-dev/libxfs/init.c 2011-11-03 12:29:50.044192568 +0100 @@ -862,7 +862,7 @@ libxfs_nproc(void) } unsigned long -libxfs_physmem(void) +libxfs_freemem(void) { - return platform_physmem(); + return platform_freemem(); } Index: xfsprogs-dev/libxfs/init.h =================================================================== --- xfsprogs-dev.orig/libxfs/init.h 2011-11-03 12:29:02.077690698 +0100 +++ xfsprogs-dev/libxfs/init.h 2011-11-03 12:29:52.296691230 +0100 @@ -32,7 +32,7 @@ extern char *platform_findblockpath (cha extern int platform_direct_blockdev (void); extern int platform_align_blockdev (void); extern int platform_nproc(void); -extern unsigned long platform_physmem(void); /* in kilobytes */ +extern unsigned long platform_freemem(void); /* in kilobytes */ extern int platform_has_uuid; #endif /* LIBXFS_INIT_H */ Index: xfsprogs-dev/libxfs/irix.c =================================================================== --- xfsprogs-dev.orig/libxfs/irix.c 2011-11-03 12:29:02.097691363 +0100 +++ xfsprogs-dev/libxfs/irix.c 2011-11-03 12:42:59.724691783 +0100 @@ -97,8 +97,13 @@ platform_nproc(void) return sysmp(MP_NPROCS); } +/* + * Return the memory that we can use freely. + * + * The return value is in kilobytes. + */ unsigned long -platform_physmem(void) +platform_freemem(void) { struct rminfo ri; @@ -107,5 +112,9 @@ platform_physmem(void) progname); exit(1); } - return (ri.physmem >> 10) * getpagesize(); /* kilobytes */ -} \ No newline at end of file + + /* + * Assume we can use all free memory. + */ + return (ri.freemem >> 10) * getpagesize(); /* kilobytes */ +} Index: xfsprogs-dev/repair/xfs_repair.c =================================================================== --- xfsprogs-dev.orig/repair/xfs_repair.c 2011-11-03 12:29:02.117690613 +0100 +++ xfsprogs-dev/repair/xfs_repair.c 2011-11-03 12:32:42.348190446 +0100 @@ -631,8 +631,11 @@ main(int argc, char **argv) mem_used = (mp->m_sb.sb_icount >> (10 - 2)) + (mp->m_sb.sb_dblocks >> (10 + 1)) + 50000; /* rough estimate of 50MB overhead */ - max_mem = max_mem_specified ? max_mem_specified * 1024 : - libxfs_physmem() * 3 / 4; + + if (max_mem_specified) + max_mem = max_mem_specified * 1024; + else + max_mem = libxfs_freemem(); if (getrlimit(RLIMIT_AS, &rlim) != -1 && rlim.rlim_cur != RLIM_INFINITY) { --zYM0uCDKw75PZbzx-- From wkendall@sgi.com Thu Nov 3 07:11:46 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3CBjTL007414 for ; Thu, 3 Nov 2011 07:11:46 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id B0F82304114; Thu, 3 Nov 2011 05:11:42 -0700 (PDT) Received: from [128.162.233.117] (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id A125D70006F6; Thu, 3 Nov 2011 07:11:42 -0500 (CDT) Message-ID: <4EB284FE.2040409@sgi.com> Date: Thu, 03 Nov 2011 07:11:42 -0500 From: Bill Kendall User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110921 Thunderbird/3.1.15 MIME-Version: 1.0 To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 0/8] xfsdump: enable support for multiple streams References: <1320268254-20583-1-git-send-email-wkendall@sgi.com> <20111103101908.GA8572@infradead.org> In-Reply-To: <20111103101908.GA8572@infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 11/03/2011 05:19 AM, Christoph Hellwig wrote: > On Wed, Nov 02, 2011 at 04:10:46PM -0500, Bill Kendall wrote: >> This series resurrects the IRIX multi-stream support for splitting a >> backup among several output files/tapes. This offers some nice >> performance improvements, particularly in xfsrestore where a single >> stream often cannot keep the filesystem/disks busy. I've observed >> a 1.7x improvement on a backup and a 5x improvement on restore. >> >> I have a couple of xfstests for this, and will submit those once >> a few outstanding xfsdump test patches have been reviewed. > > I've put both your two previous xfsdump patches and the all your > xfstests changes into the kernel.org repositories. They all look > fine to me. Thanks for the reviews. I'm going to be out of town for a few days but will look at your suggestion to use POSIX semaphores when I return. > > Btw, one thing I'd love to see is a script that uses the userspace > iscsi target (tgt) infrastructure to emulate a tape, and thus allow to > run the tape tests in any VM without requiring a real tape device. > > Maybe there is someone on the list feels challenged by this? I've been using an open source vtl to achieve the same goal, and have given some thought to integrating (or at a minimum documenting) how to use it with the xfsdump tests. https://github.com/markh794/mhvtl Bill From rjevskiy@gmail.com Thu Nov 3 07:14:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3CEEM8007624 for ; Thu, 3 Nov 2011 07:14:14 -0500 X-ASG-Debug-ID: 1320322452-7c4f000a0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 759481CDE7E3 for ; Thu, 3 Nov 2011 05:14:12 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id ALQiYtx87AJ0grdl for ; Thu, 03 Nov 2011 05:14:12 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1257539bkb.26 for ; Thu, 03 Nov 2011 05:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=xq2Yg448mAbPizMIRYN/PFBB4aUdQ5NeOoKJ8XQM8iM=; b=W/LkGnKALevoCkv6YbORKc5HCtujukYSp/AruoRsNWm94ZXLjHgNHUuLXWFpzDMc58 Yj49nzPRFfXrKXrAFALxwMdylKapnf0hIw3nY84uQjXu59KnAhNyXdgFVLt60SvCJY7E u+qqo4RYIGOwUWx+m7fY9757vl5gR70iAQFO4= Received: by 10.204.148.68 with SMTP id o4mr8074768bkv.21.1320322452041; Thu, 03 Nov 2011 05:14:12 -0700 (PDT) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id a27sm5403649bku.9.2011.11.03.05.14.06 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 05:14:07 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Christoph Hellwig Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress In-Reply-To: <87zkgdy2fi.fsf@dmbot.sw.ru> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> <87zkgdy2fi.fsf@dmbot.sw.ru> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Thu, 03 Nov 2011 16:14:04 +0400 Message-ID: <87ty6lxv1v.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320322453 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79201 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, 03 Nov 2011 13:34:41 +0400, Dmitry Monakhov wrote: > On Wed, 2 Nov 2011 15:55:34 -0400, Christoph Hellwig wrote: > > On Sat, Oct 29, 2011 at 04:48:15AM +0400, Dmitry Monakhov wrote: > > > Related bug: http://patchwork.ozlabs.org/patch/118863 > > > > > > Signed-off-by: Dmitry Monakhov > > > > This causes a very weird XFS failure in test 117 for me. While it > > obviously is an xfs bug that you uncovered (good!) I'm a bit worried > > about simply enabling operations in existing tests. > Ohh. i've got what your are talking about. We can not add new ops > for tests there seed is passed explicitly. And yes i've braked this, but > this is because such frozen tests was written in not determined > way :). Good determined test should has not just seed opt, but also > explicit set of operations. All others (non determined) tests which use > fsstress may benefit from new ops. So I'll redo my patch queue like this: > 1) Add explicit option set for all frozen tests i've stuck immediately on first test ;(. on my kernel 107'th test always failed (commit: 73a57c642cdfa660 "1.1.0 release"), is it expected? Which xfs-tree and commit should i use? > 2) Add new features to fssstress with non zero probability. > > > > Alex, Eric, Dave - should we add new tests with the new operations > > Dmitry added, or is adding new ops to the existing tests fine? > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From wkendall@sgi.com Thu Nov 3 07:14:33 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_42, J_CHICKENPOX_64,T_LOTS_OF_MONEY autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3CEXE3007653 for ; Thu, 3 Nov 2011 07:14:33 -0500 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 33F3B3040DC; Thu, 3 Nov 2011 05:14:33 -0700 (PDT) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 1D3B170006F6; Thu, 3 Nov 2011 07:14:33 -0500 (CDT) Received: by augusta (Postfix, from userid 2022) id D171E21E23D1; Thu, 3 Nov 2011 07:14:32 -0500 (CDT) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH] xfstests: add 059 and 060 for multi-stream xfsdump Date: Thu, 3 Nov 2011 07:14:24 -0500 Message-Id: <1320322464-6559-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Implement the multi-stream placeholder tests 059 and 060. The comments imply that these tests existed on IRIX, but I was unable to find them. Test 059 does a simple 4-way dump and restore. Test 060 does a 4-way dump, then restores each dump file individually (in a cumulative fashion). The tests are skipped if the installed xfsdump does not support multi-stream output. Signed-off-by: Bill Kendall --- 059 | 23 ++++++++++++++--------- 060 | 29 ++++++++++++++++++++--------- common.dump | 40 ++++++++++++++++++++++++++++++++++++++-- group | 4 ++-- 4 files changed, 74 insertions(+), 22 deletions(-) diff --git a/059 b/059 index 6348f0e..1f3f8e3 100755 --- a/059 +++ b/059 @@ -1,10 +1,10 @@ #! /bin/bash # FS QA Test No. 059 # -# place holder for IRIX 059 test for xfsdump/xfsrestore multi streams +# Test multi-stream xfsdump/xfsrestore. # #----------------------------------------------------------------------- -# Copyright (c) 2000-2001 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2000-2001, 2011 SGI. All Rights Reserved. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License as @@ -22,26 +22,31 @@ #----------------------------------------------------------------------- # # creator -owner=ivanr@sgi.com +owner=wkendall@sgi.com seq=`basename $0` echo "QA output created by $seq" here=`pwd` tmp=/tmp/$$ -status=1 # failure is the default! +status=0 # success is the default! trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc -. ./common.filter - -_notrun "Place holder for IRIX test 059" +. ./common.dump # real QA test starts here _supported_fs xfs -_supported_os IRIX +_supported_os Linux + +_require_multi_stream + +_create_dumpdir_fill_multi +_do_dump_multi_file --multi 4 +_do_restore_multi_file --multi 4 +_ls_compare_sub +_diff_compare # success, all done -status=0 exit diff --git a/060 b/060 index 6e58e5b..4adc00f 100755 --- a/060 +++ b/060 @@ -1,10 +1,10 @@ #! /bin/bash # FS QA Test No. 060 # -# place holder for IRIX 060 test for xfsdump/xfsrestore multi streams +# Test multi-stream xfsdump and restoring one stream at a time. # #----------------------------------------------------------------------- -# Copyright (c) 2000-2001 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2000-2001, 2011 SGI. All Rights Reserved. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License as @@ -22,26 +22,37 @@ #----------------------------------------------------------------------- # # creator -owner=ivanr@sgi.com +owner=wkendall@sgi.com seq=`basename $0` echo "QA output created by $seq" here=`pwd` tmp=/tmp/$$ -status=1 # failure is the default! +status=0 # success is the default! trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc -. ./common.filter - -_notrun "Place holder for IRIX test 060" +. ./common.dump # real QA test starts here _supported_fs xfs -_supported_os IRIX +_supported_os Linux + +_require_multi_stream + +_create_dumpdir_fill_multi +_do_dump_multi_file --multi 4 +_prepare_restore_dir +base=$dump_file +opts="" +for x in 0 1 2 3; do + _do_restore_file_cum -f $base.$x $opts + opts=-R +done +_ls_compare_sub +_diff_compare # success, all done -status=0 exit diff --git a/common.dump b/common.dump index 076517d..31a75c5 100644 --- a/common.dump +++ b/common.dump @@ -75,6 +75,14 @@ done have_mtvariable=false [ `uname` = "Linux" ] && have_mtvariable=true + +_require_multi_stream() +{ + $XFSDUMP_PROG -JF -f /dev/null -f /dev/null 2> /dev/null | + grep -q "too many -f arguments" && + _notrun "xfsdump multi-stream support required" +} + # # do a remote/local mt # @@ -536,6 +544,18 @@ _do_create_dumpdir_fill() cd $here } +_mk_fillconfig_multi() +{ + _mk_fillconfig1 + cat <>$tmp.config +# pathname size in bytes +# +large000 8874368 $nobody $nobody +large111 2582912 $nobody $nobody +large222 7825792 $nobody $nobody +End-of-File +} + _create_dumpdir_largefile() { _wipe_fs @@ -580,6 +600,17 @@ _create_dumpdir_fill_ea() _stable_fs } +# +# Create enough files, and a few large enough files, so that +# some files are likely to be split across streams. +# +_create_dumpdir_fill_multi() +{ + _wipe_fs + _mk_fillconfig_multi + _do_create_dumpdir_fill + _stable_fs +} # # Append a subset of the fill'ed files @@ -802,9 +833,11 @@ _dump_filter_main() -e 's/time:[ ].*/time: TIME/' \ -e 's/date:[ ].*/date: DATE/' \ -e 's/dump begun .*/dump begun DATE/' \ + -e 's/previously begun .*/previously begun DATE/' \ -e 's/[0-9][0-9]* seconds/SECS seconds/' \ -e 's/restore.[0-9][0-9]*/restore.PID/' \ - -e 's/ino [0-9][0-9]*/ino INO/' \ + -e 's/ino [0-9][0-9]*/ino INO/g' \ + -e '/stream [0-9]:/s/offset [0-9][0-9]*/offset NUM/g' \ -e '/: dump size/s/[0-9][0-9]*/NUM/' \ -e '/dump size:/s/[0-9][0-9]*/NUM/' \ -e '/dump size per stream:/s/[0-9][0-9]*/NUM/' \ @@ -862,7 +895,7 @@ _invutil_filter() _dir_filter() { sed \ - -e "s#$dump_file#DUMP_FILE#" \ + -e "s#$dump_file#DUMP_FILE#g" \ -e "s#$SCRATCH_DEV#SCRATCH_DEV#" \ -e "s#$SCRATCH_RAWDEV#SCRATCH_DEV#" \ -e "s#$dumptape#TAPE_DEV#" \ @@ -959,6 +992,9 @@ _parse_restore_args() --no-check-quota) do_quota_check=false ;; + -R) + restore_args="$restore_args $1" + ;; *) _fail "invalid argument to common.dump function: $1" ;; diff --git a/group b/group index 8a23320..efeb977 100644 --- a/group +++ b/group @@ -169,8 +169,8 @@ deprecated 056 dump ioctl auto quick 057 acl auto 058 acl auto -059 dump ioctl -060 dump ioctl +059 dump ioctl auto quick +060 dump ioctl auto quick 061 dump ioctl auto quick 062 attr udf auto quick 063 dump attr auto quick -- 1.7.0.4 From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 07:28:26 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3CSPlF009013 for ; Thu, 3 Nov 2011 07:28:26 -0500 X-ASG-Debug-ID: 1320323304-62bb039f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 40D9921C900; Thu, 3 Nov 2011 05:28:24 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id g1UnOhLgINvZEk9m; Thu, 03 Nov 2011 05:28:24 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RLwP8-0002A5-B1; Thu, 03 Nov 2011 12:28:22 +0000 Date: Thu, 3 Nov 2011 08:28:22 -0400 From: Christoph Hellwig To: Dmitry Monakhov Cc: Christoph Hellwig , linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Message-ID: <20111103122822.GA8282@infradead.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> <87zkgdy2fi.fsf@dmbot.sw.ru> <87ty6lxv1v.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87ty6lxv1v.fsf@dmbot.sw.ru> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320323305 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0207 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79203 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 04:14:04PM +0400, Dmitry Monakhov wrote: > i've stuck immediately on first test ;(. > on my kernel 107'th test always failed (commit: 73a57c642cdfa660 > "1.1.0 release"), is it expected? Which xfs-tree and commit should i use? 106 and 107 have been failing basically forever - that's why they aren't part of the auto group. From dmonlist@gmail.com Thu Nov 3 09:25:25 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPPfi017874 for ; Thu, 3 Nov 2011 09:25:25 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 64CDD12FF47D for ; Thu, 3 Nov 2011 07:25:22 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id 11wVMVG7qZCYqpQd for ; Thu, 03 Nov 2011 07:25:22 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer; bh=morOJP/OCMqe0Q5t3RR1fM693S1suJ4b5Oce1G1YJ2Q=; b=ggBkeDnlN7V0BuEzGK27JZjwg9Utsh9cvfC0r5QBC17DevWYj7syZ/r7ID5kyg1esT iEFj7W+IaQ1OS/pZ/4fx0IiYODyiXmFecluD6nG7WRbbDH8faBIhsbvHSvbJmPBYit/v NxHQbcgXDHrEpSW3UeudA+XxIx6EHTmkS/1GY= Received: by 10.204.135.195 with SMTP id o3mr8416780bkt.80.1320330316541; Thu, 03 Nov 2011 07:25:16 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.07 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:08 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 00/12] Bunch of new stress tests -v4 Subject: [PATCH 00/12] Bunch of new stress tests -v4 Date: Thu, 3 Nov 2011 18:24:49 +0400 Message-Id: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330324 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Changes from v3 ( in request to Christoph's comments) - hard code fsstress's options for explicit regression tests - add two new regression tests Changes from v2 ( in request to Christoph's comments) - rearange patch sequance - codestyle cleanups - give better description - add tunable mask for FS_IOC_SETFLAGS test. LOG: # Following patches improves fsstress logging, # First three patches was already pulled by Chris to it's internal tree, # but not push it to public tree yet xfstests: fsstress dump inode info when possible xfstests: add different logging option to fsstress xfstests: fsstress should kill children tasks before exit # First patch add dump configuration in command line style, second use it's output xfstests: fsstress add command line style output for show_opts xfstests: freeze fsstress options for 117'th # Following patches add new operations to fsstress. xfstests: add fallocate support to fsstress xfstests: fsstress add FS_IOC_{SET,GET}FLAGS operations v2 xfstests: add fiemap operation to fsstress # Following testcase may fail on recent kernel in case of ext4 # Some fixes are already available here: # http://www.spinics.net/lists/linux-ext4/msg27857.html # http://www.spinics.net/lists/linux-ext4/msg28552.html xfstests: add a new test that runs fsstress under ENOSPC conditions xfstress: add regression testcase for d583fb87a3ff0 # Fixes available here: # http://patchwork.ozlabs.org/patch/120581/ # http://patchwork.ozlabs.org/patch/120582/ xfstests: add a new quota test that runs fsstress under ENOSPC conditions # Following testcase may fail on recent kernel in case of ext3/4 # In case of ext4 kernel will probably panic. xfstress: Test data journaling flag switch for a single file From dmonlist@gmail.com Thu Nov 3 09:25:27 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_42,J_CHICKENPOX_43,J_CHICKENPOX_52,J_CHICKENPOX_83, LOCAL_GNU_PATCH,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPQFY017879 for ; Thu, 3 Nov 2011 09:25:26 -0500 X-ASG-Debug-ID: 1320330324-4930018f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 78D5D5683B9 for ; Thu, 3 Nov 2011 07:25:24 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id DrPatcvzylL3Fp4E for ; Thu, 03 Nov 2011 07:25:24 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=7v/Cd85iZPSC9BWm9AcNdwWBvtZpC8uT74a4YRvs1F4=; b=LPXyH8Ik376FVsKK1Ci4t11Z6TWS7fkDXEA9+hKIqYT8tAHyHa7JuzyxCY8Qd3jK8J wmyhtbzXHjaBG0uJMaR/U1SMKGQ/wDosEPLxQIV/QRLhmfkk5LXmZ8RD2y0gWYzWnkRE UsJ3XJ/4xW3bWUAI/dBwijZ47DHIM0AYlIXHc= Received: by 10.204.10.67 with SMTP id o3mr8588980bko.3.1320330324243; Thu, 03 Nov 2011 07:25:24 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.22 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:23 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 01/12] xfstests: fsstress dump inode info when possible Subject: [PATCH 01/12] xfstests: fsstress dump inode info when possible Date: Thu, 3 Nov 2011 18:24:50 +0400 Message-Id: <1320330301-2682-2-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330325 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Fsstress exec behaviour is not completely determinated in case of low resources mode due to ENOMEM, ENOSPC, etc. In some places we call stat(2). This information may be halpfull for future investigations purposes. Let's dump stat info where possible. Reviewed-by: Christoph Hellwig Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 83 +++++++++++++++++++++++++++++++++++++------------------- 1 files changed, 55 insertions(+), 28 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index c37cddf..51ecda2 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -1391,6 +1391,14 @@ zero_freq(void) p->freq = 0; } +void inode_info(char *str, size_t sz, struct stat64 *s, int verbose) +{ + if (verbose) + snprintf(str, sz, "[%ld %ld %d %d %lld %lld]", (long)s->st_ino, + (long)s->st_nlink, s->st_uid, s->st_gid, + (long long) s->st_blocks, (long long) s->st_size); +} + void allocsp_f(int opno, long r) { @@ -1402,6 +1410,7 @@ allocsp_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -1428,6 +1437,7 @@ allocsp_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; @@ -1435,9 +1445,10 @@ allocsp_f(int opno, long r) fl.l_start = off; fl.l_len = 0; e = xfsctl(f.path, fd, XFS_IOC_ALLOCSP64, &fl) < 0 ? errno : 0; - if (v) - printf("%d/%d: xfsctl(XFS_IOC_ALLOCSP64) %s %lld 0 %d\n", - procid, opno, f.path, (long long)off, e); + if (v) { + printf("%d/%d: xfsctl(XFS_IOC_ALLOCSP64) %s%s %lld 0 %d\n", + procid, opno, f.path, st, (long long)off, e); + } free_pathname(&f); close(fd); } @@ -1779,6 +1790,7 @@ dread_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -1800,15 +1812,16 @@ dread_f(int opno, long r) if (fstat64(fd, &stb) < 0) { if (v) printf("%d/%d: dread - fstat64 %s failed %d\n", - procid, opno, f.path, errno); + procid, opno, f.path, errno); free_pathname(&f); close(fd); return; } + inode_info(st, sizeof(st), &stb, v); if (stb.st_size == 0) { if (v) - printf("%d/%d: dread - %s zero size\n", procid, opno, - f.path); + printf("%d/%d: dread - %s%s zero size\n", procid, opno, + f.path, st); free_pathname(&f); close(fd); return; @@ -1816,8 +1829,8 @@ dread_f(int opno, long r) if (xfsctl(f.path, fd, XFS_IOC_DIOINFO, &diob) < 0) { if (v) printf( - "%d/%d: dread - xfsctl(XFS_IOC_DIOINFO) %s failed %d\n", - procid, opno, f.path, errno); + "%d/%d: dread - xfsctl(XFS_IOC_DIOINFO) %s%s failed %d\n", + procid, opno, f.path, st, errno); free_pathname(&f); close(fd); return; @@ -1837,8 +1850,8 @@ dread_f(int opno, long r) e = read(fd, buf, len) < 0 ? errno : 0; free(buf); if (v) - printf("%d/%d: dread %s [%lld,%d] %d\n", - procid, opno, f.path, (long long)off, (int)len, e); + printf("%d/%d: dread %s%s [%lld,%d] %d\n", + procid, opno, f.path, st, (long long)off, (int)len, e); free_pathname(&f); close(fd); } @@ -1857,6 +1870,7 @@ dwrite_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -1883,11 +1897,12 @@ dwrite_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); if (xfsctl(f.path, fd, XFS_IOC_DIOINFO, &diob) < 0) { if (v) printf("%d/%d: dwrite - xfsctl(XFS_IOC_DIOINFO)" - " %s failed %d\n", - procid, opno, f.path, errno); + " %s%s failed %d\n", + procid, opno, f.path, st, errno); free_pathname(&f); close(fd); return; @@ -1910,8 +1925,8 @@ dwrite_f(int opno, long r) e = write(fd, buf, len) < 0 ? errno : 0; free(buf); if (v) - printf("%d/%d: dwrite %s [%lld,%d] %d\n", - procid, opno, f.path, (long long)off, (int)len, e); + printf("%d/%d: dwrite %s%s [%lld,%d] %d\n", + procid, opno, f.path, st, (long long)off, (int)len, e); free_pathname(&f); close(fd); } @@ -1960,6 +1975,7 @@ freesp_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -1986,6 +2002,7 @@ freesp_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; @@ -1994,8 +2011,8 @@ freesp_f(int opno, long r) fl.l_len = 0; e = xfsctl(f.path, fd, XFS_IOC_FREESP64, &fl) < 0 ? errno : 0; if (v) - printf("%d/%d: xfsctl(XFS_IOC_FREESP64) %s %lld 0 %d\n", - procid, opno, f.path, (long long)off, e); + printf("%d/%d: xfsctl(XFS_IOC_FREESP64) %s%s %lld 0 %d\n", + procid, opno, f.path, st, (long long)off, e); free_pathname(&f); close(fd); } @@ -2198,6 +2215,7 @@ read_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -2224,10 +2242,11 @@ read_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); if (stb.st_size == 0) { if (v) - printf("%d/%d: read - %s zero size\n", procid, opno, - f.path); + printf("%d/%d: read - %s%s zero size\n", procid, opno, + f.path, st); free_pathname(&f); close(fd); return; @@ -2240,8 +2259,8 @@ read_f(int opno, long r) e = read(fd, buf, len) < 0 ? errno : 0; free(buf); if (v) - printf("%d/%d: read %s [%lld,%d] %d\n", - procid, opno, f.path, (long long)off, (int)len, e); + printf("%d/%d: read %s%s [%lld,%d] %d\n", + procid, opno, f.path, st, (long long)off, (int)len, e); free_pathname(&f); close(fd); } @@ -2348,6 +2367,7 @@ resvsp_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -2374,6 +2394,7 @@ resvsp_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; @@ -2382,8 +2403,8 @@ resvsp_f(int opno, long r) fl.l_len = (off64_t)(random() % (1024 * 1024)); e = xfsctl(f.path, fd, XFS_IOC_RESVSP64, &fl) < 0 ? errno : 0; if (v) - printf("%d/%d: xfsctl(XFS_IOC_RESVSP64) %s %lld %lld %d\n", - procid, opno, f.path, + printf("%d/%d: xfsctl(XFS_IOC_RESVSP64) %s%s %lld %lld %d\n", + procid, opno, f.path, st, (long long)off, (long long)fl.l_len, e); free_pathname(&f); close(fd); @@ -2506,6 +2527,7 @@ truncate_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -2523,14 +2545,15 @@ truncate_f(int opno, long r) free_pathname(&f); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; e = truncate64_path(&f, off) < 0 ? errno : 0; check_cwd(); if (v) - printf("%d/%d: truncate %s %lld %d\n", procid, opno, f.path, - (long long)off, e); + printf("%d/%d: truncate %s%s %lld %d\n", procid, opno, f.path, + st, (long long)off, e); free_pathname(&f); } @@ -2574,6 +2597,7 @@ unresvsp_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { @@ -2600,6 +2624,7 @@ unresvsp_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; @@ -2608,8 +2633,8 @@ unresvsp_f(int opno, long r) fl.l_len = (off64_t)(random() % (1 << 20)); e = xfsctl(f.path, fd, XFS_IOC_UNRESVSP64, &fl) < 0 ? errno : 0; if (v) - printf("%d/%d: xfsctl(XFS_IOC_UNRESVSP64) %s %lld %lld %d\n", - procid, opno, f.path, + printf("%d/%d: xfsctl(XFS_IOC_UNRESVSP64) %s%s %lld %lld %d\n", + procid, opno, f.path, st, (long long)off, (long long)fl.l_len, e); free_pathname(&f); close(fd); @@ -2627,6 +2652,7 @@ write_f(int opno, long r) off64_t off; struct stat64 stb; int v; + char st[1024]; init_pathname(&f); if (!get_fname(FT_REGm, r, &f, NULL, NULL, &v)) { @@ -2653,6 +2679,7 @@ write_f(int opno, long r) close(fd); return; } + inode_info(st, sizeof(st), &stb, v); lr = ((__int64_t)random() << 32) + random(); off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); off %= maxfsize; @@ -2663,8 +2690,8 @@ write_f(int opno, long r) e = write(fd, buf, len) < 0 ? errno : 0; free(buf); if (v) - printf("%d/%d: write %s [%lld,%d] %d\n", - procid, opno, f.path, (long long)off, (int)len, e); + printf("%d/%d: write %s%s [%lld,%d] %d\n", + procid, opno, f.path, st, (long long)off, (int)len, e); free_pathname(&f); close(fd); } -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:34 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,J_CHICKENPOX_43, J_CHICKENPOX_53,J_CHICKENPOX_64,J_CHICKENPOX_73,J_CHICKENPOX_84, T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPXsL017913 for ; Thu, 3 Nov 2011 09:25:33 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0001-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D60F512FF48D for ; Thu, 3 Nov 2011 07:25:32 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id 1921OvEOg0Gpjonm for ; Thu, 03 Nov 2011 07:25:32 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=wTIpKCSj1HcPR2Pzv9L0PIjH0OkjaAl5JAkbXh+cpqo=; b=KzUJCUulpRyGz+BewS+lmUOLpw+xyeFSAS/D6nkPgueNRNF/ZXVL4JUaTPlJw0Za+h splyCDVCosog7wnhHUzDXBMnrVVSGmQ25rpoCETyX0MkRIy6yLC1T0EACu51x8ikKiW4 dfcoRD3zrTLFTAjgpJm1L12kwsx4fqDOhqt4Y= Received: by 10.204.142.80 with SMTP id p16mr8451730bku.10.1320330326254; Thu, 03 Nov 2011 07:25:26 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.24 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:25 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 02/12] xfstests: add different logging option to fsstress Subject: [PATCH 02/12] xfstests: add different logging option to fsstress Date: Thu, 3 Nov 2011 18:24:51 +0400 Message-Id: <1320330301-2682-3-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330332 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Currently the only way to log fsstress's output is to redirect it's shared stdout to pipe which is very painfull because: 1) Pipe writers are serialized via i_mutex so we waste cpu-cores power on stupid sinchronization for loging purpose, instead of hunting real race conditions, and bugs inside file system. 2) Usually output is corrupted due to luck of sychronization on shared stdout. Since fsstress's children operate on independend paths, let's just open didicated log file for each child and simply avoid useless sycnhronization. Reviewed-by: Christoph Hellwig Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 33 ++++++++++++++++++++++++++++++++- 1 files changed, 32 insertions(+), 1 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 51ecda2..c7001f3 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -258,6 +258,8 @@ int main(int argc, char **argv) char buf[10]; int c; char *dirname = NULL; + char *logname = NULL; + char rpath[PATH_MAX]; int fd; int i; int j; @@ -273,7 +275,7 @@ int main(int argc, char **argv) nops = sizeof(ops) / sizeof(ops[0]); ops_end = &ops[nops]; myprog = argv[0]; - while ((c = getopt(argc, argv, "d:e:f:i:m:n:p:rs:vwzHS")) != -1) { + while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:vwzHS")) != -1) { switch (c) { case 'd': dirname = optarg; @@ -311,6 +313,10 @@ int main(int argc, char **argv) case 'n': operations = atoi(optarg); break; + case 'o': + logname = optarg; + break; + case 'p': nproc = atoi(optarg); break; @@ -351,10 +357,26 @@ int main(int argc, char **argv) } (void)mkdir(dirname, 0777); + if (logname && logname[0] != '/') { + if (getcwd(rpath, sizeof(rpath)) < 0){ + perror("getcwd failed"); + exit(1); + } + } else { + rpath[0] = '\0'; + } if (chdir(dirname) < 0) { perror(dirname); exit(1); } + if (logname) { + char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/%s", rpath, logname); + if (freopen(path, "a", stdout) == NULL) { + perror("freopen logfile failed"); + exit(1); + } + } sprintf(buf, "fss%x", (unsigned int)getpid()); fd = creat(buf, 0666); if (lseek64(fd, (off64_t)(MAXFSIZE32 + 1ULL), SEEK_SET) < 0) @@ -409,6 +431,15 @@ int main(int argc, char **argv) close(fd); for (i = 0; i < nproc; i++) { if (fork() == 0) { + if (logname) { + char path[PATH_MAX]; + snprintf(path, sizeof(path), "%s/%s.%d", + rpath, logname, i); + if (freopen(path, "a", stdout) == NULL) { + perror("freopen logfile failed"); + exit(1); + } + } procid = i; doproc(); return 0; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:36 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_43,J_CHICKENPOX_92,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPZRM017927 for ; Thu, 3 Nov 2011 09:25:36 -0500 X-ASG-Debug-ID: 1320330324-4930018f0001-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 27AFA5683CC for ; Thu, 3 Nov 2011 07:25:34 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id fII8HWNYAhkHcU0s for ; Thu, 03 Nov 2011 07:25:34 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=lwrKmQs+5w3+7Nzd3wTkq/sIcyUdMK+X3RK0zcTiwpg=; b=ck5BrVTGJIfLeNGGNeaJrUIyHabpwnQPUNdtktc7Ux5Y/1jWxQiAusHKaCjnXMllZ1 piIWfEP1Bj79YGFDbkV88aJNzXML98egwAiRWzDW9w5N/n1NJ9h3BD35r+Q4pP/g5Z88 NMv68o1sQmC8TSzmiAxJ2ZcqpO/yrCLzT3a94= Received: by 10.204.141.65 with SMTP id l1mr8315692bku.82.1320330334441; Thu, 03 Nov 2011 07:25:34 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.32 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:33 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 06/12] xfstests: add fallocate support to fsstress Subject: [PATCH 06/12] xfstests: add fallocate support to fsstress Date: Thu, 3 Nov 2011 18:24:55 +0400 Message-Id: <1320330301-2682-7-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330335 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Add tests for fallocate(2) syscall - fallocate: reserve the disk space - punch: de-allocates the disk space Since FALLOC_FL_PUNCH_HOLE is relatively new it's value defined explicitly if not yet defined. Later we may clear that define. Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 126 insertions(+), 1 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 77e0682..f4c8d81 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -24,7 +24,13 @@ #ifdef HAVE_ATTR_ATTRIBUTES_H #include #endif - +#ifdef FALLOCATE +#include +#ifndef FALLOC_FL_PUNCH_HOLE +/* Copy-paste from linux/falloc.h */ +#define FALLOC_FL_PUNCH_HOLE 0x02 /* de-allocates range */ +#endif +#endif #ifndef HAVE_ATTR_LIST #define attr_list(path, buf, size, flags, cursor) (errno = -ENOSYS, -1) #endif @@ -48,6 +54,7 @@ typedef enum { OP_CREAT, OP_DREAD, OP_DWRITE, + OP_FALLOCATE, OP_FDATASYNC, OP_FREESP, OP_FSYNC, @@ -55,6 +62,7 @@ typedef enum { OP_LINK, OP_MKDIR, OP_MKNOD, + OP_PUNCH, OP_READ, OP_READLINK, OP_RENAME, @@ -128,6 +136,7 @@ void chown_f(int, long); void creat_f(int, long); void dread_f(int, long); void dwrite_f(int, long); +void fallocate_f(int, long); void fdatasync_f(int, long); void freesp_f(int, long); void fsync_f(int, long); @@ -135,6 +144,7 @@ void getdents_f(int, long); void link_f(int, long); void mkdir_f(int, long); void mknod_f(int, long); +void punch_f(int, long); void read_f(int, long); void readlink_f(int, long); void rename_f(int, long); @@ -159,6 +169,7 @@ opdesc_t ops[] = { { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DREAD, "dread", dread_f, 4, 0 }, { OP_DWRITE, "dwrite", dwrite_f, 4, 1 }, + { OP_FALLOCATE, "fallocate", fallocate_f, 1, 1 }, { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, @@ -166,6 +177,7 @@ opdesc_t ops[] = { { OP_LINK, "link", link_f, 1, 1 }, { OP_MKDIR, "mkdir", mkdir_f, 2, 1 }, { OP_MKNOD, "mknod", mknod_f, 2, 1 }, + { OP_PUNCH, "punch", punch_f, 1, 1 }, { OP_READ, "read", read_f, 1, 0 }, { OP_READLINK, "readlink", readlink_f, 1, 0 }, { OP_RENAME, "rename", rename_f, 2, 1 }, @@ -2012,6 +2024,63 @@ dwrite_f(int opno, long r) } void +fallocate_f(int opno, long r) +{ +#ifdef FALLOCATE + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + off64_t len; + struct stat64 stb; + int v; + char st[1024]; + int mode = 0; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: fallocate - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: fallocate - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: fallocate - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + len = (off64_t)(random() % (1024 * 1024)); + mode |= FALLOC_FL_KEEP_SIZE & random(); + e = fallocate(fd, mode, (loff_t)off, (loff_t)len) < 0 ? errno : 0; + if (v) + printf("%d/%d: fallocate(%d) %s %st %lld %lld %d\n", + procid, opno, mode, + f.path, st, (long long)off, (long long)len, e); + free_pathname(&f); + close(fd); +#endif +} + + +void fdatasync_f(int opno, long r) { int e; @@ -2284,6 +2353,62 @@ mknod_f(int opno, long r) } void +punch_f(int opno, long r) +{ +#ifdef FALLOCATE + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + off64_t len; + struct stat64 stb; + int v; + char st[1024]; + int mode = FALLOC_FL_PUNCH_HOLE; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: punch hole - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: punch hole - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: punch hole - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + len = (off64_t)(random() % (1024 * 1024)); + mode |= FALLOC_FL_KEEP_SIZE & random(); + e = fallocate(fd, mode, (loff_t)off, (loff_t)len) < 0 ? errno : 0; + if (v) + printf("%d/%d: punch hole(%d) %s %s %lld %lld %d\n", + procid, opno, mode, + f.path, st, (long long)off, (long long)len, e); + free_pathname(&f); + close(fd); +#endif +} + +void read_f(int opno, long r) { char *buf; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:38 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_43, J_CHICKENPOX_84,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPcQl017943 for ; Thu, 3 Nov 2011 09:25:38 -0500 X-ASG-Debug-ID: 1320330324-4930018f0002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8C4C05683D3 for ; Thu, 3 Nov 2011 07:25:37 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id e3DMoxBlh16uI3dP for ; Thu, 03 Nov 2011 07:25:37 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=9EgyH5BgtfikhoD4i+eHj+qkVSdEvwL54q8oYLUuIpU=; b=SxzmrzihwGGhK2mHe+6lVKJ0cAMpkRbLPwssOljRp3EmVpMM7ekpTNiedPSU5b74LX SJSytEXfUNcmEZ8ZPBDMp5/oQjHWsUgCPkh05+6P3UrPUn3vjajc9Qe2uOj3j4wo9AxC gpcS1WMjLs9RAZNglTCsKK7NrkKADvVhBV8Xc= Received: by 10.204.13.133 with SMTP id c5mr8669014bka.15.1320330328230; Thu, 03 Nov 2011 07:25:28 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.26 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:27 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 03/12] xfstests: fsstress should kill children tasks before exit Subject: [PATCH 03/12] xfstests: fsstress should kill children tasks before exit Date: Thu, 3 Nov 2011 18:24:52 +0400 Message-Id: <1320330301-2682-4-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330337 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean It is very hard to predict runtime for fsstress. In many cases it is useful to give test to run a reasonable time, and then kill it. But currently there is no reliable way to kill test without leaving running children. This patch add sanity cleanup logic which looks follow: - On sigterm received by parent, it resend signal to it's children - Wait for each child to terminates - EXTRA_SANITY: Even if parent was killed by other signal, children will be terminated with SIGKILL to preven staled children. So now one can simply run fsstress like this: ./fsstress -p 1000 -n999999999 -d $TEST_DIR & PID=$! sleep 300 kill $PID wait $PID Signed-off-by: Dmitry Monakhov --- aclocal.m4 | 5 +++++ configure.in | 1 + ltp/fsstress.c | 37 ++++++++++++++++++++++++++++++++++++- 3 files changed, 42 insertions(+), 1 deletions(-) diff --git a/aclocal.m4 b/aclocal.m4 index 168eb59..5532606 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -16,6 +16,11 @@ AC_DEFUN([AC_PACKAGE_WANT_LINUX_FIEMAP_H], AC_SUBST(have_fiemap) ]) +AC_DEFUN([AC_PACKAGE_WANT_LINUX_PRCTL_H], + [ AC_CHECK_HEADERS([sys/prctl.h], [ have_prctl=true ], [ have_prctl=false ]) + AC_SUBST(have_prctl) + ]) + AC_DEFUN([AC_PACKAGE_WANT_FALLOCATE], [ AC_MSG_CHECKING([for fallocate]) AC_TRY_LINK([ diff --git a/configure.in b/configure.in index c697b4f..76d23e4 100644 --- a/configure.in +++ b/configure.in @@ -67,6 +67,7 @@ in AC_PACKAGE_WANT_DMAPI AC_PACKAGE_WANT_LINUX_FIEMAP_H AC_PACKAGE_WANT_FALLOCATE + AC_PACKAGE_WANT_LINUX_PRCTL_H ;; esac diff --git a/ltp/fsstress.c b/ltp/fsstress.c index c7001f3..133a247 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -28,7 +28,9 @@ #ifndef HAVE_ATTR_LIST #define attr_list(path, buf, size, flags, cursor) (errno = -ENOSYS, -1) #endif - +#ifdef HAVE_SYS_PRCTL_H +#include +#endif #include #define XFS_ERRTAG_MAX 17 #define XFS_IDMODULO_MAX 31 /* user/group IDs (1 << x) */ @@ -209,6 +211,7 @@ int rtpct; unsigned long seed = 0; ino_t top_ino; int verbose = 0; +sig_atomic_t should_stop = 0; void add_to_flist(int, int, int); void append_pathname(pathname_t *, char *); @@ -253,6 +256,11 @@ void usage(void); void write_freq(void); void zero_freq(void); +void sg_handler(int signum) +{ + should_stop = 1; +} + int main(int argc, char **argv) { char buf[10]; @@ -269,6 +277,7 @@ int main(int argc, char **argv) ptrdiff_t srval; int nousage = 0; xfs_error_injection_t err_inj; + struct sigaction action; errrange = errtag = 0; umask(0); @@ -429,8 +438,27 @@ int main(int argc, char **argv) } } else close(fd); + + setpgid(0, 0); + action.sa_handler = sg_handler; + sigemptyset(&action.sa_mask); + action.sa_flags = 0; + if (sigaction(SIGTERM, &action, 0)) { + perror("sigaction failed"); + exit(1); + } + for (i = 0; i < nproc; i++) { if (fork() == 0) { + action.sa_handler = SIG_DFL; + sigemptyset(&action.sa_mask); + if (sigaction(SIGTERM, &action, 0)) + return 1; +#ifdef HAVE_SYS_PRCTL_H + prctl(PR_SET_PDEATHSIG, SIGKILL); + if (getppid() == 1) /* parent died already? */ + return 0; +#endif if (logname) { char path[PATH_MAX]; snprintf(path, sizeof(path), "%s/%s.%d", @@ -445,8 +473,15 @@ int main(int argc, char **argv) return 0; } } + while (wait(&stat) > 0 && !should_stop) { + continue; + } + action.sa_flags = SA_RESTART; + sigaction(SIGTERM, &action, 0); + kill(-getpid(), SIGTERM); while (wait(&stat) > 0) continue; + if (errtag != 0) { err_inj.errtag = 0; err_inj.fd = fd; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:39 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_43,J_CHICKENPOX_52,J_CHICKENPOX_64,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPcgQ017947 for ; Thu, 3 Nov 2011 09:25:39 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1AA7012FF495 for ; Thu, 3 Nov 2011 07:25:36 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id CD0DFu43iGitE4Fc for ; Thu, 03 Nov 2011 07:25:36 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=3+vHByiBfpQezOj0eiaSFOIBtgpeklBS/upqbTlF5V0=; b=qrXXcxnG1Sf48MMI/CgLudQz2F80bI1a0DFpuljIkcodHGRVv5/neg15Dcu6L8CvnW nW0MHwJ5GEY7BhzprzsvMCDbtoG7hjGob3ChdiOXLFqdz8fpp9ya/ps3FhBBdfL+FdIR IByS0PohxM4muMbBvihBsIFE12hmpwz2+5RY4= Received: by 10.204.129.151 with SMTP id o23mr8531580bks.51.1320330336244; Thu, 03 Nov 2011 07:25:36 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.34 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:35 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 07/12] xfstests: fsstress add FS_IOC_{SET,GET}FLAGS operations v2 Subject: [PATCH 07/12] xfstests: fsstress add FS_IOC_{SET,GET}FLAGS operations v2 Date: Thu, 3 Nov 2011 18:24:56 +0400 Message-Id: <1320330301-2682-8-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330337 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Add two new operations: - getattr: ioctl(fd, FS_IOC_GETFLAGS, &fl) - setattr: ioctl(fd, FS_IOC_SETFLAGS, &random_flags) Attribute mask may be passed via -M opt, by default is (~0). By default FS_IOC_SETFLAGS has zero probability because it may produce inodes with APPEND or IMMUTABLE flags which are not deletable by default. Let's assumes that one who enable it knows how to delete such inodes. For example like follows: find $TEST_PATH -exec chattr -i -a {} \; rm -rf $TEST_PATH Signed-off-by: Dmitry Monakhov --- aclocal.m4 | 4 +++ configure.in | 1 + ltp/fsstress.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletions(-) diff --git a/aclocal.m4 b/aclocal.m4 index 5532606..5739004 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -21,6 +21,10 @@ AC_DEFUN([AC_PACKAGE_WANT_LINUX_PRCTL_H], AC_SUBST(have_prctl) ]) +AC_DEFUN([AC_PACKAGE_WANT_LINUX_FS_H], + [ AC_CHECK_HEADER([linux/fs.h]) + ]) + AC_DEFUN([AC_PACKAGE_WANT_FALLOCATE], [ AC_MSG_CHECKING([for fallocate]) AC_TRY_LINK([ diff --git a/configure.in b/configure.in index 76d23e4..3b40e55 100644 --- a/configure.in +++ b/configure.in @@ -68,6 +68,7 @@ in AC_PACKAGE_WANT_LINUX_FIEMAP_H AC_PACKAGE_WANT_FALLOCATE AC_PACKAGE_WANT_LINUX_PRCTL_H + AC_PACKAGE_WANT_LINUX_FS_H ;; esac diff --git a/ltp/fsstress.c b/ltp/fsstress.c index f4c8d81..997f068 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -37,6 +37,15 @@ #ifdef HAVE_SYS_PRCTL_H #include #endif + +#include +#ifndef FS_IOC_GETFLAGS +#define FS_IOC_GETFLAGS _IOR('f', 1, long) +#endif +#ifndef FS_IOC_SETFLAGS +#define FS_IOC_SETFLAGS _IOW('f', 2, long) +#endif + #include #define XFS_ERRTAG_MAX 17 #define XFS_IDMODULO_MAX 31 /* user/group IDs (1 << x) */ @@ -58,6 +67,7 @@ typedef enum { OP_FDATASYNC, OP_FREESP, OP_FSYNC, + OP_GETATTR, OP_GETDENTS, OP_LINK, OP_MKDIR, @@ -68,6 +78,7 @@ typedef enum { OP_RENAME, OP_RESVSP, OP_RMDIR, + OP_SETATTR, OP_SETXATTR, OP_STAT, OP_SYMLINK, @@ -140,6 +151,7 @@ void fallocate_f(int, long); void fdatasync_f(int, long); void freesp_f(int, long); void fsync_f(int, long); +void getattr_f(int, long); void getdents_f(int, long); void link_f(int, long); void mkdir_f(int, long); @@ -150,6 +162,7 @@ void readlink_f(int, long); void rename_f(int, long); void resvsp_f(int, long); void rmdir_f(int, long); +void setattr_f(int, long); void setxattr_f(int, long); void stat_f(int, long); void symlink_f(int, long); @@ -173,6 +186,7 @@ opdesc_t ops[] = { { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, + { OP_GETATTR, "getattr", getattr_f, 1, 0 }, { OP_GETDENTS, "getdents", getdents_f, 1, 0 }, { OP_LINK, "link", link_f, 1, 1 }, { OP_MKDIR, "mkdir", mkdir_f, 2, 1 }, @@ -183,6 +197,7 @@ opdesc_t ops[] = { { OP_RENAME, "rename", rename_f, 2, 1 }, { OP_RESVSP, "resvsp", resvsp_f, 1, 1 }, { OP_RMDIR, "rmdir", rmdir_f, 1, 1 }, + { OP_SETATTR, "setattr", setattr_f, 0, 1 }, { OP_SETXATTR, "setxattr", setxattr_f, 1, 1 }, { OP_STAT, "stat", stat_f, 1, 0 }, { OP_SYMLINK, "symlink", symlink_f, 2, 1 }, @@ -218,6 +233,7 @@ int nops; int nproc = 1; int operations = 1; unsigned int idmodulo = XFS_IDMODULO_MAX; +unsigned int attr_mask = ~0; int procid; int rtpct; unsigned long seed = 0; @@ -296,7 +312,7 @@ int main(int argc, char **argv) nops = sizeof(ops) / sizeof(ops[0]); ops_end = &ops[nops]; myprog = argv[0]; - while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:S:vwzH")) != -1) { + while ((c = getopt(argc, argv, "d:e:f:i:m:M:n:o:p:rs:S:vwzH")) != -1) { switch (c) { case 'd': dirname = optarg; @@ -356,6 +372,9 @@ int main(int argc, char **argv) case 'z': zero_freq(); break; + case 'M': + attr_mask = strtoul(optarg, NULL, 0); + break; case 'S': i = 0; if (optarg[0] == 'c') @@ -2199,6 +2218,29 @@ fsync_f(int opno, long r) } void +getattr_f(int opno, long r) +{ + int fd; + int e; + pathname_t f; + uint fl; + int v; + + init_pathname(&f); + if (!get_fname(FT_ANYm, r, &f, NULL, NULL, &v)) + append_pathname(&f, "."); + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + + e = ioctl(fd, FS_IOC_GETFLAGS, &fl); + if (v) + printf("%d/%d: getattr %s %u %d\n", procid, opno, f.path, fl, e); + free_pathname(&f); + close(fd); +} + +void getdents_f(int opno, long r) { DIR *dir; @@ -2644,6 +2686,30 @@ rmdir_f(int opno, long r) } void +setattr_f(int opno, long r) +{ + int fd; + int e; + pathname_t f; + uint fl; + int v; + + init_pathname(&f); + if (!get_fname(FT_ANYm, r, &f, NULL, NULL, &v)) + append_pathname(&f, "."); + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + + fl = attr_mask & (uint)random(); + e = ioctl(fd, FS_IOC_SETFLAGS, &fl); + if (v) + printf("%d/%d: setattr %s %x %d\n", procid, opno, f.path, fl, e); + free_pathname(&f); + close(fd); +} + +void stat_f(int opno, long r) { int e; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_43, J_CHICKENPOX_64,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPeDM017968 for ; Thu, 3 Nov 2011 09:25:41 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0003-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 00DAC12FF497 for ; Thu, 3 Nov 2011 07:25:39 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id pSUjAs4dkCsSzCsn for ; Thu, 03 Nov 2011 07:25:39 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=LlqyoRSo8YMqqaSZNPVlMIVae72EkbrpmiZGJBXnCTo=; b=P2MjMKJD83gvyowddmg+SdRMx+fiJx7AMklPg4Re+MEEVu/ufBIOoEGkSTe5giQDqP MQOualoFLCAURpyhzKJnK1oeyWTp+/Weo2reUYJSYiIemTzVwLJbqQb148uzluMVKqGw AdVewdqF9nCwKbyKWJOCB7VDxNfe3k7msyEqc= Received: by 10.204.148.145 with SMTP id p17mr8459939bkv.61.1320330330072; Thu, 03 Nov 2011 07:25:30 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.28 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:29 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 04/12] xfstests: fsstress add command line style output for show_opts Subject: [PATCH 04/12] xfstests: fsstress add command line style output for show_opts Date: Thu, 3 Nov 2011 18:24:53 +0400 Message-Id: <1320330301-2682-5-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330340 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Once some combination of seed+fs_ops result in regression it is reasonable to document that combination. It is usefull to dump that configuration in command line style. Later this line may be simply hardcoded in to regression test. Signed-off-by: Dmitry Monakhov --- 070 | 2 +- 117 | 2 +- ltp/fsstress.c | 22 ++++++++++++++++++---- 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/070 b/070 index 4458011..f48c33c 100755 --- a/070 +++ b/070 @@ -62,7 +62,7 @@ $FSSTRESS_PROG \ -f unresvsp=0 \ -f attr_set=100 \ -f attr_remove=100 \ - -S -p 1 -n 10000 >$seq.full 2>&1 + -p 1 -n 10000 -S c >$seq.full 2>&1 status=$? exit diff --git a/117 b/117 index fcf0485..edec157 100755 --- a/117 +++ b/117 @@ -85,7 +85,7 @@ while [ $i -lt $ITERATIONS ]; do -f attr_set=100 \ -f attr_remove=100 \ -s $seed \ - -S -p 1 -n 1000 >>$seq.full 2>&1 + -p 1 -n 1000 -S c >>$seq.full 2>&1 let i=$i+1 done diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 133a247..77e0682 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -284,7 +284,7 @@ int main(int argc, char **argv) nops = sizeof(ops) / sizeof(ops[0]); ops_end = &ops[nops]; myprog = argv[0]; - while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:vwzHS")) != -1) { + while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:S:vwzH")) != -1) { switch (c) { case 'd': dirname = optarg; @@ -345,7 +345,10 @@ int main(int argc, char **argv) zero_freq(); break; case 'S': - show_ops(0, NULL); + i = 0; + if (optarg[0] == 'c') + i = 1; + show_ops(1, NULL); printf("\n"); nousage=1; break; @@ -1310,7 +1313,8 @@ show_ops(int flag, char *lead_str) x+=printf("%s ", p->name); } printf("\n"); - } else { + } else if (flag == 0) { + /* Table view style */ int f; for (f = 0, p = ops; p < ops_end; p++) f += p->freq; @@ -1327,6 +1331,16 @@ show_ops(int flag, char *lead_str) (p->iswrite == 0) ? " " : "write op"); } } + } else { + /* Command line style */ + if (lead_str != NULL) + printf("%s", lead_str); + printf ("-z -s %ld -m %d -n %d -p %d ", seed, idmodulo, + operations, nproc); + for (p = ops; p < ops_end; p++) + if (p->freq > 0) + printf("-f%s=%d ",p->name, p->freq); + printf("\n"); } } @@ -1433,7 +1447,7 @@ usage(void) printf(" -v specifies verbose mode\n"); printf(" -w zeros frequencies of non-write operations\n"); printf(" -z zeros frequencies of all operations\n"); - printf(" -S prints the table of operations (omitting zero frequency)\n"); + printf(" -S [c,t] prints the list of operations (omitting zero frequency) in command line or table style\n"); printf(" -H prints usage and exits\n"); } -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:43 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPhSr017984 for ; Thu, 3 Nov 2011 09:25:43 -0500 X-ASG-Debug-ID: 1320330324-4930018f0003-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id F0E775683DE for ; Thu, 3 Nov 2011 07:25:42 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id V7DaKsaqGYbhAxZW for ; Thu, 03 Nov 2011 07:25:42 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=ea/9F15k5AIV7M3Fz1LosYy4zrUGwswABYYTd8kYo1w=; b=oFmK9PYW1d/+pYoKrYgdP/opgMCyo0C2Lyxsn83rSqgAVEtIFwmho2WMM2+2UkU8a8 0OZqX833Wx/p4qEG0Qs1vgRJqzK/vrRNfNK0opI76du2Nmox1Qis70lwBP+FDE78cZW6 x7PlZosB1UsSNQz+GzOwkgzcEvCgxD6a9mK8M= Received: by 10.204.148.145 with SMTP id p17mr8460154bkv.61.1320330342307; Thu, 03 Nov 2011 07:25:42 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.40 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:41 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 10/12] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Subject: [PATCH 10/12] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Date: Thu, 3 Nov 2011 18:24:59 +0400 Message-Id: <1320330301-2682-11-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330342 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean - Same as 264 but with quota enabled. - IO performed from $qa_user user - fsstress granted with CAP_CHOWN capability. Signed-off-by: Dmitry Monakhov --- 265 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 265.out | 8 +++++ group | 1 + 3 files changed, 117 insertions(+), 0 deletions(-) create mode 100755 265 create mode 100644 265.out diff --git a/265 b/265 new file mode 100755 index 0000000..b568fdd --- /dev/null +++ b/265 @@ -0,0 +1,108 @@ +#! /bin/bash +# FSQA Test No. 265 +# +# Run fsstress and ENSPC hitters in parallel, check quota and +# fs consistency an the end +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.quota + +# Disable all sync operations to get higher load +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" +_workout() +{ + echo "" + echo "Run fsstress" + echo "" + num_iterations=10 + enospc_time=2 + out=$SCRATCH_MNT/fsstress.$$ + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" + echo "fsstress $args" >> $here/$seq.full + # Grant chown capability + cp $FSSTRESS_PROG $tmp.fsstress.bin + setcap cap_chown=epi $tmp.fsstress.bin + + (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 + pid=$! + + echo "Run dd writers in parallel" + for ((i=0; i < num_iterations; i++)) + do + # File will be opened with O_TRUNC each time + su $qa_user -c "dd if=/dev/zero \ + of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M " \ + > /dev/null 2>&1 + sleep $enospc_time + done + + killall $tmp.fsstress.bin + wait $pid +} + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_quota +_require_user +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 +_scratch_mount "-o usrquota,grpquota" +chmod 777 $SCRATCH_MNT +quotacheck -u -g $SCRATCH_MNT 2>/dev/null +quotaon -u -g $SCRATCH_MNT 2>/dev/null + +if ! _workout; then + _scratch_unmount 2>/dev/null + exit +fi + +if ! _check_quota_usage; then + _scratch_unmount 2>/dev/null + status=1 + exit +fi + +echo Comparing filesystem consistency +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +_check_scratch_fs +status=$? +exit diff --git a/265.out b/265.out new file mode 100644 index 0000000..7637291 --- /dev/null +++ b/265.out @@ -0,0 +1,8 @@ +QA output created by 265 + +Run fsstress + +Run dd writers in parallel +Comparing user usage +Comparing group usage +Comparing filesystem consistency diff --git a/group b/group index e79c29b..487051a 100644 --- a/group +++ b/group @@ -378,3 +378,4 @@ deprecated 262 auto quick quota 263 rw auto quick 264 auto rw prealloc ioctl enospc +265 auto quota rw prealloc ioctl enospc -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPiLv017988 for ; Thu, 3 Nov 2011 09:25:44 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0004-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 758B712FF779 for ; Thu, 3 Nov 2011 07:25:42 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id pDgtW9uQCzZPpxd8 for ; Thu, 03 Nov 2011 07:25:42 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=68neXnZw59Ov9iWpHPbl7AXX1hWPtfBFfOmrB3VQLO0=; b=MLrJiCHL91rWq7z39UZeEA6wp9OH6U6+5mu5yrwC3dM99WL67Mcf1H14Il17f00015 wxfNXuI9vtvyaqZfXN2uQc1nKsdbNNwmRhFBCSZE+5o+yWCR3arhP2hA3aHoeuxLpyv4 GKnRGeht4G51uh1U7QMvww+kER3hGkqXngjCo= Received: by 10.204.140.129 with SMTP id i1mr8445432bku.19.1320330340413; Thu, 03 Nov 2011 07:25:40 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.38 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:39 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Date: Thu, 3 Nov 2011 18:24:58 +0400 Message-Id: <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330343 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean During stress testing we want to cover as much code paths as possible fsstress is very good for this purpose. But it has expandable nature (disk usage almost continually grow). So once it goes in no ENOSPC condition it will be where till the end. But by running 'dd' writers in parallel we can regularly trigger ENOSPC but only for a limited periods of time because each time it opens the same file with O_TRUNC. Signed-off-by: Dmitry Monakhov --- 264 | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 264.out | 5 +++ group | 1 + 3 files changed, 92 insertions(+), 0 deletions(-) create mode 100755 264 create mode 100644 264.out diff --git a/264 b/264 new file mode 100755 index 0000000..fc6df23 --- /dev/null +++ b/264 @@ -0,0 +1,86 @@ +#! /bin/bash +# FSQA Test No. 264 +# +# Run fsstress and ENSPC hitters in parallel, check fs consistency an the end +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +# Disable all sync operations to get higher load +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" +_workout() +{ + echo "" + echo "Run fsstress" + echo "" + num_iterations=10 + enospc_time=2 + out=$SCRATCH_MNT/fsstress.$$ + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" + echo "fsstress $args" >> $here/$seq.full + $FSSTRESS_PROG $args > /dev/null 2>&1 & + pid=$! + echo "Run dd writers in parallel" + for ((i=0; i < num_iterations; i++)) + do + # File will be opened with O_TRUNC each time + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ + > /dev/null 2>&1 + sleep $enospc_time + done + kill $pid + wait $pid +} + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 +_scratch_mount + +if ! _workout; then + umount $SCRATCH_DEV 2>/dev/null + exit +fi + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +_check_scratch_fs +status=$? +exit diff --git a/264.out b/264.out new file mode 100644 index 0000000..81b50e5 --- /dev/null +++ b/264.out @@ -0,0 +1,5 @@ +QA output created by 264 + +Run fsstress + +Run dd writers in parallel diff --git a/group b/group index 2a8970c..e79c29b 100644 --- a/group +++ b/group @@ -377,3 +377,4 @@ deprecated 261 auto quick quota 262 auto quick quota 263 rw auto quick +264 auto rw prealloc ioctl enospc -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:49 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, J_CHICKENPOX_47,J_CHICKENPOX_56,J_CHICKENPOX_57,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPmdJ018037 for ; Thu, 3 Nov 2011 09:25:48 -0500 X-ASG-Debug-ID: 1320330324-4930018f0005-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A59705683F4 for ; Thu, 3 Nov 2011 07:25:47 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id s5x1pKiYP8yWKBAW for ; Thu, 03 Nov 2011 07:25:47 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=K51xC98Y5ldB9WY1rY3ywp+hldq/Mpt9bx2cQyxGcyo=; b=esg0EQunPgY8Q9AWsa+xjCnQ2+ckyrmqI4QedmFxzOoQQey2xoCzCfjcb177GbriCh FGAB2DEOsbEg57pgPxMMd7BJZSQmmI2XJdUGkmUT651Ks6TOFLEmYMbiznzp4x68o4lp rHywb1QY5cMVlBGPZ8indEBAjk3MnE1eovRhU= Received: by 10.204.13.133 with SMTP id c5mr8669335bka.15.1320330346912; Thu, 03 Nov 2011 07:25:46 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.44 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:45 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 12/12] xfstress: Test data journaling flag switch for a single file Subject: [PATCH 12/12] xfstress: Test data journaling flag switch for a single file Date: Thu, 3 Nov 2011 18:25:01 +0400 Message-Id: <1320330301-2682-13-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330347 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Signed-off-by: Dmitry Monakhov --- 267 | 111 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 267.out | 37 +++++++++++++++++++++ group | 1 + 3 files changed, 149 insertions(+), 0 deletions(-) create mode 100644 267 create mode 100644 267.out diff --git a/267 b/267 new file mode 100644 index 0000000..e2ce347 --- /dev/null +++ b/267 @@ -0,0 +1,111 @@ +#! /bin/bash +# FSQA Test No. 267 +# +# Test data journaling flag switch for a single file +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +_workout() +{ + echo "" + echo " Switch data journalling mode" + echo "" + write_opt_list="iflag=noatime conv=notrunc conv=fsync oflag=direct" + chattr_opt_list="+j -j +jS -j" + idx=0 + # + for write_opt in $write_opt_list + do + idx=$((idx + 1)) + for chattr_opt in $chattr_opt_list + do + + echo "OP write_opt: $write_opt 4M, \ +chattr_opt: $chattr_opt" + dd if=/dev/zero of=$SCRATCH_MNT/file.$idx \ + bs=1M count=4 $write_opt \ + > /dev/null 2>&1 || exit + chattr $chattr_opt $SCRATCH_MNT/file.$idx \ + || exit + done + done + sync + # Same as previous test, but for one file, + # and with ENOSPC triggering + for write_opt in $write_opt_list + do + idx=$((idx + 1)) + for chattr_opt in $chattr_opt_list + do + + echo "OP write_opt: $write_opt ENOSPC, \ +chattr_opt: $chattr_opt" + dd if=/dev/zero of=$SCRATCH_MNT/file.$idx \ + bs=1M $write_opt > /dev/null 2>&1 + chattr $chattr_opt $SCRATCH_MNT/file.$idx \ + || exit + done + sync + unlink $SCRATCH_MNT/file.$idx + done +} + +# real QA test starts here +_supported_fs ext3 ext4 +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((64 * 1024 * 1024)) >> $seq.full 2>&1 + +# -onoload and EXT4_SYNC_FL on file is important becase result in +# metadata sync writes inside ext4_handle_dirty_metadata() +_scratch_mount + +if ! _workout; then + echo "workout failed" + _scratch_unmount + status=1 + exit +fi + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +echo "Check filesystem" +_check_scratch_fs +status=$? +exit diff --git a/267.out b/267.out new file mode 100644 index 0000000..d46b6f4 --- /dev/null +++ b/267.out @@ -0,0 +1,37 @@ +QA output created by 267 + + Switch data journalling mode + +OP write_opt: iflag=noatime 4M, chattr_opt: +j +OP write_opt: iflag=noatime 4M, chattr_opt: -j +OP write_opt: iflag=noatime 4M, chattr_opt: +jS +OP write_opt: iflag=noatime 4M, chattr_opt: -j +OP write_opt: conv=notrunc 4M, chattr_opt: +j +OP write_opt: conv=notrunc 4M, chattr_opt: -j +OP write_opt: conv=notrunc 4M, chattr_opt: +jS +OP write_opt: conv=notrunc 4M, chattr_opt: -j +OP write_opt: conv=fsync 4M, chattr_opt: +j +OP write_opt: conv=fsync 4M, chattr_opt: -j +OP write_opt: conv=fsync 4M, chattr_opt: +jS +OP write_opt: conv=fsync 4M, chattr_opt: -j +OP write_opt: oflag=direct 4M, chattr_opt: +j +OP write_opt: oflag=direct 4M, chattr_opt: -j +OP write_opt: oflag=direct 4M, chattr_opt: +jS +OP write_opt: oflag=direct 4M, chattr_opt: -j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: +j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: -j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: +jS +OP write_opt: iflag=noatime ENOSPC, chattr_opt: -j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: +j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: -j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: +jS +OP write_opt: conv=notrunc ENOSPC, chattr_opt: -j +OP write_opt: conv=fsync ENOSPC, chattr_opt: +j +OP write_opt: conv=fsync ENOSPC, chattr_opt: -j +OP write_opt: conv=fsync ENOSPC, chattr_opt: +jS +OP write_opt: conv=fsync ENOSPC, chattr_opt: -j +OP write_opt: oflag=direct ENOSPC, chattr_opt: +j +OP write_opt: oflag=direct ENOSPC, chattr_opt: -j +OP write_opt: oflag=direct ENOSPC, chattr_opt: +jS +OP write_opt: oflag=direct ENOSPC, chattr_opt: -j +Check filesystem diff --git a/group b/group index dfc6fa7..5efb43c 100644 --- a/group +++ b/group @@ -380,3 +380,4 @@ deprecated 264 auto rw prealloc ioctl enospc 265 auto quota rw prealloc ioctl enospc 266 rw auto quick +267 rw auto quick -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:45 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPioD018002 for ; Thu, 3 Nov 2011 09:25:44 -0500 X-ASG-Debug-ID: 1320330324-4930018f0004-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3A2145683EB for ; Thu, 3 Nov 2011 07:25:43 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id LzmPSn5soQJ0imJy for ; Thu, 03 Nov 2011 07:25:43 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398069bkb.26 for ; Thu, 03 Nov 2011 07:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=U3+dMW0fjq5f1ucH1GyO6/mhttD+2jzYiQzZdxNemvo=; b=S0DKuYkFUw9+Fjppf7h9bZf9POCuBPz9tjWZWtusrTc25USjLyQmD4rzxuyFqeucGh U1FV8mPIs5hrrX78pZh0eT1Le9lL6kEI4xVEJdreIRygFDp25Yb2TLK8miZtKWgh0Utv H+nLt75Rzbc7CUyLvzkFSgt5x2HyCZkLtTEqA= Received: by 10.204.15.137 with SMTP id k9mr8562131bka.74.1320330332282; Thu, 03 Nov 2011 07:25:32 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.30 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:31 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 05/12] xfstests: freeze fsstress options for 117'th Subject: [PATCH 05/12] xfstests: freeze fsstress options for 117'th Date: Thu, 3 Nov 2011 18:24:54 +0400 Message-Id: <1320330301-2682-6-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330344 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is regression tescase for explicit combination of seed+fs_opts. Let's hardcode all options to prevent interactions with fsstress changes in future. Signed-off-by: Dmitry Monakhov --- 117 | 39 ++++++++++++++++++++++++++++----------- 1 files changed, 28 insertions(+), 11 deletions(-) diff --git a/117 b/117 index edec157..2e3f658 100755 --- a/117 +++ b/117 @@ -33,8 +33,34 @@ tmp=/tmp/$$ status=1 # failure is the default! trap "_cleanup; exit \$status" 0 1 2 3 15 -# seed for fsstress known to cause curruptions +# seed and options for fsstress known to cause curruptions seed=1122413794 +fss_ops=" -z \ + -f attr_remove=100 \ + -f attr_set=100 \ + -f chown=3 \ + -f creat=4 \ + -f dread=4 \ + -f dwrite=4 \ + -f fdatasync=1 \ + -f fsync=1 \ + -f getdents=1 \ + -f link=1 \ + -f mkdir=2 \ + -f mknod=2 \ + -f read=1 \ + -f readlink=1 \ + -f rename=2 \ + -f rmdir=1 \ + -f setxattr=1 \ + -f stat=1 \ + -f symlink=2 \ + -f sync=1 \ + -f truncate=2 \ + -f unlink=1 \ + -f write=4 \ + -s $seed \ + -m 31 -n 1000 -p 1" ITERATIONS=10 @@ -76,16 +102,7 @@ while [ $i -lt $ITERATIONS ]; do echo fsstress iteration: $i | tee -a $seq.full $FSSTRESS_PROG \ -d $SCRATCH_MNT/fsstress \ - -f allocsp=0 \ - -f freesp=0 \ - -f bulkstat=0 \ - -f bulkstat1=0 \ - -f resvsp=0 \ - -f unresvsp=0 \ - -f attr_set=100 \ - -f attr_remove=100 \ - -s $seed \ - -p 1 -n 1000 -S c >>$seq.full 2>&1 + $fss_ops -S c >>$seq.full 2>&1 let i=$i+1 done -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:46 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPjpx018014 for ; Thu, 3 Nov 2011 09:25:45 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0005-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CF53A12FF779 for ; Thu, 3 Nov 2011 07:25:44 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id JmDFoDR6GmfDjvQO for ; Thu, 03 Nov 2011 07:25:44 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=WzeNBbWqraSnorEyCNZgQwDEqVeHWP9HJwVl0VZ2Y2c=; b=Y7aNave28wp73qj7pSRdGoR4mvsel46kFp3AmVloc02+/M39xc0NlDgDkywKJ9cOrt yQd1j9CtQsoIY8cl2GmhuBz38lSw7mrSlClYhz5Z2Eoz3pJOsowS3O+JK1TOeVdMSsQ3 CjU5riyB3jC29B+aCvgTacthR55+ZWqMDtPak= Received: by 10.204.131.146 with SMTP id x18mr4775393bks.37.1320330344067; Thu, 03 Nov 2011 07:25:44 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.42 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:43 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 11/12] xfstress: add regression testcase for d583fb87a3ff0 Subject: [PATCH 11/12] xfstress: add regression testcase for d583fb87a3ff0 Date: Thu, 3 Nov 2011 18:25:00 +0400 Message-Id: <1320330301-2682-12-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330344 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Should be fixed by following patch http://patchwork.ozlabs.org/patch/120883/ Signed-off-by: Dmitry Monakhov --- 266 | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 266.out | 44 ++++++++++++++++++++++++++++++++++++++++ group | 1 + 3 files changed, 114 insertions(+), 0 deletions(-) create mode 100755 266 create mode 100644 266.out diff --git a/266 b/266 new file mode 100755 index 0000000..4209532 --- /dev/null +++ b/266 @@ -0,0 +1,69 @@ +#! /bin/bash +# FSQA Test No. 266 +# +# Regression testcase for d583fb87a3ff0 (ext4 extent corruption) +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs ext4 +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((128 * 1024 * 1024)) >> $seq.full 2>&1 + +# -onoload and EXT4_SYNC_FL on file is important becase result in +# metadata sync writes inside ext4_handle_dirty_metadata() +_scratch_mount -onoload +touch $SCRATCH_MNT/file +chattr +S $SCRATCH_MNT/file +# Create sparse file +for ((i = 0; i < 21; i++)) +do + dd if=/dev/zero of=$SCRATCH_MNT/file bs=4k count=1 seek=$((i*10)) +done +# truncate last extent +truncate $SCRATCH_MNT/file --size $((4096* 200)) + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +echo "Check filesystem" +_check_scratch_fs +status=$? +exit diff --git a/266.out b/266.out new file mode 100644 index 0000000..a248232 --- /dev/null +++ b/266.out @@ -0,0 +1,44 @@ +QA output created by 266 +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +Check filesystem diff --git a/group b/group index 487051a..dfc6fa7 100644 --- a/group +++ b/group @@ -379,3 +379,4 @@ deprecated 263 rw auto quick 264 auto rw prealloc ioctl enospc 265 auto quota rw prealloc ioctl enospc +266 rw auto quick -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 09:25:57 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_52,J_CHICKENPOX_66,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EPuet018075 for ; Thu, 3 Nov 2011 09:25:56 -0500 X-ASG-Debug-ID: 1320330322-1d7501ad0006-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D8D2412FF785 for ; Thu, 3 Nov 2011 07:25:55 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id RUJcl5WyTVkdi6GG for ; Thu, 03 Nov 2011 07:25:55 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1398030bkb.26 for ; Thu, 03 Nov 2011 07:25:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=7z+h2cHDBd9+THUTwnZsKDaeYZU3mVNpAqLTA+yWSKQ=; b=cTQk/PXTSwQpTQwXQh8jlt1IDnCzjcjvPJAqCTqHdEwgK4fbr8JSrbXT1OWAu0fob2 CfUOTgEhRW1h8CYG1AhFRzp3WpJPpxZ4eGsByoUopWxcjcpLo6ngxuO0Mi93s5Q4W05A prn5qmmwKWXLr8vaoi0+QL8Y9791aBxt/YxYM= Received: by 10.204.141.134 with SMTP id m6mr8518602bku.91.1320330338038; Thu, 03 Nov 2011 07:25:38 -0700 (PDT) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id j9sm5771845bkd.2.2011.11.03.07.25.36 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 07:25:37 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 08/12] xfstests: add fiemap operation to fsstress Subject: [PATCH 08/12] xfstests: add fiemap operation to fsstress Date: Thu, 3 Nov 2011 18:24:57 +0400 Message-Id: <1320330301-2682-9-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320330355 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79210 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Related bug: http://patchwork.ozlabs.org/patch/118863 Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 76 insertions(+), 0 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 997f068..cddfe74 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -24,6 +24,9 @@ #ifdef HAVE_ATTR_ATTRIBUTES_H #include #endif +#ifdef HAVE_LINUX_FIEMAP_H +#include +#endif #ifdef FALLOCATE #include #ifndef FALLOC_FL_PUNCH_HOLE @@ -65,6 +68,7 @@ typedef enum { OP_DWRITE, OP_FALLOCATE, OP_FDATASYNC, + OP_FIEMAP, OP_FREESP, OP_FSYNC, OP_GETATTR, @@ -149,6 +153,7 @@ void dread_f(int, long); void dwrite_f(int, long); void fallocate_f(int, long); void fdatasync_f(int, long); +void fiemap_f(int, long); void freesp_f(int, long); void fsync_f(int, long); void getattr_f(int, long); @@ -184,6 +189,7 @@ opdesc_t ops[] = { { OP_DWRITE, "dwrite", dwrite_f, 4, 1 }, { OP_FALLOCATE, "fallocate", fallocate_f, 1, 1 }, { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, + { OP_FIEMAP, "fiemap", fiemap_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, { OP_GETATTR, "getattr", getattr_f, 1, 0 }, @@ -2131,6 +2137,76 @@ fdatasync_f(int opno, long r) free_pathname(&f); close(fd); } +void +fiemap_f(int opno, long r) +{ +#ifdef HAVE_LINUX_FIEMAP_H + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + struct stat64 stb; + int v; + char st[1024]; + int blocks_to_map; + struct fiemap *fiemap; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: fiemap - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: fiemap - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: fiemap - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + blocks_to_map = random() & 0xffff; + fiemap = (struct fiemap *)malloc(sizeof(struct fiemap) + + (blocks_to_map * sizeof(struct fiemap_extent))); + if (!fiemap) { + if (v) + printf("%d/%d: malloc failed \n", procid, opno); + free_pathname(&f); + close(fd); + return; + } + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + fiemap->fm_flags = random() & (FIEMAP_FLAGS_COMPAT | 0x10000); + fiemap->fm_extent_count = blocks_to_map; + fiemap->fm_mapped_extents = random() & 0xffff; + fiemap->fm_start = off; + fiemap->fm_length = ((__int64_t)random() << 32) + random(); + + e = ioctl(fd, FS_IOC_FIEMAP, (unsigned long)fiemap); + if (v) + printf("%d/%d: ioctl(FIEMAP) %s%s %lld %lld %x %d\n", + procid, opno, f.path, st, (long long)fiemap->fm_start, + (long long) fiemap->fm_length, fiemap->fm_flags, e); + free(fiemap); + free_pathname(&f); + close(fd); +#endif +} void freesp_f(int opno, long r) -- 1.7.1 From dawood@bbmotors.co.bw Thu Nov 3 09:39:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: **** X-Spam-Status: No, score=4.6 required=5.0 tests=BAYES_50,HTML_MESSAGE, RCVD_IN_SORBS_WEB,UNPARSEABLE_RELAY autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Ed8J2018948 for ; Thu, 3 Nov 2011 09:39:10 -0500 X-ASG-Debug-ID: 1320331143-1bec000c0000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from vbnemlrel7.vbnservices.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 52E221CDF16F for ; Thu, 3 Nov 2011 07:39:04 -0700 (PDT) Received: from vbnemlrel7.vbnservices.net (vbnemlrel7.vbnservices.net [83.143.24.55]) by cuda.sgi.com with ESMTP id mJH8mevGFaekNGRn for ; Thu, 03 Nov 2011 07:39:04 -0700 (PDT) Received: from localhost (unknown [127.0.0.1]) by vbnemlrel7.vbnservices.net (Postfix) with ESMTP id 6D2692A526 for ; Thu, 3 Nov 2011 14:51:39 +0000 (UTC) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at example.com Received: from vbnemlrel7.vbnservices.net ([127.0.0.1]) by localhost (vbnemlrel7.vbnservices.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cLdFrL+tocgs for ; Thu, 3 Nov 2011 16:51:39 +0200 (CAT) Received: from virtualtest.com (unknown [83.143.30.30]) by vbnemlrel7.vbnservices.net (Postfix) with ESMTP id D5EAD2AF28 for ; Thu, 3 Nov 2011 16:51:31 +0200 (CAT) X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=127.0.0.1; Received: from bbmotors.co.bw (unverified [127.0.0.1]) by vbn-pop-10 (SurgeMail 5.3h2) with ESMTP id 250312-1872809 for multiple; Thu, 03 Nov 2011 16:04:06 +0200 To: (Recipient List Suppressed) Received: from 65.49.14.81 by HTTP Sender: dawood@bbmotors.co.bw From: "System Administrator" Reply-To: dawood@bbmotors.co.bw X-ASG-Orig-Subj: System Administrator Subject: System Administrator X-Mailer: Quality Web Email v3.1t X-Originating-IP: 65.49.14.81 Date: Thu, 03 Nov 2011 14:04:06 GMT Priority: normal Message-id: <4eb29f56.8c.1e4.7421@bbmotors.co.bw> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-=_vbn-nwk-monitor4eb29f56" X-To-Not-Matched: true X-Authenticated-User: dawood@bbmotors.co.bw X-Barracuda-Connect: vbnemlrel7.vbnservices.net[83.143.24.55] X-Barracuda-Start-Time: 1320331145 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4909 1.0000 0.0000 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=HTML_MESSAGE, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79211 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 0.00 HTML_MESSAGE BODY: HTML included in message X-Virus-Status: Clean This is a multi-part message in MIME format. ---=_vbn-nwk-monitor4eb29f56 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit System Administrator Your Mailbox Has Exceeded It Storage Limit As Set By Your Administrator, And You Will Not Be Able To Receive New Mails Until You Re-Validate It. To Re-Validate - > Click Here: System Administrator. ---=_vbn-nwk-monitor4eb29f56 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: 7bit

System Administrator
Your Mailbox Has Exceeded It Storage Limit As Set By Your Administrator, And You Will Not Be Able To Receive New Mails Until You Re-Validate It. To Re-Validate  - > Click Here: System Administrator.

---=_vbn-nwk-monitor4eb29f56-- From hch@lst.de Thu Nov 3 09:55:51 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3EtpRF020594 for ; Thu, 3 Nov 2011 09:55:51 -0500 X-ASG-Debug-ID: 1320332149-1fce003e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from newverein.lst.de (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9136412FF996 for ; Thu, 3 Nov 2011 07:55:50 -0700 (PDT) Received: from newverein.lst.de (verein.lst.de [213.95.11.211]) by cuda.sgi.com with ESMTP id Ip3qNYMmtvCUO0IK for ; Thu, 03 Nov 2011 07:55:50 -0700 (PDT) Received: by newverein.lst.de (Postfix, from userid 2407) id 1F29713E7F; Thu, 3 Nov 2011 15:55:49 +0100 (CET) Date: Thu, 3 Nov 2011 15:55:49 +0100 From: Christoph Hellwig To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 00/12] Bunch of new stress tests -v4 Subject: Re: [PATCH 00/12] Bunch of new stress tests -v4 Message-ID: <20111103145549.GA8612@lst.de> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.17 (2007-11-01) X-Barracuda-Connect: verein.lst.de[213.95.11.211] X-Barracuda-Start-Time: 1320332150 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79212 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 06:24:49PM +0400, Dmitry Monakhov wrote: > Changes from v3 ( in request to Christoph's comments) > - hard code fsstress's options for explicit regression tests > - add two new regression tests I already have commited your first three patches to the kernel.org repository and various other tests that have used the nubers you're using. If you want to get it in ASAP please resend, otherwise I will do it once I get a bit time. From rjevskiy@gmail.com Thu Nov 3 10:21:28 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3FLRvQ022923 for ; Thu, 3 Nov 2011 10:21:27 -0500 X-ASG-Debug-ID: 1320333685-6cac03200000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C3F991012289 for ; Thu, 3 Nov 2011 08:21:26 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id xPEIuhUsPZuteXyq for ; Thu, 03 Nov 2011 08:21:26 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1464049bkb.26 for ; Thu, 03 Nov 2011 08:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=9mv9YMU1dz9YpX0p1haNXA5A/kb7gEXF6YHl3yLkgvE=; b=k8TbA2foJkE/T6syYERIUGmt6bv9t3QJPhWY3rLpzbbqTGJSC6dkMr3cTetfi9jFK8 /CYgReUPhZ3GwJHauzqxgv6lC9NO69H7cKcK/hadK19VDyIGq17Z/scIcwSEC/g8s3ry chZCqpb7i9N21rzqOtcwhy7H5CJONrLtMfFkw= Received: by 10.204.135.195 with SMTP id o3mr8676402bkt.80.1320333685142; Thu, 03 Nov 2011 08:21:25 -0700 (PDT) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id zv16sm5918107bkb.6.2011.11.03.08.21.23 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 08:21:23 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Christoph Hellwig Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 00/12] Bunch of new stress tests -v4 Subject: Re: [PATCH 00/12] Bunch of new stress tests -v4 In-Reply-To: <20111103145549.GA8612@lst.de> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> <20111103145549.GA8612@lst.de> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Thu, 03 Nov 2011 19:21:21 +0400 Message-ID: <87obwtxmdq.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320333686 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0006 1.0000 -2.0172 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79214 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, 3 Nov 2011 15:55:49 +0100, Christoph Hellwig wrote: > On Thu, Nov 03, 2011 at 06:24:49PM +0400, Dmitry Monakhov wrote: > > Changes from v3 ( in request to Christoph's comments) > > - hard code fsstress's options for explicit regression tests > > - add two new regression tests > > I already have commited your first three patches to the kernel.org > repository and various other tests that have used the nubers you're > using. If you want to get it in ASAP please resend, otherwise I will > do it once I get a bit time. Oh.. yes kernel.org, i have just forgot to switch my tree to that repo again. No problem, Will rebase, test, and resubmit. > > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From tytso@thunk.org Thu Nov 3 11:06:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3G6A0D027296 for ; Thu, 3 Nov 2011 11:06:12 -0500 X-ASG-Debug-ID: 1320336369-493002e10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from test.thunk.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A178821AB9C; Thu, 3 Nov 2011 09:06:09 -0700 (PDT) Received: from test.thunk.org (li9-11.members.linode.com [67.18.176.11]) by cuda.sgi.com with ESMTP id mxL4M7Fb1Whz1HJB; Thu, 03 Nov 2011 09:06:09 -0700 (PDT) Received: from root (helo=tytso-glaptop.cam.corp.google.com) by test.thunk.org with local-esmtp (Exim 4.69) (envelope-from ) id 1RLznj-0005Uz-VM; Thu, 03 Nov 2011 16:06:00 +0000 Received: from tytso by tytso-glaptop.cam.corp.google.com with local (Exim 4.71) (envelope-from ) id 1RLznj-0004bX-N0; Thu, 03 Nov 2011 12:05:59 -0400 Date: Thu, 3 Nov 2011 12:05:59 -0400 From: "Ted Ts'o" To: Dmitry Monakhov Cc: Christoph Hellwig , linux-fsdevel@vger.kernel.org, aelder@sgi.com, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Subject: Re: [PATCH 6/8] xfstests: add fiemap operation to fsstress Message-ID: <20111103160559.GB15045@thunk.org> References: <1319849297-3506-1-git-send-email-dmonakhov@openvz.org> <1319849297-3506-7-git-send-email-dmonakhov@openvz.org> <20111102195534.GB22500@infradead.org> <87wrbhxya6.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87wrbhxya6.fsf@dmbot.sw.ru> User-Agent: Mutt/1.5.20 (2009-06-14) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: tytso@thunk.org X-SA-Exim-Scanned: No (on test.thunk.org); SAEximRunCond expanded to false X-Barracuda-Connect: li9-11.members.linode.com[67.18.176.11] X-Barracuda-Start-Time: 1320336369 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79217 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 03:04:17PM +0400, Dmitry Monakhov wrote: > On Thu, 3 Nov 2011 06:54:16 -0400, Theodore Tso wrote: > > > > On Nov 2, 2011, at 3:55 PM, Christoph Hellwig wrote: > > > > > Alex, Eric, Dave - should we add new tests with the new operations > > > Dmitry added, or is adding new ops to the existing tests fine? > > > > One argument for adding new ops to existing tests is that it makes > > the run time of the entire test suite take longer. A QA pass is > > already taking quite a while, and it would be nice if we could > > keep xfstests as efficient as possible in terms of the maximum > > testing coverage per time spent running the test suite…. > > Yes, but regression test with explicit seed option should be > preserved. Number of such test is not too big, so it is reasonable to > hardcode set of operations in such tests and let all others use new features. That's not what I was talking about. Of course there should be a way to run a regression test with an explicit seed option (although in general I think a specific test in xfstests should by default use a random seed, and have a way to easily specify an explicit seed without having to reverse engineer the test and running fsstress manually). What I was talking about was the fact we already have several (half a dozen or so, if memory serves correctly) xfstests that use fsstress with a different set of fsstress options. In some cases it makes to add a new numbered xfstest subtest, but I'd rather not find that we've doubled the number of tests using fsstress in the future, and with it, doubled the run-time of the auto or quick xfstests group.... - Ted From dmonlist@gmail.com Thu Nov 3 12:56:02 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_43, J_CHICKENPOX_64,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Hu1bd038333 for ; Thu, 3 Nov 2011 12:56:02 -0500 X-ASG-Debug-ID: 1320342960-509000910000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1DA59568D6E for ; Thu, 3 Nov 2011 10:56:00 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id eCUJazPM2pAHfynV for ; Thu, 03 Nov 2011 10:56:00 -0700 (PDT) Received: by bkbzs8 with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:55:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer; bh=pjWOr+m3kLr7EhM5fyT0SdeJAexvb3tk4o8bVf4lctk=; b=srq5QoqLEOVcScaYO4ogN+xc/yTG2X25yaAunBKCIRWsffVeUYibKyqrG4esARfkV+ TsHvAQMP++bAuNYXc/hPQEW5LZN8bnuTM9PZS50aCKJ2OpCoLhjITd6iy3xoQ5Lm7Bmi Bls7Hrox58aeSRUf3QqtZB2WRf3ykKcoybCCc= Received: by 10.204.34.148 with SMTP id l20mr9252622bkd.95.1320342959594; Thu, 03 Nov 2011 10:55:59 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.55.56 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:55:58 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 1/9] xfstests: fsstress add command line style output for show_opts Subject: [PATCH 1/9] xfstests: fsstress add command line style output for show_opts Date: Thu, 3 Nov 2011 21:55:45 +0400 Message-Id: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342961 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Once some combination of seed+fs_ops result in regression it is reasonable to document that combination. It is usefull to dump that configuration in command line style. Later this line may be simply hardcoded in to regression test. Signed-off-by: Dmitry Monakhov --- 070 | 2 +- 117 | 2 +- ltp/fsstress.c | 21 +++++++++++++++++---- 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/070 b/070 index 4458011..f48c33c 100755 --- a/070 +++ b/070 @@ -62,7 +62,7 @@ $FSSTRESS_PROG \ -f unresvsp=0 \ -f attr_set=100 \ -f attr_remove=100 \ - -S -p 1 -n 10000 >$seq.full 2>&1 + -p 1 -n 10000 -S c >$seq.full 2>&1 status=$? exit diff --git a/117 b/117 index fcf0485..edec157 100755 --- a/117 +++ b/117 @@ -85,7 +85,7 @@ while [ $i -lt $ITERATIONS ]; do -f attr_set=100 \ -f attr_remove=100 \ -s $seed \ - -S -p 1 -n 1000 >>$seq.full 2>&1 + -p 1 -n 1000 -S c >>$seq.full 2>&1 let i=$i+1 done diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 133a247..f9d7d6e 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -284,7 +284,7 @@ int main(int argc, char **argv) nops = sizeof(ops) / sizeof(ops[0]); ops_end = &ops[nops]; myprog = argv[0]; - while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:vwzHS")) != -1) { + while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:S:vwzH")) != -1) { switch (c) { case 'd': dirname = optarg; @@ -345,7 +345,10 @@ int main(int argc, char **argv) zero_freq(); break; case 'S': - show_ops(0, NULL); + i = 0; + if (optarg[0] == 'c') + i = 1; + show_ops(1, NULL); printf("\n"); nousage=1; break; @@ -1310,7 +1313,8 @@ show_ops(int flag, char *lead_str) x+=printf("%s ", p->name); } printf("\n"); - } else { + } else if (flag == 0) { + /* Table view style */ int f; for (f = 0, p = ops; p < ops_end; p++) f += p->freq; @@ -1327,6 +1331,15 @@ show_ops(int flag, char *lead_str) (p->iswrite == 0) ? " " : "write op"); } } + } else { + /* Command line style */ + if (lead_str != NULL) + printf("%s", lead_str); + printf ("-z -s %ld -m %d -n %d -p %d \\\n", seed, idmodulo, + operations, nproc); + for (p = ops; p < ops_end; p++) + if (p->freq > 0) + printf("-f %s=%d \\\n",p->name, p->freq); } } @@ -1433,7 +1446,7 @@ usage(void) printf(" -v specifies verbose mode\n"); printf(" -w zeros frequencies of non-write operations\n"); printf(" -z zeros frequencies of all operations\n"); - printf(" -S prints the table of operations (omitting zero frequency)\n"); + printf(" -S [c,t] prints the list of operations (omitting zero frequency) in command line or table style\n"); printf(" -H prints usage and exits\n"); } -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:04 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Hu4pr038343 for ; Thu, 3 Nov 2011 12:56:04 -0500 X-ASG-Debug-ID: 1320342960-509000910001-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A3231568D72 for ; Thu, 3 Nov 2011 10:56:03 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id tgmcLBWIUZKQeK94 for ; Thu, 03 Nov 2011 10:56:03 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=U3+dMW0fjq5f1ucH1GyO6/mhttD+2jzYiQzZdxNemvo=; b=B36QTIuuKJi3+4xSpns/U+xs7wAUjv0PzFqGXGMdlKHIPNYinoK5jM2XOBD6JrEz8V X4KeaTp3YM9fc2yAC6tsL2j8ea0elDuae1KLHkiNHCR7jPqCZ8CW3RTLyFWe2ztBye0n irpzx3xHe8P4Z2KxN+hgsDJiClZhjMBikZ5nM= Received: by 10.204.141.211 with SMTP id n19mr9340597bku.71.1320342962567; Thu, 03 Nov 2011 10:56:02 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.55.59 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:01 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 2/9] xfstests: freeze fsstress options for 117'th Subject: [PATCH 2/9] xfstests: freeze fsstress options for 117'th Date: Thu, 3 Nov 2011 21:55:46 +0400 Message-Id: <1320342953-7973-2-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342963 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is regression tescase for explicit combination of seed+fs_opts. Let's hardcode all options to prevent interactions with fsstress changes in future. Signed-off-by: Dmitry Monakhov --- 117 | 39 ++++++++++++++++++++++++++++----------- 1 files changed, 28 insertions(+), 11 deletions(-) diff --git a/117 b/117 index edec157..2e3f658 100755 --- a/117 +++ b/117 @@ -33,8 +33,34 @@ tmp=/tmp/$$ status=1 # failure is the default! trap "_cleanup; exit \$status" 0 1 2 3 15 -# seed for fsstress known to cause curruptions +# seed and options for fsstress known to cause curruptions seed=1122413794 +fss_ops=" -z \ + -f attr_remove=100 \ + -f attr_set=100 \ + -f chown=3 \ + -f creat=4 \ + -f dread=4 \ + -f dwrite=4 \ + -f fdatasync=1 \ + -f fsync=1 \ + -f getdents=1 \ + -f link=1 \ + -f mkdir=2 \ + -f mknod=2 \ + -f read=1 \ + -f readlink=1 \ + -f rename=2 \ + -f rmdir=1 \ + -f setxattr=1 \ + -f stat=1 \ + -f symlink=2 \ + -f sync=1 \ + -f truncate=2 \ + -f unlink=1 \ + -f write=4 \ + -s $seed \ + -m 31 -n 1000 -p 1" ITERATIONS=10 @@ -76,16 +102,7 @@ while [ $i -lt $ITERATIONS ]; do echo fsstress iteration: $i | tee -a $seq.full $FSSTRESS_PROG \ -d $SCRATCH_MNT/fsstress \ - -f allocsp=0 \ - -f freesp=0 \ - -f bulkstat=0 \ - -f bulkstat1=0 \ - -f resvsp=0 \ - -f unresvsp=0 \ - -f attr_set=100 \ - -f attr_remove=100 \ - -s $seed \ - -p 1 -n 1000 -S c >>$seq.full 2>&1 + $fss_ops -S c >>$seq.full 2>&1 let i=$i+1 done -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:07 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_43,J_CHICKENPOX_92,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3Hu7NN038363 for ; Thu, 3 Nov 2011 12:56:07 -0500 X-ASG-Debug-ID: 1320342960-509000910002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B69C5568D7F for ; Thu, 3 Nov 2011 10:56:06 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id 6jjBUPOWHGqVdVE5 for ; Thu, 03 Nov 2011 10:56:06 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=3snxcA4KpV88AgzwQBsytuJWHT0gYGAQ2XOrx9z7ToI=; b=k1XbWtkk+JBd95x2pkEaN9JGIM5fTRQwokAGxjFEKC1ovLHh/IAoQD3rob9PnFm+W2 yYz4Rjr1h1WQrBB2rz19qzODgcYjIHBpuDYrwxxRuzP9p/lrfwuK8WQnG6r8q9VrgFqF xMpuOq5eJVZi1J7sbchhjIrVT6LU0DMozFV70= Received: by 10.204.15.80 with SMTP id j16mr9439529bka.26.1320342965847; Thu, 03 Nov 2011 10:56:05 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.02 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:04 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 3/9] xfstests: add fallocate support to fsstress Subject: [PATCH 3/9] xfstests: add fallocate support to fsstress Date: Thu, 3 Nov 2011 21:55:47 +0400 Message-Id: <1320342953-7973-3-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342966 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Add tests for fallocate(2) syscall - fallocate: reserve the disk space - punch: de-allocates the disk space Since FALLOC_FL_PUNCH_HOLE is relatively new it's value defined explicitly if not yet defined. Later we may clear that define. Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 126 insertions(+), 1 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index f9d7d6e..e9e4e67 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -24,7 +24,13 @@ #ifdef HAVE_ATTR_ATTRIBUTES_H #include #endif - +#ifdef FALLOCATE +#include +#ifndef FALLOC_FL_PUNCH_HOLE +/* Copy-paste from linux/falloc.h */ +#define FALLOC_FL_PUNCH_HOLE 0x02 /* de-allocates range */ +#endif +#endif #ifndef HAVE_ATTR_LIST #define attr_list(path, buf, size, flags, cursor) (errno = -ENOSYS, -1) #endif @@ -48,6 +54,7 @@ typedef enum { OP_CREAT, OP_DREAD, OP_DWRITE, + OP_FALLOCATE, OP_FDATASYNC, OP_FREESP, OP_FSYNC, @@ -55,6 +62,7 @@ typedef enum { OP_LINK, OP_MKDIR, OP_MKNOD, + OP_PUNCH, OP_READ, OP_READLINK, OP_RENAME, @@ -128,6 +136,7 @@ void chown_f(int, long); void creat_f(int, long); void dread_f(int, long); void dwrite_f(int, long); +void fallocate_f(int, long); void fdatasync_f(int, long); void freesp_f(int, long); void fsync_f(int, long); @@ -135,6 +144,7 @@ void getdents_f(int, long); void link_f(int, long); void mkdir_f(int, long); void mknod_f(int, long); +void punch_f(int, long); void read_f(int, long); void readlink_f(int, long); void rename_f(int, long); @@ -159,6 +169,7 @@ opdesc_t ops[] = { { OP_CREAT, "creat", creat_f, 4, 1 }, { OP_DREAD, "dread", dread_f, 4, 0 }, { OP_DWRITE, "dwrite", dwrite_f, 4, 1 }, + { OP_FALLOCATE, "fallocate", fallocate_f, 1, 1 }, { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, @@ -166,6 +177,7 @@ opdesc_t ops[] = { { OP_LINK, "link", link_f, 1, 1 }, { OP_MKDIR, "mkdir", mkdir_f, 2, 1 }, { OP_MKNOD, "mknod", mknod_f, 2, 1 }, + { OP_PUNCH, "punch", punch_f, 1, 1 }, { OP_READ, "read", read_f, 1, 0 }, { OP_READLINK, "readlink", readlink_f, 1, 0 }, { OP_RENAME, "rename", rename_f, 2, 1 }, @@ -2011,6 +2023,63 @@ dwrite_f(int opno, long r) } void +fallocate_f(int opno, long r) +{ +#ifdef FALLOCATE + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + off64_t len; + struct stat64 stb; + int v; + char st[1024]; + int mode = 0; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: fallocate - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: fallocate - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: fallocate - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + len = (off64_t)(random() % (1024 * 1024)); + mode |= FALLOC_FL_KEEP_SIZE & random(); + e = fallocate(fd, mode, (loff_t)off, (loff_t)len) < 0 ? errno : 0; + if (v) + printf("%d/%d: fallocate(%d) %s %st %lld %lld %d\n", + procid, opno, mode, + f.path, st, (long long)off, (long long)len, e); + free_pathname(&f); + close(fd); +#endif +} + + +void fdatasync_f(int opno, long r) { int e; @@ -2283,6 +2352,62 @@ mknod_f(int opno, long r) } void +punch_f(int opno, long r) +{ +#ifdef FALLOCATE + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + off64_t len; + struct stat64 stb; + int v; + char st[1024]; + int mode = FALLOC_FL_PUNCH_HOLE; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: punch hole - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: punch hole - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: punch hole - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + len = (off64_t)(random() % (1024 * 1024)); + mode |= FALLOC_FL_KEEP_SIZE & random(); + e = fallocate(fd, mode, (loff_t)off, (loff_t)len) < 0 ? errno : 0; + if (v) + printf("%d/%d: punch hole(%d) %s %s %lld %lld %d\n", + procid, opno, mode, + f.path, st, (long long)off, (long long)len, e); + free_pathname(&f); + close(fd); +#endif +} + +void read_f(int opno, long r) { char *buf; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_43,J_CHICKENPOX_52,J_CHICKENPOX_64,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuBxV038379 for ; Thu, 3 Nov 2011 12:56:11 -0500 X-ASG-Debug-ID: 1320342960-509000910003-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 251B8568D87 for ; Thu, 3 Nov 2011 10:56:09 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id 8YGXR8ZsUCAgUvbp for ; Thu, 03 Nov 2011 10:56:09 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=Bxw2Bw//7KiM6jm0om3vEr9MtztgrOMPkIyWAMFT/ao=; b=ZurcC2taFlmlbZODeY+5HUdYOBxJk8iIh4SFfllrr1po/RzYNl066996hwE+YiMC9l vN0lBIyRC3ahFpqcyuxm07G+lWVVaSwv3CiewhPnEO9AR1QUrKTf+wPeuji8jC+K/1pt 8uwIXh51QE363qEhdpbBU2fjDxPSUsUkxl6HY= Received: by 10.204.133.194 with SMTP id g2mr9448508bkt.46.1320342968931; Thu, 03 Nov 2011 10:56:08 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.06 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:08 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 4/9] xfstests: fsstress add FS_IOC_{SET,GET}FLAGS operations v2 Subject: [PATCH 4/9] xfstests: fsstress add FS_IOC_{SET,GET}FLAGS operations v2 Date: Thu, 3 Nov 2011 21:55:48 +0400 Message-Id: <1320342953-7973-4-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342970 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Add two new operations: - getattr: ioctl(fd, FS_IOC_GETFLAGS, &fl) - setattr: ioctl(fd, FS_IOC_SETFLAGS, &random_flags) Attribute mask may be passed via -M opt, by default is (~0). By default FS_IOC_SETFLAGS has zero probability because it may produce inodes with APPEND or IMMUTABLE flags which are not deletable by default. Let's assumes that one who enable it knows how to delete such inodes. For example like follows: find $TEST_PATH -exec chattr -i -a {} \; rm -rf $TEST_PATH Signed-off-by: Dmitry Monakhov --- aclocal.m4 | 4 +++ configure.in | 1 + ltp/fsstress.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 72 insertions(+), 1 deletions(-) diff --git a/aclocal.m4 b/aclocal.m4 index 5532606..5739004 100644 --- a/aclocal.m4 +++ b/aclocal.m4 @@ -21,6 +21,10 @@ AC_DEFUN([AC_PACKAGE_WANT_LINUX_PRCTL_H], AC_SUBST(have_prctl) ]) +AC_DEFUN([AC_PACKAGE_WANT_LINUX_FS_H], + [ AC_CHECK_HEADER([linux/fs.h]) + ]) + AC_DEFUN([AC_PACKAGE_WANT_FALLOCATE], [ AC_MSG_CHECKING([for fallocate]) AC_TRY_LINK([ diff --git a/configure.in b/configure.in index 76d23e4..3b40e55 100644 --- a/configure.in +++ b/configure.in @@ -68,6 +68,7 @@ in AC_PACKAGE_WANT_LINUX_FIEMAP_H AC_PACKAGE_WANT_FALLOCATE AC_PACKAGE_WANT_LINUX_PRCTL_H + AC_PACKAGE_WANT_LINUX_FS_H ;; esac diff --git a/ltp/fsstress.c b/ltp/fsstress.c index e9e4e67..c258ee9 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -37,6 +37,15 @@ #ifdef HAVE_SYS_PRCTL_H #include #endif + +#include +#ifndef FS_IOC_GETFLAGS +#define FS_IOC_GETFLAGS _IOR('f', 1, long) +#endif +#ifndef FS_IOC_SETFLAGS +#define FS_IOC_SETFLAGS _IOW('f', 2, long) +#endif + #include #define XFS_ERRTAG_MAX 17 #define XFS_IDMODULO_MAX 31 /* user/group IDs (1 << x) */ @@ -58,6 +67,7 @@ typedef enum { OP_FDATASYNC, OP_FREESP, OP_FSYNC, + OP_GETATTR, OP_GETDENTS, OP_LINK, OP_MKDIR, @@ -68,6 +78,7 @@ typedef enum { OP_RENAME, OP_RESVSP, OP_RMDIR, + OP_SETATTR, OP_SETXATTR, OP_STAT, OP_SYMLINK, @@ -140,6 +151,7 @@ void fallocate_f(int, long); void fdatasync_f(int, long); void freesp_f(int, long); void fsync_f(int, long); +void getattr_f(int, long); void getdents_f(int, long); void link_f(int, long); void mkdir_f(int, long); @@ -150,6 +162,7 @@ void readlink_f(int, long); void rename_f(int, long); void resvsp_f(int, long); void rmdir_f(int, long); +void setattr_f(int, long); void setxattr_f(int, long); void stat_f(int, long); void symlink_f(int, long); @@ -173,6 +186,7 @@ opdesc_t ops[] = { { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, + { OP_GETATTR, "getattr", getattr_f, 1, 0 }, { OP_GETDENTS, "getdents", getdents_f, 1, 0 }, { OP_LINK, "link", link_f, 1, 1 }, { OP_MKDIR, "mkdir", mkdir_f, 2, 1 }, @@ -183,6 +197,7 @@ opdesc_t ops[] = { { OP_RENAME, "rename", rename_f, 2, 1 }, { OP_RESVSP, "resvsp", resvsp_f, 1, 1 }, { OP_RMDIR, "rmdir", rmdir_f, 1, 1 }, + { OP_SETATTR, "setattr", setattr_f, 0, 1 }, { OP_SETXATTR, "setxattr", setxattr_f, 1, 1 }, { OP_STAT, "stat", stat_f, 1, 0 }, { OP_SYMLINK, "symlink", symlink_f, 2, 1 }, @@ -218,6 +233,7 @@ int nops; int nproc = 1; int operations = 1; unsigned int idmodulo = XFS_IDMODULO_MAX; +unsigned int attr_mask = ~0; int procid; int rtpct; unsigned long seed = 0; @@ -296,7 +312,7 @@ int main(int argc, char **argv) nops = sizeof(ops) / sizeof(ops[0]); ops_end = &ops[nops]; myprog = argv[0]; - while ((c = getopt(argc, argv, "d:e:f:i:m:n:o:p:rs:S:vwzH")) != -1) { + while ((c = getopt(argc, argv, "d:e:f:i:m:M:n:o:p:rs:S:vwzH")) != -1) { switch (c) { case 'd': dirname = optarg; @@ -356,6 +372,9 @@ int main(int argc, char **argv) case 'z': zero_freq(); break; + case 'M': + attr_mask = strtoul(optarg, NULL, 0); + break; case 'S': i = 0; if (optarg[0] == 'c') @@ -2198,6 +2217,29 @@ fsync_f(int opno, long r) } void +getattr_f(int opno, long r) +{ + int fd; + int e; + pathname_t f; + uint fl; + int v; + + init_pathname(&f); + if (!get_fname(FT_ANYm, r, &f, NULL, NULL, &v)) + append_pathname(&f, "."); + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + + e = ioctl(fd, FS_IOC_GETFLAGS, &fl); + if (v) + printf("%d/%d: getattr %s %u %d\n", procid, opno, f.path, fl, e); + free_pathname(&f); + close(fd); +} + +void getdents_f(int opno, long r) { DIR *dir; @@ -2643,6 +2685,30 @@ rmdir_f(int opno, long r) } void +setattr_f(int opno, long r) +{ + int fd; + int e; + pathname_t f; + uint fl; + int v; + + init_pathname(&f); + if (!get_fname(FT_ANYm, r, &f, NULL, NULL, &v)) + append_pathname(&f, "."); + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + + fl = attr_mask & (uint)random(); + e = ioctl(fd, FS_IOC_SETFLAGS, &fl); + if (v) + printf("%d/%d: setattr %s %x %d\n", procid, opno, f.path, fl, e); + free_pathname(&f); + close(fd); +} + +void stat_f(int opno, long r) { int e; -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,J_CHICKENPOX_14, J_CHICKENPOX_52,J_CHICKENPOX_66,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuEkA038397 for ; Thu, 3 Nov 2011 12:56:14 -0500 X-ASG-Debug-ID: 1320342960-509000910004-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5048A568D91 for ; Thu, 3 Nov 2011 10:56:12 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id mmtb3zU0Y9jtUI77 for ; Thu, 03 Nov 2011 10:56:12 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=zL/s+DVlZAoOrxkEtWfKSXSLWDzQz6CO5BG264xcA3w=; b=Hkdrm/9noZ1PtPiN2g+OtkM9ITEYQKPybwEuxwM0a7ovaLUHQNeoZ4VOvUoWxItMQL AZecv+i0b2IEebXND2XNJ3HJiANJed+ovhlCrRqopIdiiYyJpxP16Z2zAf/eptzRx+v9 FKH/iQPYyZot4PwShKt4rrcdJ1SZJ5egj1yiY= Received: by 10.204.139.84 with SMTP id d20mr9658942bku.13.1320342971866; Thu, 03 Nov 2011 10:56:11 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.09 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:11 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 5/9] xfstests: add fiemap operation to fsstress Subject: [PATCH 5/9] xfstests: add fiemap operation to fsstress Date: Thu, 3 Nov 2011 21:55:49 +0400 Message-Id: <1320342953-7973-5-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342973 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Related bug: http://patchwork.ozlabs.org/patch/118863 Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 76 insertions(+), 0 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index c258ee9..09c9529 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -24,6 +24,9 @@ #ifdef HAVE_ATTR_ATTRIBUTES_H #include #endif +#ifdef HAVE_LINUX_FIEMAP_H +#include +#endif #ifdef FALLOCATE #include #ifndef FALLOC_FL_PUNCH_HOLE @@ -65,6 +68,7 @@ typedef enum { OP_DWRITE, OP_FALLOCATE, OP_FDATASYNC, + OP_FIEMAP, OP_FREESP, OP_FSYNC, OP_GETATTR, @@ -149,6 +153,7 @@ void dread_f(int, long); void dwrite_f(int, long); void fallocate_f(int, long); void fdatasync_f(int, long); +void fiemap_f(int, long); void freesp_f(int, long); void fsync_f(int, long); void getattr_f(int, long); @@ -184,6 +189,7 @@ opdesc_t ops[] = { { OP_DWRITE, "dwrite", dwrite_f, 4, 1 }, { OP_FALLOCATE, "fallocate", fallocate_f, 1, 1 }, { OP_FDATASYNC, "fdatasync", fdatasync_f, 1, 1 }, + { OP_FIEMAP, "fiemap", fiemap_f, 1, 1 }, { OP_FREESP, "freesp", freesp_f, 1, 1 }, { OP_FSYNC, "fsync", fsync_f, 1, 1 }, { OP_GETATTR, "getattr", getattr_f, 1, 0 }, @@ -2130,6 +2136,76 @@ fdatasync_f(int opno, long r) free_pathname(&f); close(fd); } +void +fiemap_f(int opno, long r) +{ +#ifdef HAVE_LINUX_FIEMAP_H + int e; + pathname_t f; + int fd; + __int64_t lr; + off64_t off; + struct stat64 stb; + int v; + char st[1024]; + int blocks_to_map; + struct fiemap *fiemap; + + init_pathname(&f); + if (!get_fname(FT_REGFILE, r, &f, NULL, NULL, &v)) { + if (v) + printf("%d/%d: fiemap - no filename\n", procid, opno); + free_pathname(&f); + return; + } + fd = open_path(&f, O_RDWR); + e = fd < 0 ? errno : 0; + check_cwd(); + if (fd < 0) { + if (v) + printf("%d/%d: fiemap - open %s failed %d\n", + procid, opno, f.path, e); + free_pathname(&f); + return; + } + if (fstat64(fd, &stb) < 0) { + if (v) + printf("%d/%d: fiemap - fstat64 %s failed %d\n", + procid, opno, f.path, errno); + free_pathname(&f); + close(fd); + return; + } + inode_info(st, sizeof(st), &stb, v); + blocks_to_map = random() & 0xffff; + fiemap = (struct fiemap *)malloc(sizeof(struct fiemap) + + (blocks_to_map * sizeof(struct fiemap_extent))); + if (!fiemap) { + if (v) + printf("%d/%d: malloc failed \n", procid, opno); + free_pathname(&f); + close(fd); + return; + } + lr = ((__int64_t)random() << 32) + random(); + off = (off64_t)(lr % MIN(stb.st_size + (1024 * 1024), MAXFSIZE)); + off %= maxfsize; + fiemap->fm_flags = random() & (FIEMAP_FLAGS_COMPAT | 0x10000); + fiemap->fm_extent_count = blocks_to_map; + fiemap->fm_mapped_extents = random() & 0xffff; + fiemap->fm_start = off; + fiemap->fm_length = ((__int64_t)random() << 32) + random(); + + e = ioctl(fd, FS_IOC_FIEMAP, (unsigned long)fiemap); + if (v) + printf("%d/%d: ioctl(FIEMAP) %s%s %lld %lld %x %d\n", + procid, opno, f.path, st, (long long)fiemap->fm_start, + (long long) fiemap->fm_length, fiemap->fm_flags, e); + free(fiemap); + free_pathname(&f); + close(fd); +#endif +} void freesp_f(int opno, long r) -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:16 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuGJr038413 for ; Thu, 3 Nov 2011 12:56:16 -0500 X-ASG-Debug-ID: 1320342960-509000910005-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DF928568D97 for ; Thu, 3 Nov 2011 10:56:15 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id k9Fvbj3m5I9AvMDW for ; Thu, 03 Nov 2011 10:56:15 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=QXa024DAMdyyBgOMPSaLctfhcQTCOnCcg3aT8ZQWIp0=; b=x7z9rOF0wn73TwstmVx4wn/2zlGX06W+JG0DOi+0V9U714z0iaRZH0/kNxDqshLk6i C4UVzYXkx/SPu2JG21sKK/3adoQZJkBk9pOaxEGeVR7GZyFNlcVF8teuML3A7cXQKqDT 6XcLWffaidlSkiXhmaTe8VwuRnNwooO4/hctk= Received: by 10.204.13.68 with SMTP id b4mr9263151bka.96.1320342975140; Thu, 03 Nov 2011 10:56:15 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.12 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:14 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Date: Thu, 3 Nov 2011 21:55:50 +0400 Message-Id: <1320342953-7973-6-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342975 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean During stress testing we want to cover as much code paths as possible fsstress is very good for this purpose. But it has expandable nature (disk usage almost continually grow). So once it goes in no ENOSPC condition it will be where till the end. But by running 'dd' writers in parallel we can regularly trigger ENOSPC but only for a limited periods of time because each time it opens the same file with O_TRUNC. Signed-off-by: Dmitry Monakhov --- 269 | 86 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 269.out | 5 +++ group | 1 + 3 files changed, 92 insertions(+), 0 deletions(-) create mode 100755 269 create mode 100644 269.out diff --git a/269 b/269 new file mode 100755 index 0000000..61e83ad --- /dev/null +++ b/269 @@ -0,0 +1,86 @@ +#! /bin/bash +# FSQA Test No. 269 +# +# Run fsstress and ENSPC hitters in parallel, check fs consistency an the end +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +# Disable all sync operations to get higher load +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" +_workout() +{ + echo "" + echo "Run fsstress" + echo "" + num_iterations=10 + enospc_time=2 + out=$SCRATCH_MNT/fsstress.$$ + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" + echo "fsstress $args" >> $here/$seq.full + $FSSTRESS_PROG $args > /dev/null 2>&1 & + pid=$! + echo "Run dd writers in parallel" + for ((i=0; i < num_iterations; i++)) + do + # File will be opened with O_TRUNC each time + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ + > /dev/null 2>&1 + sleep $enospc_time + done + kill $pid + wait $pid +} + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 +_scratch_mount + +if ! _workout; then + umount $SCRATCH_DEV 2>/dev/null + exit +fi + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +_check_scratch_fs +status=$? +exit diff --git a/269.out b/269.out new file mode 100644 index 0000000..0a83965 --- /dev/null +++ b/269.out @@ -0,0 +1,5 @@ +QA output created by 269 + +Run fsstress + +Run dd writers in parallel diff --git a/group b/group index aaa8eba..8a5c334 100644 --- a/group +++ b/group @@ -382,3 +382,4 @@ deprecated 266 dump ioctl auto quick 267 dump ioctl tape 268 dump ioctl tape +269 auto rw prealloc ioctl enospc -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:20 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuJVa038428 for ; Thu, 3 Nov 2011 12:56:20 -0500 X-ASG-Debug-ID: 1320342960-509000910006-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 20D54568D99 for ; Thu, 3 Nov 2011 10:56:18 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id sHvlfgnDaDcQ1dzP for ; Thu, 03 Nov 2011 10:56:18 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=giwsby1EZv0KCLO3BtHDVUIIDrv6rJhpoYS3vzJc9AM=; b=hNh+Gtj/rwXZyiXQGZ0qC+2h6AXMqfjRCTs9Vw/oMLhq/l/B+jtddNEcA1g87hXy2r b1azFSQT3n+GwGzllD3zmK+Ct4b8rKVrGptLHSfHs/i6XHv91dUgsNzL7gsKpjCJ/xZ+ HgEseSziqFSOwfPN0DK/tM8j7GmtNDrB+QqT4= Received: by 10.204.140.153 with SMTP id i25mr9180340bku.102.1320342978468; Thu, 03 Nov 2011 10:56:18 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.15 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:17 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Subject: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Date: Thu, 3 Nov 2011 21:55:51 +0400 Message-Id: <1320342953-7973-7-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342979 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean - Same as 264 but with quota enabled. - IO performed from $qa_user user - fsstress granted with CAP_CHOWN capability. Signed-off-by: Dmitry Monakhov --- 270 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 270.out | 8 +++++ group | 1 + 3 files changed, 117 insertions(+), 0 deletions(-) create mode 100755 270 create mode 100644 270.out diff --git a/270 b/270 new file mode 100755 index 0000000..ed1e31e --- /dev/null +++ b/270 @@ -0,0 +1,108 @@ +#! /bin/bash +# FSQA Test No. 270 +# +# Run fsstress and ENSPC hitters in parallel, check quota and +# fs consistency an the end +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.quota + +# Disable all sync operations to get higher load +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" +_workout() +{ + echo "" + echo "Run fsstress" + echo "" + num_iterations=10 + enospc_time=2 + out=$SCRATCH_MNT/fsstress.$$ + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" + echo "fsstress $args" >> $here/$seq.full + # Grant chown capability + cp $FSSTRESS_PROG $tmp.fsstress.bin + setcap cap_chown=epi $tmp.fsstress.bin + + (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 + pid=$! + + echo "Run dd writers in parallel" + for ((i=0; i < num_iterations; i++)) + do + # File will be opened with O_TRUNC each time + su $qa_user -c "dd if=/dev/zero \ + of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M " \ + > /dev/null 2>&1 + sleep $enospc_time + done + + killall $tmp.fsstress.bin + wait $pid +} + +# real QA test starts here +_supported_fs generic +_supported_os Linux +_require_scratch +_require_quota +_require_user +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 +_scratch_mount "-o usrquota,grpquota" +chmod 777 $SCRATCH_MNT +quotacheck -u -g $SCRATCH_MNT 2>/dev/null +quotaon -u -g $SCRATCH_MNT 2>/dev/null + +if ! _workout; then + _scratch_unmount 2>/dev/null + exit +fi + +if ! _check_quota_usage; then + _scratch_unmount 2>/dev/null + status=1 + exit +fi + +echo Comparing filesystem consistency +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +_check_scratch_fs +status=$? +exit diff --git a/270.out b/270.out new file mode 100644 index 0000000..bbd8026 --- /dev/null +++ b/270.out @@ -0,0 +1,8 @@ +QA output created by 270 + +Run fsstress + +Run dd writers in parallel +Comparing user usage +Comparing group usage +Comparing filesystem consistency diff --git a/group b/group index 8a5c334..84e81f5 100644 --- a/group +++ b/group @@ -383,3 +383,4 @@ deprecated 267 dump ioctl tape 268 dump ioctl tape 269 auto rw prealloc ioctl enospc +270 auto quota rw prealloc ioctl enospc -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuNqJ038442 for ; Thu, 3 Nov 2011 12:56:23 -0500 X-ASG-Debug-ID: 1320342960-509000910007-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5C5A6568DA0 for ; Thu, 3 Nov 2011 10:56:21 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id hqmvi016ID7DIkcH for ; Thu, 03 Nov 2011 10:56:21 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=AKy1qAOMjs1bRlrGqqUZFW/nZ0JD3tBcGJoaxibecto=; b=Z4Ubnoww8hhPodGFC3ModyVT08qi3BrHy3Y9QQ95ib1Q5iaxMFztdtXbjQ8enDk7ai gvOOuABR+HLsYpqT6rOoUHeBdmKpzPUhZPglJbzED7e3CJ9tB789jhnpJuCwUtKIKgoG 0T7enIjYnpKwiIpsyL7W6TgsBP9DuodwATYwY= Received: by 10.204.34.148 with SMTP id l20mr9253917bkd.95.1320342981712; Thu, 03 Nov 2011 10:56:21 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.18 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:20 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 8/9] xfstress: add regression testcase for d583fb87a3ff0 Subject: [PATCH 8/9] xfstress: add regression testcase for d583fb87a3ff0 Date: Thu, 3 Nov 2011 21:55:52 +0400 Message-Id: <1320342953-7973-8-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342982 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Should be fixed by following patch http://patchwork.ozlabs.org/patch/120883/ Signed-off-by: Dmitry Monakhov --- 271 | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 271.out | 44 ++++++++++++++++++++++++++++++++++++++++ group | 1 + 3 files changed, 114 insertions(+), 0 deletions(-) create mode 100755 271 create mode 100644 271.out diff --git a/271 b/271 new file mode 100755 index 0000000..b4e6043 --- /dev/null +++ b/271 @@ -0,0 +1,69 @@ +#! /bin/bash +# FSQA Test No. 271 +# +# Regression testcase for d583fb87a3ff0 (ext4 extent corruption) +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs ext4 +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((128 * 1024 * 1024)) >> $seq.full 2>&1 + +# -onoload and EXT4_SYNC_FL on file is important becase result in +# metadata sync writes inside ext4_handle_dirty_metadata() +_scratch_mount -onoload +touch $SCRATCH_MNT/file +chattr +S $SCRATCH_MNT/file +# Create sparse file +for ((i = 0; i < 21; i++)) +do + dd if=/dev/zero of=$SCRATCH_MNT/file bs=4k count=1 seek=$((i*10)) +done +# truncate last extent +truncate $SCRATCH_MNT/file --size $((4096* 200)) + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +echo "Check filesystem" +_check_scratch_fs +status=$? +exit diff --git a/271.out b/271.out new file mode 100644 index 0000000..22a3c50 --- /dev/null +++ b/271.out @@ -0,0 +1,44 @@ +QA output created by 271 +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +1+0 records in +1+0 records out +Check filesystem diff --git a/group b/group index 84e81f5..fefa580 100644 --- a/group +++ b/group @@ -384,3 +384,4 @@ deprecated 268 dump ioctl tape 269 auto rw prealloc ioctl enospc 270 auto quota rw prealloc ioctl enospc +271 auto rw quick -- 1.7.1 From dmonlist@gmail.com Thu Nov 3 12:56:26 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, J_CHICKENPOX_47,J_CHICKENPOX_56,J_CHICKENPOX_57,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3HuQv1038461 for ; Thu, 3 Nov 2011 12:56:26 -0500 X-ASG-Debug-ID: 1320342960-509000910008-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 73D93568DA7 for ; Thu, 3 Nov 2011 10:56:24 -0700 (PDT) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id hhiQgjdCTt0p9wAt for ; Thu, 03 Nov 2011 10:56:24 -0700 (PDT) Received: by mail-bw0-f53.google.com with SMTP id zs8so1634560bkb.26 for ; Thu, 03 Nov 2011 10:56:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=yq9gv2dVvB50CWoYDCh8B31XvS09A6T+hhDAJCseXtA=; b=ABso4xysn4r1L8ztfwr1ro/ZPPl6sn7vmi2dX32ZsUOZXTO93n354ZGadlJTamUuPA ls12LwHGI4nCnPBMkS19rJCdotzrzNhGzUk6dNMy8HDJQDexNgf3B2aRDiBKXp3asTgv 3nloF4VNJB3blulJLGL6bAyND9gLrSsp2ga6M= Received: by 10.204.140.153 with SMTP id i25mr9180705bku.102.1320342984732; Thu, 03 Nov 2011 10:56:24 -0700 (PDT) Received: from localhost.localdomain ([178.176.142.120]) by mx.google.com with ESMTPS id z9sm6365051bkn.7.2011.11.03.10.56.21 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 10:56:23 -0700 (PDT) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org Cc: xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 9/9] xfstress: Test data journaling flag switch for a single file Subject: [PATCH 9/9] xfstress: Test data journaling flag switch for a single file Date: Thu, 3 Nov 2011 21:55:53 +0400 Message-Id: <1320342953-7973-9-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320342985 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79225 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Signed-off-by: Dmitry Monakhov --- 272 | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 272.out | 37 +++++++++++++++++++++ group | 1 + 3 files changed, 146 insertions(+), 0 deletions(-) create mode 100755 272 create mode 100644 272.out diff --git a/272 b/272 new file mode 100755 index 0000000..a35ef83 --- /dev/null +++ b/272 @@ -0,0 +1,108 @@ +#! /bin/bash +# FSQA Test No. 272 +# +# Test data journaling flag switch for a single file +# +#----------------------------------------------------------------------- +# Copyright (c) 2006 Silicon Graphics, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=dmonakhov@openvz.org + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +_workout() +{ + echo "" + echo " Switch data journalling mode" + echo "" + write_opt_list="iflag=noatime conv=notrunc conv=fsync oflag=direct" + chattr_opt_list="+j -j +jS -j" + idx=0 + # + for write_opt in $write_opt_list + do + idx=$((idx + 1)) + for chattr_opt in $chattr_opt_list + do + + echo "OP write_opt: $write_opt 4M, \ +chattr_opt: $chattr_opt" + dd if=/dev/zero of=$SCRATCH_MNT/file.$idx \ + bs=1M count=4 $write_opt \ + >> $seq.full 2>&1 || exit + chattr $chattr_opt $SCRATCH_MNT/file.$idx >> $seq.full \ + || exit + done + done + sync + # Same as previous test, but for one file, + # and with ENOSPC triggering + for write_opt in $write_opt_list + do + idx=$((idx + 1)) + for chattr_opt in $chattr_opt_list + do + + echo "OP write_opt: $write_opt ENOSPC, \ +chattr_opt: $chattr_opt" + dd if=/dev/zero of=$SCRATCH_MNT/file.$idx \ + bs=1M $write_opt >> $seq.full 2>&1 + chattr $chattr_opt $SCRATCH_MNT/file.$idx \ + >> $seq.full || exit + done + sync + unlink $SCRATCH_MNT/file.$idx + done +} + +# real QA test starts here +_supported_fs ext3 ext4 +_supported_os Linux +_require_scratch +_need_to_be_root + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((64 * 1024 * 1024)) >> $seq.full 2>&1 +_scratch_mount + +if ! _workout; then + echo "workout failed" + _scratch_unmount + status=1 + exit +fi + +if ! _scratch_unmount; then + echo "failed to umount" + status=1 + exit +fi +echo "Check filesystem" +_check_scratch_fs +status=$? +exit diff --git a/272.out b/272.out new file mode 100644 index 0000000..cc687aa --- /dev/null +++ b/272.out @@ -0,0 +1,37 @@ +QA output created by 272 + + Switch data journalling mode + +OP write_opt: iflag=noatime 4M, chattr_opt: +j +OP write_opt: iflag=noatime 4M, chattr_opt: -j +OP write_opt: iflag=noatime 4M, chattr_opt: +jS +OP write_opt: iflag=noatime 4M, chattr_opt: -j +OP write_opt: conv=notrunc 4M, chattr_opt: +j +OP write_opt: conv=notrunc 4M, chattr_opt: -j +OP write_opt: conv=notrunc 4M, chattr_opt: +jS +OP write_opt: conv=notrunc 4M, chattr_opt: -j +OP write_opt: conv=fsync 4M, chattr_opt: +j +OP write_opt: conv=fsync 4M, chattr_opt: -j +OP write_opt: conv=fsync 4M, chattr_opt: +jS +OP write_opt: conv=fsync 4M, chattr_opt: -j +OP write_opt: oflag=direct 4M, chattr_opt: +j +OP write_opt: oflag=direct 4M, chattr_opt: -j +OP write_opt: oflag=direct 4M, chattr_opt: +jS +OP write_opt: oflag=direct 4M, chattr_opt: -j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: +j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: -j +OP write_opt: iflag=noatime ENOSPC, chattr_opt: +jS +OP write_opt: iflag=noatime ENOSPC, chattr_opt: -j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: +j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: -j +OP write_opt: conv=notrunc ENOSPC, chattr_opt: +jS +OP write_opt: conv=notrunc ENOSPC, chattr_opt: -j +OP write_opt: conv=fsync ENOSPC, chattr_opt: +j +OP write_opt: conv=fsync ENOSPC, chattr_opt: -j +OP write_opt: conv=fsync ENOSPC, chattr_opt: +jS +OP write_opt: conv=fsync ENOSPC, chattr_opt: -j +OP write_opt: oflag=direct ENOSPC, chattr_opt: +j +OP write_opt: oflag=direct ENOSPC, chattr_opt: -j +OP write_opt: oflag=direct ENOSPC, chattr_opt: +jS +OP write_opt: oflag=direct ENOSPC, chattr_opt: -j +Check filesystem diff --git a/group b/group index fefa580..616cbdd 100644 --- a/group +++ b/group @@ -385,3 +385,4 @@ deprecated 269 auto rw prealloc ioctl enospc 270 auto quota rw prealloc ioctl enospc 271 auto rw quick +272 auto enospc rw -- 1.7.1 From BATV+9df61a0795d9022ba6a7+2993+infradead.org+hch@bombadil.srs.infradead.org Thu Nov 3 14:39:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3JdhFQ048695 for ; Thu, 3 Nov 2011 14:39:43 -0500 X-ASG-Debug-ID: 1320349178-53e503470000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0D983125D86F; Thu, 3 Nov 2011 12:39:38 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id VdIATM2zR6QUDaMt; Thu, 03 Nov 2011 12:39:38 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RM38R-00069l-Rd; Thu, 03 Nov 2011 19:39:35 +0000 Date: Thu, 3 Nov 2011 15:39:35 -0400 From: Christoph Hellwig To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Message-ID: <20111103193935.GA23014@infradead.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> <1320342953-7973-7-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320342953-7973-7-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320349179 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79232 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 09:55:51PM +0400, Dmitry Monakhov wrote: > - Same as 264 but with quota enabled. 269 is the new 270 > - IO performed from $qa_user user > - fsstress granted with CAP_CHOWN capability. This fails in my test VM because it doesn't have setcap. If you want to use additional non-standard tools please do a _notrun if they aren't present. I don't quite understand why it would need CAP_CHOWN - chowns from owned files do not require it, and the test runs fine with the setcap line commented out. I'll commit it as-is for now, but unless I get some feedback on why the setcap is require I'll add another commit to remove it ASAP. From arekm@maven.pl Thu Nov 3 15:02:19 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3K2J4i050248 for ; Thu, 3 Nov 2011 15:02:19 -0500 X-ASG-Debug-ID: 1320350534-391900f10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 968BA10132C7 for ; Thu, 3 Nov 2011 13:02:14 -0700 (PDT) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id AIK9kkSAFuYGffhf for ; Thu, 03 Nov 2011 13:02:14 -0700 (PDT) Received: by faai28 with SMTP id i28so2057394faa.26 for ; Thu, 03 Nov 2011 13:02:14 -0700 (PDT) Received: by 10.223.65.83 with SMTP id h19mr910395fai.1.1320350534167; Thu, 03 Nov 2011 13:02:14 -0700 (PDT) Received: from t400.localnet (89-69-21-174.dynamic.chello.pl. [89.69.21.174]) by mx.google.com with ESMTPS id j5sm13769831faf.14.2011.11.03.13.02.13 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 13:02:13 -0700 (PDT) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: Christoph Hellwig X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Date: Thu, 3 Nov 2011 21:02:12 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-05616-gf362f98; KDE/4.7.3; x86_64; ; ) Cc: xfs@oss.sgi.com References: <201110311156.20421.arekm@maven.pl> <20111103110306.GA8382@infradead.org> <20111103115429.GA15726@infradead.org> In-Reply-To: <20111103115429.GA15726@infradead.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201111032102.12137.arekm@maven.pl> X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320350535 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0001 1.0000 -2.0205 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79232 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thursday 03 of November 2011, Christoph Hellwig wrote: > Can you give this patch a quick try on the image in that constrained > setup? Ate up all 8GB of ram and then was killed: [217802.127030] Out of memory: Kill process 22898:#0 (xfs_repair) score 784= or=20 sacrifice child [217802.127033] Killed process 22898:#0 (xfs_repair) total-vm:7156064kB, an= on- rss:6652784kB, file-rss:620kB Now the question is if 8GB should be enough for repairing 7TB filesystem? =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From arekm@maven.pl Thu Nov 3 15:21:22 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3KLMLH051543 for ; Thu, 3 Nov 2011 15:21:22 -0500 X-ASG-Debug-ID: 1320351680-4ae303a90000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9ADAE1CE098E for ; Thu, 3 Nov 2011 13:21:21 -0700 (PDT) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id EAYp2VpDBXsBBYLc for ; Thu, 03 Nov 2011 13:21:21 -0700 (PDT) Received: by faai28 with SMTP id i28so2074705faa.26 for ; Thu, 03 Nov 2011 13:21:20 -0700 (PDT) Received: by 10.223.76.217 with SMTP id d25mr18628449fak.31.1320351678672; Thu, 03 Nov 2011 13:21:18 -0700 (PDT) Received: from t400.localnet (89-69-21-174.dynamic.chello.pl. [89.69.21.174]) by mx.google.com with ESMTPS id o16sm13804186fag.21.2011.11.03.13.21.17 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 13:21:18 -0700 (PDT) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: Christoph Hellwig X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Date: Thu, 3 Nov 2011 21:21:16 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-05616-gf362f98; KDE/4.7.3; x86_64; ; ) Cc: xfs@oss.sgi.com References: <201110311156.20421.arekm@maven.pl> <201111031157.39179.arekm@maven.pl> <20111103110306.GA8382@infradead.org> In-Reply-To: <20111103110306.GA8382@infradead.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201111032121.16891.arekm@maven.pl> X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320351681 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0006 1.0000 -2.0173 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79235 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thursday 03 of November 2011, Christoph Hellwig wrote: > On Thu, Nov 03, 2011 at 11:57:38AM +0100, Arkadiusz Mi??kiewicz wrote: > > > In the meantime is there any chance you could send the output of > > >=20 > > > xfs_repair -n -vv -m 1 > > >=20 > > > for this filesystem? > >=20 > > Will such repair done on metadumped & restored image be enough for you? >=20 > Yes. >=20 > I also looked at the memory heuristics and found some fairly obvious > flaws. I'll have some test patches for you ASAP. # LC_ALL=3DC ./xfs_repair -vv -m 1 /dev/vg_sys/lv_storage1 Phase 1 - find and verify superblock... - max_mem =3D 1024, icount =3D 97495744, imem =3D 380842, dblock = =3D=20 1884752896, dmem =3D 920289 Required memory for repair is greater that the maximum specified with the -m option. Please increase it to at least 1319. =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From sekharan@us.ibm.com Thu Nov 3 16:44:54 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA3LiruA054867 for ; Thu, 3 Nov 2011 16:44:54 -0500 X-ASG-Debug-ID: 1320356692-371402010000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from e34.co.us.ibm.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7623821CFA1 for ; Thu, 3 Nov 2011 14:44:52 -0700 (PDT) Received: from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.152]) by cuda.sgi.com with ESMTP id k43q4M2vJGmK0SKp for ; Thu, 03 Nov 2011 14:44:52 -0700 (PDT) X-ASG-Whitelist: Barracuda Reputation Received: from /spool/local by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 3 Nov 2011 15:44:51 -0600 Received: from d03relay03.boulder.ibm.com ([9.17.195.228]) by e34.co.us.ibm.com ([192.168.1.134]) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 3 Nov 2011 15:43:54 -0600 Received: from d03av02.boulder.ibm.com (d03av02.boulder.ibm.com [9.17.195.168]) by d03relay03.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id pA3LhlTg109326 for ; Thu, 3 Nov 2011 15:43:49 -0600 Received: from d03av02.boulder.ibm.com (loopback [127.0.0.1]) by d03av02.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id pA3Lhl0p013535 for ; Thu, 3 Nov 2011 15:43:47 -0600 Received: from [9.53.40.164] (chandra-lucid.austin.ibm.com [9.53.40.164]) by d03av02.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id pA3LhlI5013496; Thu, 3 Nov 2011 15:43:47 -0600 X-ASG-Orig-Subj: Hang seen in xfstests 234 Subject: Hang seen in xfstests 234 From: Chandra Seetharaman Reply-To: sekharan@us.ibm.com To: Alex Elder Cc: XFS Mailing List Content-Type: text/plain; charset="UTF-8" Organization: IBM Date: Thu, 03 Nov 2011 16:43:46 -0500 Message-ID: <1320356626.17713.32.camel@chandra-lucid.austin.ibm.com> Mime-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: 7bit x-cbid: 11110321-1780-0000-0000-000000AD2094 X-Barracuda-Connect: e34.co.us.ibm.com[32.97.110.152] X-Barracuda-Start-Time: 1320356693 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi Alex, I ran the xfstests 234 with procs set to 20 and I can consistently make it hang. Looks like, the log buffer is full and all the processes are sleeping at xlog_wait() or blocked on a mutex which was held by a process that is sleeping on xlog_wait(). wake up happens when sync is invoked and all processes run free to completion. In order to verify the hypothesis, I added the following code ------------------------- diff --git a/fs/xfs/xfs_qm_syscalls.c b/fs/xfs/xfs_qm_syscalls.c index 5cc3dde..b2c0d49 100644 --- a/fs/xfs/xfs_qm_syscalls.c +++ b/fs/xfs/xfs_qm_syscalls.c @@ -621,6 +621,7 @@ xfs_qm_scall_setqlim( dqp->dq_flags |= XFS_DQ_DIRTY; xfs_trans_log_dquot(tp, dqp); + xfs_trans_set_sync(tp); error = xfs_trans_commit(tp, 0); xfs_qm_dqrele(dqp); -------------------------- and it seems to work well. Not sure if this can be a fix or the nudge has to happen somewhere else. Please comment. Regards, Chandra From guaneryu@gmail.com Thu Nov 3 21:40:25 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA42eOXO064882 for ; Thu, 3 Nov 2011 21:40:25 -0500 X-ASG-Debug-ID: 1320374423-232200fa0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-vw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 501C521EBD1 for ; Thu, 3 Nov 2011 19:40:23 -0700 (PDT) Received: from mail-vw0-f53.google.com (mail-vw0-f53.google.com [209.85.212.53]) by cuda.sgi.com with ESMTP id yuXv7XiL1vYDutrb for ; Thu, 03 Nov 2011 19:40:23 -0700 (PDT) Received: by vws19 with SMTP id 19so559611vws.26 for ; Thu, 03 Nov 2011 19:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer; bh=WRjEdOt4mi6zkiiaqYJ3k3klu3DYqchPrgetfzNO07s=; b=rfg9TA5Q9ey8V6A0GWNtDPOG8hzYxMoq0keH9nU4nSYmgFXqj9TjqxsK59JrzOh+Br SNGTmS1IiFiRhQfEqbKjfBoYicXMGLs/349qAJZ5HF8XSYCxXMoI0hyQHgEMDYawx1/a To1zge3PrXbXsARfWgMX3kzJTxv8fx5npeZRU= Received: by 10.52.69.235 with SMTP id h11mr12689124vdu.92.1320374423186; Thu, 03 Nov 2011 19:40:23 -0700 (PDT) Received: from localhost ([123.119.248.25]) by mx.google.com with ESMTPS id et6sm11084254vdc.15.2011.11.03.19.40.19 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 19:40:22 -0700 (PDT) From: Eryu Guan To: xfs@oss.sgi.com Cc: Eryu Guan X-ASG-Orig-Subj: [PATCH 1/2] xfstests: 083 make variable filesize as size in bytes Subject: [PATCH 1/2] xfstests: 083 make variable filesize as size in bytes Date: Fri, 4 Nov 2011 10:38:03 +0800 Message-Id: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> X-Mailer: git-send-email 1.7.7.1 X-Barracuda-Connect: mail-vw0-f53.google.com[209.85.212.53] X-Barracuda-Start-Time: 1320374424 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0532 1.0000 -1.6799 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.68 X-Barracuda-Spam-Status: No, SCORE=-1.68 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79259 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The first argument of _scratch_mkfs_sized() should be file system size in bytes, so '100m' is not a valid number, and the 'expr' expression in _scratch_mkfs_sized() complains 'non-numeric argument' about it. Signed-off-by: Eryu Guan --- 083 | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/083 b/083 index 441284e..e0670b9 100755 --- a/083 +++ b/083 @@ -91,7 +91,7 @@ workout() echo "*** test out-of-space handling for random write operations" -filesize=100m +filesize=`expr 100 \* 1024 \* 1024` agcount=6 numprocs=15 numops=1500 -- 1.7.7.1 From guaneryu@gmail.com Thu Nov 3 21:42:20 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,FREEMAIL_FROM, J_CHICKENPOX_45,J_CHICKENPOX_62,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA42gKdh064945 for ; Thu, 3 Nov 2011 21:42:20 -0500 X-ASG-Debug-ID: 1320374538-2fd400f60000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-vw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5A9F4101469E for ; Thu, 3 Nov 2011 19:42:18 -0700 (PDT) Received: from mail-vw0-f53.google.com (mail-vw0-f53.google.com [209.85.212.53]) by cuda.sgi.com with ESMTP id QrWj2WDKJ0PIRTg4 for ; Thu, 03 Nov 2011 19:42:18 -0700 (PDT) Received: by vws19 with SMTP id 19so560615vws.26 for ; Thu, 03 Nov 2011 19:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; bh=C1s18YjCfbhNR+zT5n+6ZWqTP7q7bHzwIplwb2KWNdo=; b=lJMBuTf9VWvR4FxMxLn7W/H5yy/M61PYjIjzGQOWCc8UupOo1u1hM9pB+b0NICppai +widfZdZ/6I7zyhLO9+ak7Xn8JFIdIUWEOXrb5v2+0E03dvreUELm7IjDhLLrLqmEqsc NIsTIKHvLGTeDB7MiihBHR748T+SJWxYYlTgE= Received: by 10.52.88.231 with SMTP id bj7mr12836287vdb.81.1320374537874; Thu, 03 Nov 2011 19:42:17 -0700 (PDT) Received: from localhost ([123.119.248.25]) by mx.google.com with ESMTPS id c20sm11118574vdt.6.2011.11.03.19.42.15 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 03 Nov 2011 19:42:17 -0700 (PDT) From: Eryu Guan To: xfs@oss.sgi.com Cc: Eryu Guan X-ASG-Orig-Subj: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Subject: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Date: Fri, 4 Nov 2011 10:38:04 +0800 Message-Id: <1320374284-1534-2-git-send-email-guaneryu@gmail.com> X-Mailer: git-send-email 1.7.7.1 In-Reply-To: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> References: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> X-Barracuda-Connect: mail-vw0-f53.google.com[209.85.212.53] X-Barracuda-Start-Time: 1320374539 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0330 1.0000 -1.8077 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.81 X-Barracuda-Spam-Status: No, SCORE=-1.81 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79260 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean btrfs requires at least 256M file system size, so check 'fssize' in _scratch_mkfs_sized first and give it a proper value. Otherwise mkfs.btrfs will complain something like "File system size 267386880 bytes is too small, 256M is required at least" This makes 015 077 and 083 run on btrfs. Signed-off-by: Eryu Guan --- common.rc | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/common.rc b/common.rc index e948169..ab61786 100644 --- a/common.rc +++ b/common.rc @@ -356,6 +356,11 @@ _scratch_mkfs_sized() /sbin/mkfs.$FSTYP $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks ;; btrfs) + # btrfs needs at least 256M file system size + if [ $fssize -lt $((256*1024*1024)) ] + then + fssize=$((256*1024*1024)) + fi $MKFS_BTRFS_PROG $MKFS_OPTIONS -b $fssize $SCRATCH_DEV ;; *) -- 1.7.7.1 From BATV+78e98da816627bc9eefb+2994+infradead.org+hch@bombadil.srs.infradead.org Fri Nov 4 00:44:29 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, J_CHICKENPOX_62 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA45iRTc074245 for ; Fri, 4 Nov 2011 00:44:29 -0500 X-ASG-Debug-ID: 1320385148-6d4f01370000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5316121EDA5 for ; Thu, 3 Nov 2011 22:39:09 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id w8WGGYLvfRn7MYDL for ; Thu, 03 Nov 2011 22:39:09 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMCUe-0005EO-Mp; Fri, 04 Nov 2011 05:39:08 +0000 Date: Fri, 4 Nov 2011 01:39:08 -0400 From: Christoph Hellwig To: Eryu Guan Cc: xfs@oss.sgi.com, linux-btrfs@vger.kernel.org X-ASG-Orig-Subj: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Subject: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Message-ID: <20111104053908.GB16720@infradead.org> References: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> <1320374284-1534-2-git-send-email-guaneryu@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320374284-1534-2-git-send-email-guaneryu@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320385149 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79271 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Nov 04, 2011 at 10:38:04AM +0800, Eryu Guan wrote: > btrfs requires at least 256M file system size, so check 'fssize' in > _scratch_mkfs_sized first and give it a proper value. Otherwise > mkfs.btrfs will complain something like > > "File system size 267386880 bytes is too small, 256M is required at least" > > This makes 015 077 and 083 run on btrfs. But it's not really going to test the ENOSPC conditions it was designed to exercise. It would be better to either find a way to make btrfs run on smaller filesystem, or call _notrun for too small filesystems on btrfs from _scratch_mkfs_sized. > > Signed-off-by: Eryu Guan > --- > common.rc | 5 +++++ > 1 files changed, 5 insertions(+), 0 deletions(-) > > diff --git a/common.rc b/common.rc > index e948169..ab61786 100644 > --- a/common.rc > +++ b/common.rc > @@ -356,6 +356,11 @@ _scratch_mkfs_sized() > /sbin/mkfs.$FSTYP $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks > ;; > btrfs) > + # btrfs needs at least 256M file system size > + if [ $fssize -lt $((256*1024*1024)) ] > + then > + fssize=$((256*1024*1024)) > + fi > $MKFS_BTRFS_PROG $MKFS_OPTIONS -b $fssize $SCRATCH_DEV > ;; > *) > -- > 1.7.7.1 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs ---end quoted text--- From guaneryu@gmail.com Fri Nov 4 01:07:15 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,FREEMAIL_FROM, J_CHICKENPOX_45,J_CHICKENPOX_62,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA467FNx074756 for ; Fri, 4 Nov 2011 01:07:15 -0500 X-ASG-Debug-ID: 1320386833-3eda02250000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ww0-f51.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7924518C2424 for ; Thu, 3 Nov 2011 23:07:13 -0700 (PDT) Received: from mail-ww0-f51.google.com (mail-ww0-f51.google.com [74.125.82.51]) by cuda.sgi.com with ESMTP id mMveGGpgpyA73u8k for ; Thu, 03 Nov 2011 23:07:13 -0700 (PDT) Received: by wwf27 with SMTP id 27so2250191wwf.32 for ; Thu, 03 Nov 2011 23:07:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:reply-to:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=9T24K3aMz4xK1v2hwpcx1GzrObTHARpr5NZDLPsa4fw=; b=u6QYNEfB6vS8o5sNS8ODc83uxc4sxOPLkZ+tqZvZ5DiP4hM7injVcTd/uPogZAThQJ 5DSiDkr91l8cPQhRtJ1QWfcfsV4LfoHY5NTJbyizmRZqkekqd9vz9+IfvG0+mrNBv2vr oHOsJ1Z1y+0FisTws+DjvjKTvi6r11HVm+MrM= Received: by 10.216.137.193 with SMTP id y43mr2950122wei.108.1320386832100; Thu, 03 Nov 2011 23:07:12 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.173.131 with HTTP; Thu, 3 Nov 2011 23:06:51 -0700 (PDT) Reply-To: guaneryu@gmail.com In-Reply-To: <20111104053908.GB16720@infradead.org> References: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> <1320374284-1534-2-git-send-email-guaneryu@gmail.com> <20111104053908.GB16720@infradead.org> From: Eryu Guan Date: Fri, 4 Nov 2011 14:06:51 +0800 Message-ID: X-ASG-Orig-Subj: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Subject: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() To: Christoph Hellwig Cc: xfs@oss.sgi.com, linux-btrfs@vger.kernel.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-ww0-f51.google.com[74.125.82.51] X-Barracuda-Start-Time: 1320386834 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79273 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Nov 4, 2011 at 1:39 PM, Christoph Hellwig wrote= : > On Fri, Nov 04, 2011 at 10:38:04AM +0800, Eryu Guan wrote: >> btrfs requires at least 256M file system size, so check 'fssize' in >> _scratch_mkfs_sized first and give it a proper value. Otherwise >> mkfs.btrfs will complain something like >> >> "File system size 267386880 bytes is too small, 256M is required at leas= t" >> >> This makes 015 077 and 083 run on btrfs. > > But it's not really going to test the ENOSPC conditions it was designed > to exercise. =A0It would be better to either find a way to make btrfs run= on > smaller filesystem, or call _notrun for too small filesystems on btrfs > from _scratch_mkfs_sized. Yes, you're right. I checked 015 (it would dd zero to btrfs to full) but fo= rgot to check 077 and 083. I'll try to find a better way. Thanks for reviewing! Eryu Guan > >> >> Signed-off-by: Eryu Guan >> --- >> =A0common.rc | =A0 =A05 +++++ >> =A01 files changed, 5 insertions(+), 0 deletions(-) >> >> diff --git a/common.rc b/common.rc >> index e948169..ab61786 100644 >> --- a/common.rc >> +++ b/common.rc >> @@ -356,6 +356,11 @@ _scratch_mkfs_sized() >> =A0 =A0 =A0 /sbin/mkfs.$FSTYP $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $= blocks >> =A0 =A0 =A0 ;; >> =A0 =A0 =A0btrfs) >> + =A0 =A0# btrfs needs at least 256M file system size >> + =A0 =A0 if [ $fssize -lt $((256*1024*1024)) ] >> + =A0 =A0 then >> + =A0 =A0 =A0 =A0 =A0 =A0 fssize=3D$((256*1024*1024)) >> + =A0 =A0 fi >> =A0 =A0 =A0 $MKFS_BTRFS_PROG $MKFS_OPTIONS -b $fssize $SCRATCH_DEV >> =A0 =A0 =A0 ;; >> =A0 =A0 =A0*) >> -- >> 1.7.7.1 >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs > ---end quoted text--- > From sbehrens@giantdisaster.de Fri Nov 4 03:21:50 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, J_CHICKENPOX_62,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA48LnsN079056 for ; Fri, 4 Nov 2011 03:21:50 -0500 X-ASG-Debug-ID: 1320394902-6d4603220000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mo-p00-ob.rzone.de (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B849321F415 for ; Fri, 4 Nov 2011 01:21:42 -0700 (PDT) Received: from mo-p00-ob.rzone.de (mo-p00-ob.rzone.de [81.169.146.162]) by cuda.sgi.com with ESMTP id LLKDYsDTAO1VzhPm for ; Fri, 04 Nov 2011 01:21:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1320394902; l=1756; s=domk; d=giantdisaster.de; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:References: Subject:CC:To:MIME-Version:From:Date:X-RZG-CLASS-ID:X-RZG-AUTH; bh=BrEh5NaC8cYgOcMesy2z4zDnL3g=; b=LcZiTbkwQYFn8UM1aQl2dV/hPPFKqU4qb32C0yftURpnjNnIAn/yYYt6Ok+3dSMs/y/ zzztwGJPwrOFUfqLcf83WrMQBYBGh8DHMRBRJzWH0zSxzHVXc4qAO6Rn38kCHi/cUO6b1 Rm+vplwYs8sQl9dz+Glu3AXoYsXIAqASwqQ= X-RZG-AUTH: :IGUKYFjleetgZuRbHZjp6Ve7NzeE1efWuTR/wV06y353QgIuD5+acdRFtJ8MDHZp4u74mTVB5dOWYQ== X-RZG-CLASS-ID: mo00 Received: from [172.24.1.80] (yian-ho01.nir.cronon.net [192.166.201.94]) by smtp.strato.de (klopstock mo10) (RZmta 26.10 AUTH) with ESMTPA id 905dc1nA47fxX1 ; Fri, 4 Nov 2011 09:21:30 +0100 (MET) Message-ID: <4EB3A08C.7010803@giantdisaster.de> Date: Fri, 04 Nov 2011 09:21:32 +0100 From: Stefan Behrens User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: guaneryu@gmail.com CC: Christoph Hellwig , xfs@oss.sgi.com, linux-btrfs@vger.kernel.org X-ASG-Orig-Subj: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() Subject: Re: [PATCH 2/2] xfstests: meet btrfs fs size requirement in _scratch_mkfs_sized() References: <1320374284-1534-1-git-send-email-guaneryu@gmail.com> <1320374284-1534-2-git-send-email-guaneryu@gmail.com> <20111104053908.GB16720@infradead.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mo-p00-ob.rzone.de[81.169.146.162] X-Barracuda-Start-Time: 1320394908 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0006 1.0000 -2.0172 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79283 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 11/4/2011 7:06 AM, Eryu Guan wrote: > On Fri, Nov 4, 2011 at 1:39 PM, Christoph Hellwig wrote: >> On Fri, Nov 04, 2011 at 10:38:04AM +0800, Eryu Guan wrote: >>> btrfs requires at least 256M file system size, so check 'fssize' in >>> _scratch_mkfs_sized first and give it a proper value. Otherwise >>> mkfs.btrfs will complain something like >>> >>> "File system size 267386880 bytes is too small, 256M is required at least" Commit bd7d6d45 from Zhong, Xin with date 6/21/2011 removed the 256MB size limitation in mkfs.c in btrfs-progs. >>> >>> This makes 015 077 and 083 run on btrfs. >> >> But it's not really going to test the ENOSPC conditions it was designed >> to exercise. It would be better to either find a way to make btrfs run on >> smaller filesystem, or call _notrun for too small filesystems on btrfs >> from _scratch_mkfs_sized. > > Yes, you're right. I checked 015 (it would dd zero to btrfs to full) but forgot > to check 077 and 083. > > I'll try to find a better way. Thanks for reviewing! > > Eryu Guan >> >>> >>> Signed-off-by: Eryu Guan >>> --- >>> common.rc | 5 +++++ >>> 1 files changed, 5 insertions(+), 0 deletions(-) >>> >>> diff --git a/common.rc b/common.rc >>> index e948169..ab61786 100644 >>> --- a/common.rc >>> +++ b/common.rc >>> @@ -356,6 +356,11 @@ _scratch_mkfs_sized() >>> /sbin/mkfs.$FSTYP $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks >>> ;; >>> btrfs) >>> + # btrfs needs at least 256M file system size >>> + if [ $fssize -lt $((256*1024*1024)) ] >>> + then >>> + fssize=$((256*1024*1024)) >>> + fi >>> $MKFS_BTRFS_PROG $MKFS_OPTIONS -b $fssize $SCRATCH_DEV >>> ;; >>> *) >>> -- >>> 1.7.7.1 From bjacke@sernet.de Fri Nov 4 06:16:56 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.6 required=5.0 tests=BAYES_00,MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA4BGtJR087756 for ; Fri, 4 Nov 2011 06:16:56 -0500 X-ASG-Debug-ID: 1320405413-7ac9011b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.SerNet.de (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BABCA21FE0D for ; Fri, 4 Nov 2011 04:16:54 -0700 (PDT) Received: from mail.SerNet.de (mail.SerNet.de [193.175.80.2]) by cuda.sgi.com with ESMTP id 8lyyOornvLDj1U3X for ; Fri, 04 Nov 2011 04:16:54 -0700 (PDT) Received: from intern.SerNet.DE by mail.SerNet.DE with esmtp (Exim 4.69 #1) id 1RMHlS-0006Jp-SL; Fri, 04 Nov 2011 12:16:50 +0100 Received: by intern.SerNet.DE id 1RMHlS-00HYp9-Hk; Fri, 04 Nov 2011 12:16:50 +0100 Received: by intern.SerNet.DE id 1RMHlR-00HYp2-EN; Fri, 04 Nov 2011 12:16:50 +0100 Received: from bjacke by pell.sernet.de with local (Exim 4.75) (envelope-from ) id 1RMHlE-0004hJ-Er; Fri, 04 Nov 2011 04:16:36 -0700 Date: Fri, 4 Nov 2011 12:16:36 +0100 From: =?iso-8859-1?Q?Bj=F6rn?= JACKE To: Anton Altaparmakov Cc: Steve French , linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, LKML , Unix Support , linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: CIFS: Rename bug on servers not supporting inode numbers Subject: Re: CIFS: Rename bug on servers not supporting inode numbers Mail-Followup-To: Anton Altaparmakov , Steve French , linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, LKML , Unix Support , linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com References: <9A609E6D-DF91-4AF9-855A-6D28E9B95D15@cam.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9A609E6D-DF91-4AF9-855A-6D28E9B95D15@cam.ac.uk> X-Q: Die Schriftsteller =?iso-8859-1?Q?k=F6nnen?= =?iso-8859-1?Q?_nicht_so_schnell_schreiben=2C_wie_die_Regierungen_Krieg?= =?iso-8859-1?Q?e?= machen; denn das Schreiben verlangt Denkarbeit. - Brecht Message-Id: Organization: SerNet GmbH, Goettingen, Germany X-Barracuda-Connect: mail.SerNet.de[193.175.80.2] X-Barracuda-Start-Time: 1320405414 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79293 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 2011-11-03 at 15:20 +0000 Anton Altaparmakov sent off: > Hi, > > Our CIFS server problems seem to have no end… The Novell CIFS server does not support server inode numbers (when I try the mount option I get the message it is being turned off as server does not support it) and thus each inode gets a different number each time it is accessed and it gets a different number again for each readdir call. > > The fun happens with rename() when the rename source and target only differ in case, e.g. > > touch foo > mv foo Foo somehow related seems https://bugzilla.kernel.org/show_bug.cgi?id=39512 (as long as kernel bugzilla is dead, see http://www.linux.sgi.com/archives/xfs-masters/2011-07/msg00022.html ) Case insensitive filesystems seem to be a problem in general on Linux. Not sure how far kernel and/or glibc are involved in the problem. As a workaround for the mess you need to do a temporary rename to a different name (not just a case equivalent name). Björn -- SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen phone: +49-551-370000-0, fax: +49-551-370000-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen From BATV+78e98da816627bc9eefb+2994+infradead.org+hch@bombadil.srs.infradead.org Fri Nov 4 10:25:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA4FPCUI100166 for ; Fri, 4 Nov 2011 10:25:12 -0500 X-ASG-Debug-ID: 1320420306-17ec010b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 838121F2D6F2 for ; Fri, 4 Nov 2011 08:25:07 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 2mL0heasNiMlpmn0 for ; Fri, 04 Nov 2011 08:25:07 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMLdi-0003E1-Fi for xfs@oss.sgi.com; Fri, 04 Nov 2011 15:25:06 +0000 Date: Fri, 4 Nov 2011 11:25:06 -0400 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH] xfs: fix attr2 vs large data fork assert Subject: [PATCH] xfs: fix attr2 vs large data fork assert Message-ID: <20111104152506.GA7869@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320420307 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79311 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean With Dmitry fsstress updates I've seen very reproducible crashes in xfs_attr_shortform_remove because xfs_attr_shortform_bytesfit claims that the attributes would not fit inline into the inode after removing an attribute. It turns out that we were operating on an inode with lots of delalloc extents, and thus an if_bytes values for the data fork that is larger than biggest possible on-disk storage for it which utterly confuses the code near the end of xfs_attr_shortform_bytesfit. Fix this by always allowing the current attribute fork, like we already do for the attr1 format, given that delalloc conversion will take care for moving either the data or attribute area out of line if it doesn't fit at that point - or making the point moot by merging extents at this point. Also document the function better, and clean up some lose bits. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/xfs/xfs_attr_leaf.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_attr_leaf.c 2011-11-04 13:46:59.481655134 +0100 +++ linux-2.6/fs/xfs/xfs_attr_leaf.c 2011-11-04 15:58:06.480155275 +0100 @@ -110,6 +110,7 @@ xfs_attr_namesp_match(int arg_flags, int /* * Query whether the requested number of additional bytes of extended * attribute space will be able to fit inline. + * * Returns zero if not, else the di_forkoff fork offset to be used in the * literal area for attribute data once the new bytes have been added. * @@ -136,11 +137,26 @@ xfs_attr_shortform_bytesfit(xfs_inode_t return (offset >= minforkoff) ? minforkoff : 0; } - if (!(mp->m_flags & XFS_MOUNT_ATTR2)) { - if (bytes <= XFS_IFORK_ASIZE(dp)) - return dp->i_d.di_forkoff; + /* + * If the requested numbers of bytes is smaller or equal to the + * current attribute fork size we can always proceed. + * + * Note that if_bytes in the data fork might actually be larger than + * the current data fork size is due to delalloc extents. In that + * case either the extent count will go down when they are converted + * to ral extents, or the delalloc conversion will take care of the + * literal area rebalancing. + */ + if (bytes <= XFS_IFORK_ASIZE(dp)) + return dp->i_d.di_forkoff; + + /* + * For attr2 we can try to move the forkoff if there is space in the + * literal area, but for the old format we are done if there is no + * space in the fixes attribute fork. + */ + if (!(mp->m_flags & XFS_MOUNT_ATTR2)) return 0; - } dsize = dp->i_df.if_bytes; @@ -157,10 +173,9 @@ xfs_attr_shortform_bytesfit(xfs_inode_t xfs_default_attroffset(dp)) dsize = XFS_BMDR_SPACE_CALC(MINDBTPTRS); break; - case XFS_DINODE_FMT_BTREE: /* - * If have data btree then keep forkoff if we have one, + * If have a data btree then keep forkoff if we have one, * otherwise we are adding a new attr, so then we set * minforkoff to where the btree root can finish so we have * plenty of room for attrs @@ -168,10 +183,9 @@ xfs_attr_shortform_bytesfit(xfs_inode_t if (dp->i_d.di_forkoff) { if (offset < dp->i_d.di_forkoff) return 0; - else - return dp->i_d.di_forkoff; - } else - dsize = XFS_BMAP_BROOT_SPACE(dp->i_df.if_broot); + return dp->i_d.di_forkoff; + } + dsize = XFS_BMAP_BROOT_SPACE(dp->i_df.if_broot); break; } @@ -186,10 +200,10 @@ xfs_attr_shortform_bytesfit(xfs_inode_t maxforkoff = XFS_LITINO(mp) - XFS_BMDR_SPACE_CALC(MINABTPTRS); maxforkoff = maxforkoff >> 3; /* rounded down */ - if (offset >= minforkoff && offset < maxforkoff) - return offset; if (offset >= maxforkoff) return maxforkoff; + if (offset >= minforkoff) + return offset; return 0; } From sandeen@sandeen.net Fri Nov 4 10:50:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA4FoAkp101660 for ; Fri, 4 Nov 2011 10:50:10 -0500 X-ASG-Debug-ID: 1320421808-36de015e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A27D4101895A for ; Fri, 4 Nov 2011 08:50:09 -0700 (PDT) Received: from mail.sandeen.net (sandeen.net [63.231.237.45]) by cuda.sgi.com with ESMTP id eEyrKebTv6XJ8bT3 for ; Fri, 04 Nov 2011 08:50:09 -0700 (PDT) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 9A91B496328B; Fri, 4 Nov 2011 10:50:08 -0500 (CDT) Message-ID: <4EB409B0.4050302@sandeen.net> Date: Fri, 04 Nov 2011 10:50:08 -0500 From: Eric Sandeen User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1 MIME-Version: 1.0 To: nanashi CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfsp_repair segfault (3.1.4 & 3.1.6) Subject: Re: xfsp_repair segfault (3.1.4 & 3.1.6) References: <4EAAED63.9050804@otenet.gr> In-Reply-To: <4EAAED63.9050804@otenet.gr> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: sandeen.net[63.231.237.45] X-Barracuda-Start-Time: 1320421809 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.71 X-Barracuda-Spam-Status: No, SCORE=-1.71 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=URI_HEX X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79312 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.32 URI_HEX URI: URI hostname has long hexadecimal sequence X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 10/28/11 12:58 PM, nanashi wrote: > Hi, > > I have a corrupted RAID5 xfs filesystem from an intel SS-4000 NAS storage. > I'm using slackware-current with 2.6.39.3-smp kernel (32bit & 64bit) > When i try xfs_repair I get > > xfs_repair: dir2.c:2133: process_dir2: Assertion `(ino > !=mp->m_sb.sb_rootino && ino != *parent) || (ino == > mp->m_sb.sb_rootino && (ino == *parent || need_root_dotdot == 1))' > > I try both x86 and x86_64, I build 3.1.4 and 3.1.6 with DEBUG=-DNDEBUG and both continue but gave me a segfault > > [23978.718305] xfs_repair[25800]: segfault at 7fffa1d81ff0 ip 00007f15c1852049 sp 00007fffa257f048 error 6 in libc-2.13.so[7f15c1715000+19b000] > > I try xfs_metadump and I get segfault too. > > The partition is 2.2TB and I don't have enough space to dd it to an image. > > I attach the xfs_repair output before the segfault. > > any help is appreciated. > A corefile + debug binary would be helpful too... Argh, if metadump segfaults, that will make things tough. At least with a core + binary we could see just where it blew up. -Eric From BATV+78e98da816627bc9eefb+2994+infradead.org+hch@bombadil.srs.infradead.org Fri Nov 4 16:03:37 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA4L3ano116779 for ; Fri, 4 Nov 2011 16:03:37 -0500 X-ASG-Debug-ID: 1320440606-650901910000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9DD9422247D for ; Fri, 4 Nov 2011 14:03:26 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id zAqa6iL8cYlOVkuC for ; Fri, 04 Nov 2011 14:03:26 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMQv6-00043S-DG; Fri, 04 Nov 2011 21:03:24 +0000 Date: Fri, 4 Nov 2011 17:03:24 -0400 From: Christoph Hellwig To: Simon Kirby Cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: XFS read hangs in 3.1-rc10 Subject: Re: XFS read hangs in 3.1-rc10 Message-ID: <20111104210324.GB14983@infradead.org> References: <20111020224214.GC22772@hostway.ca> <20111021132240.GA24136@infradead.org> <20111021202857.GB30100@hostway.ca> <20111024082219.GA19941@infradead.org> <20111025200748.GA25043@hostway.ca> <20111026112521.GA10432@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111026112521.GA10432@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320440611 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0004 1.0000 -2.0185 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79333 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Sorry Simon, I haven't really been able to get back to this as fast as I wanted to do. Can you do another run and check if the patch still helps your issue even if you remove the xfs_log_force line? From ligsae@sina.com Fri Nov 4 20:30:57 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: ** X-Spam-Status: No, score=2.4 required=5.0 tests=BAYES_50,MIME_8BIT_HEADER, MIME_BASE64_BLANKS,RCVD_IN_BL_SPAMCOP_NET autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA51Uurp126466 for ; Fri, 4 Nov 2011 20:30:57 -0500 X-ASG-Debug-ID: 1320456651-3eb8000b0000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail3-184.sinamail.sina.com.cn (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C859A1CE5FD2 for ; Fri, 4 Nov 2011 18:30:52 -0700 (PDT) Received: from mail3-184.sinamail.sina.com.cn (mail3-184.sinamail.sina.com.cn [202.108.3.184]) by cuda.sgi.com with ESMTP id Cnn0G19OrI4SK8dJ for ; Fri, 04 Nov 2011 18:30:52 -0700 (PDT) X-Originating-IP: [183.13.198.195] X-Auth-ID: ligsae X-Rcptcnt: gt3 X-Rcptcnt: gt10 Received: from unknown (HELO dwsx) ([183.13.198.195]) by irja2-171.sinamail.sina.com.cn with ESMTP; 05 Nov 2011 09:30:32 +0800 Reply-To: Message-ID: <2D994F9F01550D3F8E9E337B3F7AAF3E@dwsx> From: "wmzsixz" To: X-ASG-Orig-Subj: =?utf-8?B?5oCA5b+15LmU5biD5pavdmQwfQ==?= Subject: =?utf-8?B?5oCA5b+15LmU5biD5pavdmQwfQ==?= Date: Sat, 5 Nov 2011 09:09:44 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5512 Disposition-Notification-To: chenda1888@163.com X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5512 X-Barracuda-Connect: mail3-184.sinamail.sina.com.cn[202.108.3.184] X-Barracuda-Start-Time: 1320456654 X-Barracuda-Bayes: INNOCENT GLOBAL 0.3973 1.0000 -0.0069 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -0.01 X-Barracuda-Spam-Status: No, SCORE=-0.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MIME_BASE64_BLANKS X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79351 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 MIME_BASE64_BLANKS RAW: Extra blank lines in base64 encoding X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean 4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh 4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh4omh 4omhICANCg0K4omh4omhICDjgJDkvJjjgJEtLeOAkOS7o+OAkS0t44CQ5Luj44CRIA0K4omh4omh ICDjgJDmg6DjgJEtLeOAkOW8gOOAkS0t44CQ5byA44CRICAgICAgICANCuKJoeKJoSAg44CQ5Luj 44CRLS3jgJDmma7jgJEtLeOAkOWinuOAkSAgDQriiaHiiaEgIOOAkOW8gOOAkS0t44CQ6YCa44CR LS3jgJDlgLzjgJEgIA0K4omh4omhICDjgJDnnJ/jgJEtLeOAkOeojuOAkS0t44CQ56iO44CRICAN CuKJoeKJoSAg44CQ5Y+R44CRLS3jgJDlj5HjgJEtLeOAkOWPkeOAkSANCuKJoeKJoSAg44CQ56Wo 44CRLS3jgJDnpajjgJEtLeOAkOelqOOAkQ0K4omh4omhICAgICAgICANCuKJoeKJoSAgICAgIOiB lOezu+S6uu+8mumZiOWFiOeUnyAg5rS96LCIDQriiaHiiaEgICAgICDnlLXor506MTM3MTUyMjEx NDMgDQriiaHiiaEgICAgIA0K4omh4omhICAgICAgICAgICAgICAgICAgICAgICAgICANCuKJoeKJ oeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJ oeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoeKJoSA= From sandy.miller73@gmail.com Sat Nov 5 00:03:37 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, SUBJ_FRIEND autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA553bru135392 for ; Sat, 5 Nov 2011 00:03:37 -0500 X-ASG-Debug-ID: 1320469414-5b2102fb0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.webtechouse.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9D720222D69 for ; Fri, 4 Nov 2011 22:03:34 -0700 (PDT) Received: from mail.webtechouse.net (mail.webtechouse.net [66.29.97.199]) by cuda.sgi.com with ESMTP id HE1oYXqdk4zZje2K for ; Fri, 04 Nov 2011 22:03:34 -0700 (PDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.webtechouse.net (Postfix) with ESMTP id 5FB2B2FA5809 for ; Sat, 5 Nov 2011 00:44:59 -0400 (EDT) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at Received: from mail.webtechouse.net ([127.0.0.1]) by localhost (mail.webtechouse.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u4y+rzFLBh7w for ; Sat, 5 Nov 2011 00:44:56 -0400 (EDT) Received: from localhost (unknown [10.20.0.1]) by mail.webtechouse.net (Postfix) with ESMTP id 4403A2FA57FC for ; Sat, 5 Nov 2011 00:44:56 -0400 (EDT) 0: To: "Friend" From: "Sandy" Reply-To: "Sandy" X-ASG-Orig-Subj: Your friend Sandy has recommended this great product from Euro Train Hobby Subject: Your friend Sandy has recommended this great product from Euro Train Hobby Message-Id: <20111105044456.4403A2FA57FC@mail.webtechouse.net> Date: Sat, 5 Nov 2011 00:44:56 -0400 (EDT) X-Barracuda-Connect: mail.webtechouse.net[66.29.97.199] X-Barracuda-Start-Time: 1320469415 X-Barracuda-Bayes: INNOCENT GLOBAL 0.3635 1.0000 -0.1067 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -0.01 X-Barracuda-Spam-Status: No, SCORE=-0.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA085 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79365 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 BSF_SC0_SA085 Custom Rule SA085 X-Virus-Status: Clean --=_658bfcb73ac71a51dc84dee07eb5b37f Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Hi Friend! Your friend, Sandy, thought that you would be interested in Refrigerator beer car "Dortmunder Hansa-Brauerei" - US Zone from Euro Train Hobby. Hi, If you can't finally make some money with this: http://neowealth.co.cc/nmo1.php?e=xfs@oss.sgi.com You may as well just quit. Sad, but true. Cheers, Sandy To unsubscribe please click the link below: http://neowealth.co.cc/un.php?e=xfs@oss.sgi.com To view the product click on the link below or copy and paste the link into your web browser: http://www.eurotrainhobby.com/product_info.php/products_id/4566 Regards, Euro Train Hobby http://www.eurotrainhobby.com/ --=_658bfcb73ac71a51dc84dee07eb5b37f Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Friend!

Your friend, Sandy, thought that you would be interested = in Refrigerator beer car "Dortmunder Hansa-Brauerei" - US Zone from Euro Tr= ain Hobby.

Hi,

If you can't finally make
some money with t= his:

http://neowealth.co.cc/nmo1.php?e=3Dxfs@oss.sgi.com

You = may as well just quit.

Sad, but true.

Cheers,
Sandy

To unsubscribe please click the link below:
http://neowealth.co.cc/= un.php?e=3Dxfs@oss.sgi.com

To view the product click on the link bel= ow or copy and paste the link into your web browser:

http://www.euro= trainhobby.com/product_info.php/products_id/4566

Regards,

Eur= o Train Hobby
http://www.eurotrainhobby.com/
--=_658bfcb73ac71a51dc84dee07eb5b37f-- From IreneK178@aol.com Sat Nov 5 01:14:22 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: **** X-Spam-Status: No, score=4.0 required=5.0 tests=BAYES_99,FREEMAIL_FROM, T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA56EMF7137444 for ; Sat, 5 Nov 2011 01:14:22 -0500 X-ASG-Debug-ID: 1320473660-06ed004f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from imr-da04.mx.aol.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 47E951CE71DE for ; Fri, 4 Nov 2011 23:14:20 -0700 (PDT) Received: from imr-da04.mx.aol.com (imr-da04.mx.aol.com [205.188.105.146]) by cuda.sgi.com with ESMTP id 9ZGXyuoeT26Ey3xf for ; Fri, 04 Nov 2011 23:14:20 -0700 (PDT) Received: from mtaomg-da03.r1000.mx.aol.com (mtaomg-da03.r1000.mx.aol.com [172.29.51.139]) by imr-da04.mx.aol.com (8.14.1/8.14.1) with ESMTP id pA56EHBS025657 for ; Sat, 5 Nov 2011 02:14:17 -0400 Received: from core-dha001c.r1000.mail.aol.com (core-dha001.r1000.mail.aol.com [172.29.208.129]) by mtaomg-da03.r1000.mx.aol.com (OMAG/Core Interface) with ESMTP id B61ECE000081 for ; Sat, 5 Nov 2011 02:14:17 -0400 (EDT) From: IreneK178@aol.com Message-ID: <3a9a.1a14aff0.3be62e39@aol.com> Date: Sat, 5 Nov 2011 02:14:17 -0400 (EDT) X-ASG-Orig-Subj: Remember Me? Subject: Remember Me? To: xfs@oss.sgi.com MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: AOL 4.0 Thin for Windows 95 sub 20 X-Originating-IP: [64.12.112.202] x-aol-global-disposition: G DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mx.aol.com; s=20110426; t=1320473657; bh=+bTa1AqYK5wuYwjTrVf/iZ13RgR2LFLJZe6CTI2wjlg=; h=From:To:Subject:Message-ID:Date:MIME-Version:Content-Type; b=j4WlkjHl4bM/sS/yvsJPvKSLD+KkMT5OLKf/JKfjUTJHsn76kcMhASN9Gy2YTLpSh 9nPfVfZqM8yxFwBYTTodNbXw0h2zvU/4VBt9tGlTAIDbIOlhC09sabXGunCwWScTwD Ljl0uX3M3gk9RmcelfhzcNITmzznFlsiJJbyEw4U= X-AOL-SCOLL-SCORE: 0:2:156671440:93952408 X-AOL-SCOLL-URL_COUNT: 0 x-aol-sid: 3039ac1d338b4eb4d4396454 X-Barracuda-Connect: imr-da04.mx.aol.com[205.188.105.146] X-Barracuda-Start-Time: 1320473661 X-Barracuda-Bayes: INNOCENT GLOBAL 0.5091 1.0000 0.7500 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 0.75 X-Barracuda-Spam-Status: No, SCORE=0.75 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED, NO_REAL_NAME X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79369 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 NO_REAL_NAME From: does not include a real name -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean 92588037 How goes it. I was messing around on the net and found this.. there doing a giveaway for an iPad2!! you need to hurry up and get yours before they run out heres the site. http://xav.cc/b33ea From BATV+a576dd851a46f4afe15d+2995+infradead.org+hch@bombadil.srs.infradead.org Sat Nov 5 03:11:15 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA58BFxZ142726 for ; Sat, 5 Nov 2011 03:11:15 -0500 X-ASG-Debug-ID: 1320480671-06ed01ea0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E2B921CE842F for ; Sat, 5 Nov 2011 01:11:11 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id qEYD0l9lZvGE45g8 for ; Sat, 05 Nov 2011 01:11:11 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMbLL-0003kq-3F for xfs@oss.sgi.com; Sat, 05 Nov 2011 08:11:11 +0000 Date: Sat, 5 Nov 2011 04:11:11 -0400 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 10/45] fix: force shutdown handling in xfs_end_io Subject: Re: [PATCH 10/45] fix: force shutdown handling in xfs_end_io Message-ID: <20111105081111.GA14382@infradead.org> References: <20111028095423.796574703@bombadil.infradead.org> <20111028100337.411618173@bombadil.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111028100337.411618173@bombadil.infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320480671 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79370 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Btw, this probably is another candidates for 3.2. On Fri, Oct 28, 2011 at 05:54:33AM -0400, Christoph Hellwig wrote: > Only ioend->io_error gets propagated back to e.g. AIO completions. > > Signed-off-by: Christoph Hellwig > > --- > fs/xfs/xfs_aops.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > Index: xfs/fs/xfs/xfs_aops.c > =================================================================== > --- xfs.orig/fs/xfs/xfs_aops.c 2011-10-27 22:39:56.068672482 +0200 > +++ xfs/fs/xfs/xfs_aops.c 2011-10-27 22:39:56.902173367 +0200 > @@ -184,7 +184,7 @@ xfs_end_io( > int error = 0; > > if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { > - error = -EIO; > + ioend->io_error = -EIO; > goto done; > } > if (ioend->io_error) > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs ---end quoted text--- From jakub@bogusz.priv.pl Sat Nov 5 04:00:32 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA590VdN145849 for ; Sat, 5 Nov 2011 04:00:32 -0500 X-ASG-Debug-ID: 1320483626-0bed03df0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.qboosh.pl (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C041A2230BA for ; Sat, 5 Nov 2011 02:00:26 -0700 (PDT) Received: from mail.qboosh.pl (mail.qboosh.pl [217.73.31.61]) by cuda.sgi.com with ESMTP id QmM0OudoMVg00jFT for ; Sat, 05 Nov 2011 02:00:26 -0700 (PDT) Received: from stranger.qboosh.pl (77-253-101-139.adsl.inetia.pl [77.253.101.139]) by mail.qboosh.pl (Postfix) with ESMTPSA id F1170CF9E7; Sat, 5 Nov 2011 10:00:59 +0100 (CET) Received: from stranger.qboosh.pl (localhost [127.0.0.1]) by stranger.qboosh.pl (8.14.3/8.14.3) with ESMTP id pA590VAU005658; Sat, 5 Nov 2011 10:00:31 +0100 Received: (from qboosh@localhost) by stranger.qboosh.pl (8.14.3/8.14.3/Submit) id pA590TUk005657; Sat, 5 Nov 2011 10:00:29 +0100 Date: Sat, 5 Nov 2011 10:00:29 +0100 From: Jakub Bogusz To: xfs@oss.sgi.com Cc: Christoph Hellwig X-ASG-Orig-Subj: Re: [PATCH v2] xfsprogs: fix various incorrect printf formats Subject: Re: [PATCH v2] xfsprogs: fix various incorrect printf formats Message-ID: <20111105090029.GA15258@stranger.qboosh.pl> References: <20111025115233.GB22103@infradead.org> <20111025155916.GA19818@stranger.qboosh.pl> <20111102092012.GA3808@infradead.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="WIyZ46R2i8wDzkSu" Content-Disposition: inline In-Reply-To: <20111102092012.GA3808@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: mail.qboosh.pl[217.73.31.61] X-Barracuda-Start-Time: 1320483627 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0003 1.0000 -2.0189 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79373 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --WIyZ46R2i8wDzkSu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wed, Nov 02, 2011 at 05:20:12AM -0400, Christoph Hellwig wrote: > Index: xfsprogs/repair/scan.c > =================================================================== > --- xfsprogs.orig/repair/scan.c 2011-10-30 05:56:36.168127218 +0100 > +++ xfsprogs/repair/scan.c 2011-10-30 05:57:45.981169953 +0100 > @@ -289,14 +289,14 @@ _("bad back (left) sibling pointer (saw > */ > set_bmap(agno, agbno, XR_E_MULT); > do_warn( > -_("inode 0x%" PRIu64 "bmap block 0x%" PRIu64 " claimed, state is %d\n"), > +_("inode 0x%" PRIu64 "bmap block 0x%" PRIx64 " claimed, state is %d\n"), > ino, bno, state); > break; > case XR_E_MULT: > case XR_E_INUSE_FS: > set_bmap(agno, agbno, XR_E_MULT); > do_warn( > -_("inode 0x%" PRIu64 " bmap block 0x%" PRIu64 " claimed, state is %d\n"), > +_("inode 0x%" PRIu64 " bmap block 0x%" PRIx64 " claimed, state is %d\n"), > ino, bno, state); > /* > * if we made it to here, this is probably a bmap block These two entries still need fix in the first format strings of both msgids... I'm attaching my version of the whole patch. -- Jakub Bogusz http://qboosh.pl/ --WIyZ46R2i8wDzkSu Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="xfsprogs-msgids.patch" Reported-by: Jakub Bogusz Signed-off-by: Christoph Hellwig Index: xfsprogs/libxfs/freebsd.c =================================================================== --- xfsprogs.orig/libxfs/freebsd.c 2011-10-30 05:56:36.180125966 +0100 +++ xfsprogs/libxfs/freebsd.c 2011-10-30 05:57:45.957126722 +0100 @@ -126,15 +126,13 @@ platform_findsizes(char *path, int fd, l } if ((st.st_mode & S_IFMT) != S_IFCHR) { - fprintf(stderr, _("%s: " - "Not a device or file: \"%s\"n"), + fprintf(stderr, _("%s: Not a device or file: \"%s\"\n"), progname, path); exit(1); } if (ioctl(fd, DIOCGMEDIASIZE, &size) != 0) { - fprintf(stderr, _("%s: " - "DIOCGMEDIASIZE failed on \"%s\": %s\n"), + fprintf(stderr, _("%s: DIOCGMEDIASIZE failed on \"%s\": %s\n"), progname, path, strerror(errno)); exit(1); } Index: xfsprogs/repair/dinode.c =================================================================== --- xfsprogs.orig/repair/dinode.c 2011-10-30 05:56:36.132124261 +0100 +++ xfsprogs/repair/dinode.c 2011-10-30 05:57:45.957126722 +0100 @@ -1439,7 +1439,7 @@ _("mismatch between format (%d) and size } if (dino->di_format == XFS_DINODE_FMT_LOCAL) { do_warn( -_("mismatch between format (%d) and size (%" PRId64 ") in symlink inode %" PRIu64 "n"), +_("mismatch between format (%d) and size (%" PRId64 ") in symlink inode %" PRIu64 "\n"), dino->di_format, (__int64_t)be64_to_cpu(dino->di_size), lino); return 1; Index: xfsprogs/repair/dir2.c =================================================================== --- xfsprogs.orig/repair/dir2.c 2011-10-30 05:56:36.156126568 +0100 +++ xfsprogs/repair/dir2.c 2011-11-02 10:16:46.593593447 +0100 @@ -1174,7 +1174,7 @@ _("would have corrected entry count in d ino, sfp->hdr.count, i); } else { do_warn( -_("corrected entry count in directory %" PRIu64 "u, was %d, now %d\n"), +_("corrected entry count in directory %" PRIu64 ", was %d, now %d\n"), ino, sfp->hdr.count, i); sfp->hdr.count = i; *dino_dirty = 1; Index: xfsprogs/repair/phase6.c =================================================================== --- xfsprogs.orig/repair/phase6.c 2011-10-30 05:56:36.144127454 +0100 +++ xfsprogs/repair/phase6.c 2011-10-30 05:57:45.973171205 +0100 @@ -2071,7 +2071,7 @@ longform_dir2_entry_check_data( if (is_inode_free(irec, ino_offset)) { nbad++; if (entry_junked( - _("entry \"%s\" in directory inode %" PRIu64 " points to free inode " PRIu64), + _("entry \"%s\" in directory inode %" PRIu64 " points to free inode %" PRIu64), fname, ip->i_ino, inum)) { dep->name[0] = '/'; libxfs_dir2_data_log_entry(tp, bp, dep); Index: xfsprogs/repair/scan.c =================================================================== --- xfsprogs.orig/repair/scan.c 2011-10-30 05:56:36.168127218 +0100 +++ xfsprogs/repair/scan.c 2011-10-30 05:57:45.981169953 +0100 @@ -289,14 +289,14 @@ _("bad back (left) sibling pointer (saw */ set_bmap(agno, agbno, XR_E_MULT); do_warn( -_("inode 0x%" PRIu64 "bmap block 0x%" PRIu64 " claimed, state is %d\n"), +_("inode %" PRIu64 "bmap block 0x%" PRIx64 " claimed, state is %d\n"), ino, bno, state); break; case XR_E_MULT: case XR_E_INUSE_FS: set_bmap(agno, agbno, XR_E_MULT); do_warn( -_("inode 0x%" PRIu64 " bmap block 0x%" PRIu64 " claimed, state is %d\n"), +_("inode %" PRIu64 " bmap block 0x%" PRIx64 " claimed, state is %d\n"), ino, bno, state); /* * if we made it to here, this is probably a bmap block @@ -311,7 +311,7 @@ _("inode 0x%" PRIu64 " bmap block 0x%" P case XR_E_BAD_STATE: default: do_warn( -_("bad state %d, inode 0x%" PRIu64 " bmap block 0x%" PRIu64 "\n"), +_("bad state %d, inode 0x%" PRIu64 " bmap block 0x%" PRIx64 "\n"), state, ino, bno); break; } @@ -338,7 +338,7 @@ _("bad state %d, inode 0x%" PRIu64 " bma if (numrecs > mp->m_bmap_dmxr[0] || (isroot == 0 && numrecs < mp->m_bmap_dmnr[0])) { do_warn( -_("inode 0x%" PRIu64 " bad # of bmap records (%u, min - %u, max - %u)\n"), +_("inode %" PRIu64 " bad # of bmap records (%u, min - %u, max - %u)\n"), ino, numrecs, mp->m_bmap_dmnr[0], mp->m_bmap_dmxr[0]); return(1); --WIyZ46R2i8wDzkSu-- From BATV+a576dd851a46f4afe15d+2995+infradead.org+hch@bombadil.srs.infradead.org Sat Nov 5 04:32:21 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA59WKcM148444 for ; Sat, 5 Nov 2011 04:32:21 -0500 X-ASG-Debug-ID: 1320485539-255e03da0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E24DF223161 for ; Sat, 5 Nov 2011 02:32:19 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id W5tbXh72yVfYSQRp for ; Sat, 05 Nov 2011 02:32:19 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMcbr-0001Nc-76; Sat, 05 Nov 2011 09:32:19 +0000 Date: Sat, 5 Nov 2011 05:32:19 -0400 From: Christoph Hellwig To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org X-ASG-Orig-Subj: XFS status update for October 2011 Subject: XFS status update for October 2011 Message-ID: <20111105093219.GA5303@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320485539 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79375 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean October finally saw the delayed release of Linux 3.1, which is a fairly boring release as XFS is concerned. In addition to a few bug fixes and cleanups the biggest item is an XFS-internal re organization of the source files, dropping all sub directories under fs/xfs. Due to the long Linux 3.1 release cycle development for 3.3 has already started full steam in October while adding a few more small optimization and fixes to the development tree for Linux 3.2, and merging that tree into mainline. Notable items for Linux 3.2 are speedup for parallel O_DIRECT reads and writes on high IOPS devices, optimizations for fsync(2) on directories and sync(2) latency, as well as further small improvements for metadata performance on highly parallel workloads. On the user space side xfsprogs saw a few more xfs_repair fixes, as well as some updates of mount point handling for the xfs_quota tools, which together with the updates from the last months was published in form of the xfsprogs 3.1.6 release. This was accompanied by an xfsdump 3.0.6 release, which does not include any new updates in October, but lots of work from the previous month. Xfstests saw two additional test cases and various fixes, and it's first versioned release ever. From BATV+a576dd851a46f4afe15d+2995+infradead.org+hch@bombadil.srs.infradead.org Sat Nov 5 04:33:20 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA59XKil148572 for ; Sat, 5 Nov 2011 04:33:20 -0500 X-ASG-Debug-ID: 1320485599-79fa02aa0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8770C223485 for ; Sat, 5 Nov 2011 02:33:19 -0700 (PDT) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id AtQbrrQFrZ6NyR4G for ; Sat, 05 Nov 2011 02:33:19 -0700 (PDT) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RMccn-0001P9-FA; Sat, 05 Nov 2011 09:33:17 +0000 Date: Sat, 5 Nov 2011 05:33:17 -0400 From: Christoph Hellwig To: Jakub Bogusz Cc: xfs@oss.sgi.com, Christoph Hellwig X-ASG-Orig-Subj: Re: [PATCH v2] xfsprogs: fix various incorrect printf formats Subject: Re: [PATCH v2] xfsprogs: fix various incorrect printf formats Message-ID: <20111105093317.GB5303@infradead.org> References: <20111025115233.GB22103@infradead.org> <20111025155916.GA19818@stranger.qboosh.pl> <20111102092012.GA3808@infradead.org> <20111105090029.GA15258@stranger.qboosh.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111105090029.GA15258@stranger.qboosh.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320485599 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79375 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean > > do_warn( > > -_("inode 0x%" PRIu64 "bmap block 0x%" PRIu64 " claimed, state is %d\n"), > > +_("inode 0x%" PRIu64 "bmap block 0x%" PRIx64 " claimed, state is %d\n"), > > ino, bno, state); > > break; > > case XR_E_MULT: > > case XR_E_INUSE_FS: > > set_bmap(agno, agbno, XR_E_MULT); > > do_warn( > > -_("inode 0x%" PRIu64 " bmap block 0x%" PRIu64 " claimed, state is %d\n"), > > +_("inode 0x%" PRIu64 " bmap block 0x%" PRIx64 " claimed, state is %d\n"), > > ino, bno, state); > > /* > > * if we made it to here, this is probably a bmap block > > These two entries still need fix in the first format strings of both > msgids... > > I'm attaching my version of the whole patch. I've already put the previous version in. I'll commit the updates attributed to you. Sorry for the mess, and thanks a lot for looking over this. From martin.rusko@gmail.com Sat Nov 5 15:07:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA5K77dn180737 for ; Sat, 5 Nov 2011 15:07:08 -0500 X-ASG-Debug-ID: 1320523626-17cf01170000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-gx0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 34BE0222C84 for ; Sat, 5 Nov 2011 13:07:06 -0700 (PDT) Received: from mail-gx0-f181.google.com (mail-gx0-f181.google.com [209.85.161.181]) by cuda.sgi.com with ESMTP id Y8VaLO6j3FxmuDq3 for ; Sat, 05 Nov 2011 13:07:06 -0700 (PDT) Received: by ggdk6 with SMTP id k6so920935ggd.26 for ; Sat, 05 Nov 2011 13:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=ieid+w2eNB+Cx9kZtenzGSXg7hZ66WnKZgdEl4Ddhhs=; b=LXSWrAK2fszL3JtlEWWIefg3lcd0Nn9sdl93MGCl1241X1wjQdcN0HK7CzhgDlkMqm sKOkCIsO4keLbK7l2UwJLrVMjzKVbifX6qbsKAf5ik8ZX9DaE3ss4MGxT6kFNBIiMWAE 5ADa3cM6B7xwYF3tyM9U4Cy8th8kfFfJt+2lE= MIME-Version: 1.0 Received: by 10.42.156.9 with SMTP id x9mr27699309icw.42.1320523625886; Sat, 05 Nov 2011 13:07:05 -0700 (PDT) Received: by 10.42.229.68 with HTTP; Sat, 5 Nov 2011 13:07:05 -0700 (PDT) Date: Sat, 5 Nov 2011 21:07:05 +0100 Message-ID: X-ASG-Orig-Subj: Online TRIM/discard performance impact Subject: Online TRIM/discard performance impact From: Martin Rusko To: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-gx0-f181.google.com[209.85.161.181] X-Barracuda-Start-Time: 1320523627 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79417 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello, I wanted to ask if following performance drop with online discard enabled for XFS is normal. I went through mailing list archives and read information from Luk=E1=A8's page (http://people.redhat.com/lczerner/discard/) so I was prepared for some performance penalty. But this big? --> Filesystem is mounted with noatime and discard options ... /dev/sdb2 on /mnt/test-xfs type xfs (rw,noatime,discard) --> Deleting freshly unpacked linux kernel sources ... # time rm -rf linux-3.0.8 real 4m50.155s user 0m0.012s sys 0m0.940s =3D=3D> It took almost 5 minutes! --> Deleting same freshly unpacked linux kernel sources this time with filesystem mounted without 'discard' option ... /dev/sdb2 on /mnt/test-xfs type xfs (rw,noatime) # time rm -rf linux-3.0.8 real 0m1.023s user 0m0.024s sys 0m0.896s =3D=3D> It took a second and something. SSD drive in question is one of the latest with SF-2281 chipset. I expected, that TRIM function will just schedule sectors for garbage collection, which happens some time later (during which drive can be potentially slower). Trying the same tests with ext4 filesystem, it got following numbers. --> ext4 with 'discard' option ... /dev/sdb3 on /mnt/test-ext4 type ext4 (rw,noatime,discard) # time rm -rf linux-3.0.8 real 0m0.486s user 0m0.012s sys 0m0.468s --> ext4 without 'discard' option ... /dev/sdb3 on /mnt/test-ext4 type ext4 (rw,noatime) root@layla:/mnt/test-ext4# time rm -rf linux-3.0.8 real 0m0.483s user 0m0.020s sys 0m0.460s Tests were repeated several times and it was consistent. Deleting files on XFS filesystem mounted with 'discard' option was painfully slow. Kernel version was ... Linux layla 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:56:25 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux Is this expected behavior? Best Regards, Martin From clintm@kshs.lincs.sch.uk Sat Nov 5 19:23:03 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA60N3ms193642 for ; Sat, 5 Nov 2011 19:23:03 -0500 X-ASG-Debug-ID: 1320538980-25ff03000000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from lsn.lincs.sch.uk (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7E999222E21 for ; Sat, 5 Nov 2011 17:23:00 -0700 (PDT) Received: from lsn.lincs.sch.uk (tattershallprimary.lincs.sch.uk [92.43.66.11]) by cuda.sgi.com with ESMTP id l4riZcr9r4DmlJcP for ; Sat, 05 Nov 2011 17:23:00 -0700 (PDT) Received: from ([10.33.177.7]) by l2-irma-002.lsn.lincs.sch.uk with ESMTP with TLS id B3R28J1.69077794; Sun, 06 Nov 2011 00:21:56 +0000 Received: from SL-EX01.sleaford.local ([::1]) by SL-EX01.sleaford.local ([::1]) with mapi id 14.01.0289.001; Sun, 6 Nov 2011 00:21:59 +0000 From: Matthew Clint X-ASG-Orig-Subj: Your mailbox has exceeded its limit Subject: Your mailbox has exceeded its limit Thread-Topic: Your mailbox has exceeded its limit Thread-Index: AcycGhiWHbukZf35R5ilGSYiVqVwJA== Date: Sun, 6 Nov 2011 00:21:58 +0000 Message-ID: <6EA9400DDC1A184FB9585DC51380859920209650@SL-EX01.sleaford.local> Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [41.184.30.28] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: tattershallprimary.lincs.sch.uk[92.43.66.11] X-Barracuda-Start-Time: 1320538981 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4975 1.0000 0.0000 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 1.23 X-Barracuda-Spam-Status: No, SCORE=1.23 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MISSING_HEADERS, THREAD_INDEX, THREAD_TOPIC, TO_CC_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79435 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig== 0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)... 1.21 MISSING_HEADERS Missing To: header 0.00 TO_CC_NONE No To: or Cc: header To: undisclosed-recipients:; X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Your mailbox has exceeded its limit; your webmail is currently running 99.7= % of its Quota limit of 100%. You cannot send or receive email properly unt= il you have updated your webmail account.To update your webmail account, cl= ick on the link below and paste in your browser to request for upgrade. https://docs.google.com/spreadsheet/viewform?formkey=3DdFlBeHh2empQYU5aaWJZ= TWNYMm45b2c6MQ We are sincerely sorry for any inconvenience this might cause you; we tend = to serve you better.Thanks for your co-operation. Webmail Update Team. =A92011= From vapier@gentoo.org Sat Nov 5 23:46:52 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA64kpd2206385 for ; Sat, 5 Nov 2011 23:46:52 -0500 X-ASG-Debug-ID: 1320554809-1deb002c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp.gentoo.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BC17C12627DD for ; Sat, 5 Nov 2011 21:46:49 -0700 (PDT) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by cuda.sgi.com with ESMTP id mIaGRnAN80XzSZwu for ; Sat, 05 Nov 2011 21:46:49 -0700 (PDT) Received: from localhost.localdomain (localhost [127.0.0.1]) by smtp.gentoo.org (Postfix) with ESMTP id 6F5DD1B403B; Sun, 6 Nov 2011 04:46:48 +0000 (UTC) From: Mike Frysinger To: xfs@oss.sgi.com Cc: wkendall@sgi.com, aelder@sgi.com X-ASG-Orig-Subj: [PATCH] path: fix 1 byte overflow with empty lists Subject: [PATCH] path: fix 1 byte overflow with empty lists Date: Sun, 6 Nov 2011 00:46:57 -0400 Message-Id: <1320554817-2594-1-git-send-email-vapier@gentoo.org> X-Mailer: git-send-email 1.7.6.1 X-Barracuda-Connect: smtp.gentoo.org[140.211.166.183] X-Barracuda-Start-Time: 1320554810 X-Barracuda-Bayes: INNOCENT GLOBAL 0.1460 1.0000 -1.1240 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.12 X-Barracuda-Spam-Status: No, SCORE=-1.12 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79452 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean If pap->pa_cnt is 0, then the local buffer is allocated as 1 byte, but the code then writes two bytes to it '/' and '\0'. Signed-off-by: Mike Frysinger --- common/path.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/common/path.c b/common/path.c index 66320de..ca24f6a 100644 --- a/common/path.c +++ b/common/path.c @@ -285,6 +285,8 @@ pa_gen( pa_t *pap ) for ( i = 0 ; i < pap->pa_cnt ; i++ ) { sz += strlen( pap->pa_array[ i ] ) + 1; } + if ( i == 0 ) + sz++; sz++; retp = ( char * )malloc( sz ); -- 1.7.6.1 From arekm@maven.pl Sun Nov 6 16:19:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,J_CHICKENPOX_45, MIME_8BIT_HEADER autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA6MJ7la012359 for ; Sun, 6 Nov 2011 16:19:08 -0600 X-ASG-Debug-ID: 1320617943-523b01310000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0143216217DC for ; Sun, 6 Nov 2011 14:19:03 -0800 (PST) Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com [209.85.161.53]) by cuda.sgi.com with ESMTP id ZYKBMlipH4MbGpMU for ; Sun, 06 Nov 2011 14:19:03 -0800 (PST) Received: by faan15 with SMTP id n15so1778853faa.26 for ; Sun, 06 Nov 2011 14:19:03 -0800 (PST) Received: by 10.223.5.201 with SMTP id 9mr43145066faw.5.1320617943092; Sun, 06 Nov 2011 14:19:03 -0800 (PST) Received: from t400.localnet (89-69-21-174.dynamic.chello.pl. [89.69.21.174]) by mx.google.com with ESMTPS id o16sm28572011fag.21.2011.11.06.14.19.02 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 06 Nov 2011 14:19:02 -0800 (PST) From: Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?= To: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Subject: Re: xfs_repair fails with corrupt dinode 17491441757, extent total = 1, nblocks = 0. This is a bug. Date: Sun, 6 Nov 2011 23:19:01 +0100 User-Agent: KMail/1.13.7 (Linux/3.1.0-09497-g5c8a0fb; KDE/4.7.3; x86_64; ; ) References: <201110311156.20421.arekm@maven.pl> In-Reply-To: <201110311156.20421.arekm@maven.pl> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201111062319.01375.arekm@maven.pl> X-Barracuda-Connect: mail-fx0-f53.google.com[209.85.161.53] X-Barracuda-Start-Time: 1320617945 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79520 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Monday 31 of October 2011, Arkadiusz Mi=C5=9Bkiewicz wrote: > xfs_repair version 3.1.6 >=20 > disconnected inode 17491441754, moving to lost+found > disconnected inode 17491441755, moving to lost+found > disconnected inode 17491441756, moving to lost+found > disconnected inode 17491441757, moving to lost+found > corrupt dinode 17491441757, extent total =3D 1, nblocks =3D 0. This is a= bug. > Please capture the filesystem metadata with xfs_metadump and > report it to xfs@oss.sgi.com. > cache_node_purge: refcount was 1, not zero (node=3D0x21450c90) >=20 > fatal error -- 117 - couldn't iget disconnected inode > In meantime I'll probably make ugly hack by making "couldn't iget > disconnected inode" non fatal, so repair will be able to finish. With this one repair finished and then while repairing for second time (this time without hack) I got glibc catching invalid free: name create failed in ino 17873999459 (117), filesystem may be out of space bad hash table for directory inode 17875023137 (brak wpisu danych): przebud= owano rebuilding directory inode 17875023137 *** glibc detected *** /sbin/xfs_repair: free(): invalid next size (normal)= : 0x00007f9ffcdc4c00 *** =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D /lib64/libc.so.6(+0x758b6)[0x7fa146a6b8b6] /lib64/libxfs.so.0(libxfs_getbufr+0x81)[0x7fa14738de81] /lib64/libxfs.so.0(cache_node_get+0xca)[0x7fa14738a39a] /lib64/libxfs.so.0(libxfs_getbuf+0x29)[0x7fa14738dfc9] /sbin/xfs_repair[0x425cd0] /sbin/xfs_repair[0x426aa6] /lib64/libpthread.so.0(+0x7ed5)[0x7fa146d8ced5] /lib64/libc.so.6(clone+0x6d)[0x7fa146acfe5d] =3D=3D=3D=3D=3D=3D=3D Memory map: =3D=3D=3D=3D=3D=3D=3D=3D 00400000-0043e000 r-xp 00000000 08:12 125854083 /s= bin/xfs_repair 0043f000-00440000 r--p 0003e000 08:12 125854083 /s= bin/xfs_repair 00440000-00441000 rw-p 0003f000 08:12 125854083 /s= bin/xfs_repair 00441000-2aa06000 rw-p 00000000 00:00 0 [h= eap] 7f9f00000000-7f9f01123000 rw-p 00000000 00:00 0 7f9f01123000-7f9f04000000 ---p 00000000 00:00 0 7f9f077ff000-7f9f07800000 ---p 00000000 00:00 0 7f9f07800000-7f9f08000000 rw-p 00000000 00:00 0 [...] =2D-=20 Arkadiusz Mi=C5=9Bkiewicz PLD/Linux Team arekm / maven.pl http://ftp.pld-linux.org/ From david@fromorbit.com Sun Nov 6 17:06:38 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA6N6bCM014865 for ; Sun, 6 Nov 2011 17:06:38 -0600 X-ASG-Debug-ID: 1320620795-641501a50000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail04.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9CD7A225611 for ; Sun, 6 Nov 2011 15:06:35 -0800 (PST) Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id urQDvmDTiUhmoL5x for ; Sun, 06 Nov 2011 15:06:35 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAGgRt055LIil/2dsb2JhbABDqX+BBoFyAQEFOhwjEAgDGC4UDRgDIRO8UxKINmMEnl+HQg Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail04.adl6.internode.on.net with ESMTP; 07 Nov 2011 09:36:33 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNBnL-0001ky-Ml; Mon, 07 Nov 2011 10:06:31 +1100 Date: Mon, 7 Nov 2011 10:06:31 +1100 From: Dave Chinner To: Eryu Guan Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] xfstests: 075 112 redirect stderr of fsx to /dev/null too Subject: Re: [PATCH] xfstests: 075 112 redirect stderr of fsx to /dev/null too Message-ID: <20111106230631.GD5534@dastard> References: <1319941555-4405-1-git-send-email-guaneryu@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1319941555-4405-1-git-send-email-guaneryu@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1320620796 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0020 1.0000 -2.0083 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79523 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sun, Oct 30, 2011 at 10:25:55AM +0800, Eryu Guan wrote: > In commit c18bf42de9d5d5fa05025754df1ff63f2147bd12 a warning message > will be printed to stderr if filesystem doesn't support fallocate(2). > The warning message will fail these tests though fsx returns SUCCESS. > > FSTYP -- ext3 > PLATFORM -- Linux/i686 debian-dev 3.1.0+ > MKFS_OPTIONS -- /dev/sda3 > MOUNT_OPTIONS -- -o acl,user_xattr /dev/sda3 /mnt/sda3 > > 075 12s ... - output mismatch (see 075.out.bad) > --- 075.out 2011-10-28 21:35:41.872573530 +0800 > +++ 075.out.bad 2011-10-30 10:31:02.607058029 +0800 > @@ -4,6 +4,9 @@ > ----------------------------------------------- > fsx.0 : -d -N numops -S 0 > ----------------------------------------------- > +fsx: main: filesystem does not support fallocate, disabling > +: Operation not supported > > So redirect stderr of fsx to /dev/null too. > > Signed-off-by: Eryu Guan > --- > 075 | 2 +- > 112 | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/075 b/075 > index 0f0a6ed..87d273f 100755 > --- a/075 > +++ b/075 > @@ -77,7 +77,7 @@ _do_test() > > # This cd and use of -P gets full debug on $here (not TEST_DEV) > cd $out > - if ! $here/ltp/fsx $_param -P $here $seq.$_n >/dev/null > + if ! $here/ltp/fsx $_param -P $here $seq.$_n &>/dev/null > then > echo " fsx ($_param) failed, $? - compare $seq.$_n.{good,bad,fsxlog}" > mv $out/$seq.$_n $here/$seq.$_n.full > diff --git a/112 b/112 > index 557350a..763b17a 100755 > --- a/112 > +++ b/112 > @@ -77,7 +77,7 @@ _do_test() > > # This cd and use of -P gets full debug on $here (not TEST_DEV) > cd $out > - if ! $here/ltp/fsx $_param -P $here $seq.$_n >/dev/null > + if ! $here/ltp/fsx $_param -P $here $seq.$_n &>/dev/null > then > echo " fsx ($_param) returned $? - see $seq.$_n.full" > mv $seq.$_n.fsxlog $here/$seq.$_n.full If this hasn't been committed, the correct fix is to pass the "-q" option to fsx to tell it not to emit these messages. We still really want any other output to stderr to cause a test failure.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From schumi.han@gmail.com Sun Nov 6 20:12:32 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,T_DKIM_INVALID,T_TO_NO_BRKTS_FREEMAIL autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA72CW2e024806 for ; Sun, 6 Nov 2011 20:12:32 -0600 X-ASG-Debug-ID: 1320631950-4095015c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-gx0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 70001225E92 for ; Sun, 6 Nov 2011 18:12:30 -0800 (PST) Received: from mail-gx0-f181.google.com (mail-gx0-f181.google.com [209.85.161.181]) by cuda.sgi.com with ESMTP id X5SYsWwB76xSOHSJ for ; Sun, 06 Nov 2011 18:12:30 -0800 (PST) Received: by ggdk6 with SMTP id k6so2671390ggd.26 for ; Sun, 06 Nov 2011 18:12:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=NHpDhRfDSfwrgKhpchkcfYjTe91vua0WPpYf4ROu8gw=; b=VCqrWnryeFp9lxyL/nPwOqubgtns2o0Bt7egTFehQrRNPzb/NEyjH0vbqw3kAKmp+w HpoyaikbXm+snEF7CVajuwwzsdJ/T4EQ5BFR5sIZCaDdIhXliTZxAslU/reyRuMU6V8W GhOABUVgoCoyiD/J2AT7EfMYb/yb78ZEJ5mk8= MIME-Version: 1.0 Received: by 10.236.185.198 with SMTP id u46mr31286510yhm.113.1320631950327; Sun, 06 Nov 2011 18:12:30 -0800 (PST) Received: by 10.236.29.196 with HTTP; Sun, 6 Nov 2011 18:12:30 -0800 (PST) Date: Mon, 7 Nov 2011 10:12:30 +0800 Message-ID: X-ASG-Orig-Subj: Advanced format disk Subject: Advanced format disk From: Zhu Han To: xfs@oss.sgi.com Content-Type: multipart/alternative; boundary=20cf3056409f473d6e04b11b969a X-Barracuda-Connect: mail-gx0-f181.google.com[209.85.161.181] X-Barracuda-Start-Time: 1320631951 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79535 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.00 HTML_MESSAGE BODY: HTML included in message X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --20cf3056409f473d6e04b11b969a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, I got several advanced format disk, whose physical size is 4096 bytes, but its logical size is 512 bytes: $ sudo fdisk -l /dev/sdb Disk /dev/sdb: 750.2 GB, 750156374016 bytes 255 heads, 63 sectors/track, 91201 cylinders Units =3D cylinders of 16065 * 512 =3D 8225280 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disk identifier: 0x00000000 Disk /dev/sdb doesn't contain a valid partition table Is there any special tuning knob I should notice before formating it? IMHO, set the sector size as 4096 bytes is enough. The default block size is 4096 bytes. $ sudo xfs_info /dev/sdb meta-data=3D/dev/sdb isize=3D256 agcount=3D4, agsize=3D45785912 = blks =3D sectsz=3D4096 attr=3D2 data =3D bsize=3D4096 blocks=3D183143646, imaxpct=3D25 =3D sunit=3D0 swidth=3D0 blks naming =3D version 2 bsize=3D4096 ascii-ci=3D0 log =3Dinternal bsize=3D4096 blocks=3D89425, ver= sion=3D2 =3D sectsz=3D4096 sunit=3D1 blks, lazy-count=3D1 realtime =3Dnone extsz=3D4096 blocks=3D0, rtextents= =3D0 best regards, =E9=9F=A9=E7=AB=B9(Zhu Han) --20cf3056409f473d6e04b11b969a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: base64 SGksPGJyPjxicj5JIGdvdCBzZXZlcmFsIGFkdmFuY2VkIGZvcm1hdCBkaXNrLCB3aG9zZSBwaHlz aWNhbCBzaXplIGlzIDQwOTYgYnl0ZXMsIGJ1dCBpdHMgbG9naWNhbCBzaXplIGlzIDUxMiBieXRl czo8YnI+JCBzdWRvIGZkaXNrIC1sIC9kZXYvc2RiPGJyPjxicj5EaXNrIC9kZXYvc2RiOiA3NTAu MiBHQiwgNzUwMTU2Mzc0MDE2IGJ5dGVzPGJyPjI1NSBoZWFkcywgNjMgc2VjdG9ycy90cmFjaywg OTEyMDEgY3lsaW5kZXJzPGJyPgpVbml0cyA9IGN5bGluZGVycyBvZiAxNjA2NSAqIDUxMiA9IDgy MjUyODAgYnl0ZXM8YnI+U2VjdG9yIHNpemUgKGxvZ2ljYWwvcGh5c2ljYWwpOiA1MTIgYnl0ZXMg LyA0MDk2IGJ5dGVzPGJyPkkvTyBzaXplIChtaW5pbXVtL29wdGltYWwpOiA0MDk2IGJ5dGVzIC8g NDA5NiBieXRlczxicj5EaXNrIGlkZW50aWZpZXI6IDB4MDAwMDAwMDA8YnI+PGJyPkRpc2sgL2Rl di9zZGIgZG9lc24mIzM5O3QgY29udGFpbiBhIHZhbGlkIHBhcnRpdGlvbiB0YWJsZTxicj4KPGJy PklzIHRoZXJlIGFueSBzcGVjaWFsIHR1bmluZyBrbm9iIEkgc2hvdWxkIG5vdGljZSBiZWZvcmUg Zm9ybWF0aW5nIGl0PyBJTUhPLCBzZXQgdGhlIHNlY3RvciBzaXplIGFzIDQwOTYgYnl0ZXMgaXMg ZW5vdWdoLiBUaGUgZGVmYXVsdCBibG9jayBzaXplIGlzIDQwOTYgYnl0ZXMuPGJyPiQgc3VkbyB4 ZnNfaW5mbyAvZGV2L3NkYjxicj5tZXRhLWRhdGE9L2Rldi9zZGLCoMKgwqDCoMKgwqDCoMKgIGlz aXplPTI1NsKgwqDCoCBhZ2NvdW50PTQsIGFnc2l6ZT00NTc4NTkxMiBibGtzPGJyPgrCoMKgwqDC oMKgwqDCoCDCoCDCoCDCoCDCoCDCoCA9wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgc2VjdHN6PTQwOTbCoCBhdHRyPTI8YnI+ZGF0YcKgwqDCoCDCoCDCoCDCoCA9 wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgYnNpemU9NDA5NsKg wqAgYmxvY2tzPTE4MzE0MzY0NiwgaW1heHBjdD0yNTxicj7CoMKgwqDCoMKgwqDCoCDCoCDCoCDC oCDCoCDCoCA9wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3Vu aXQ9MMKgwqDCoMKgwqAgc3dpZHRoPTAgYmxrczxicj5uYW1pbmfCoMKgwqDCoMKgID3CoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHZlcnNpb24gMsKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIGJzaXplPTQwOTbCoMKgIGFzY2lpLWNpPTA8YnI+CmxvZ8KgwqDC oMKgwqDCoMKgwqDCoMKgwqAgPWludGVybmFswqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBi c2l6ZT00MDk2wqDCoCBibG9ja3M9ODk0MjUsIHZlcnNpb249Mjxicj7CoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCA9wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgc2VjdHN6PTQwOTbCoCBzdW5pdD0xIGJsa3MsIGxhenktY291bnQ9MTxicj5yZWFsdGlt ZcKgwqAgPW5vbmXCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZXh0c3o9NDA5 NsKgwqAgYmxvY2tzPTAsIHJ0ZXh0ZW50cz0wPGJyPgo8YnI+PGJyIGNsZWFyPSJhbGwiPmJlc3Qg cmVnYXJkcyw8YnI+6Z+p56u5KFpodSBIYW4pPGJyPjxicj4K --20cf3056409f473d6e04b11b969a-- From rjevskiy@gmail.com Mon Nov 7 00:15:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID, T_MIME_NO_TEXT,T_TVD_MIME_NO_HEADERS autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA76FfaK040059 for ; Mon, 7 Nov 2011 00:15:41 -0600 X-ASG-Debug-ID: 1320646539-3dd603ba0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 65B132264EE for ; Sun, 6 Nov 2011 22:15:39 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id HUnQHpT0Xcod5Q0S for ; Sun, 06 Nov 2011 22:15:39 -0800 (PST) Received: by bkaq10 with SMTP id q10so1718093bka.26 for ; Sun, 06 Nov 2011 22:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=bbmL/bN1670S1XOtb+Eow8pz1LHkBvemI5BDpnZWjnE=; b=dkv/FT6I0uBzxzgMb+HM1Hi0BfVTN2Rykf0FqsON45eCo6Ht/dYewI2HWb/7F/GOed J6hZnvWZHYmH06zlegKuqfkDbtGqTWGOutml9cvlv5ie5PszeaT4gc4yrjvqnGLKM7Eu jhnPZBKjs5wd1PwJLBozh3OjSCIfkXZXqCObw= Received: by 10.205.81.141 with SMTP id zy13mr18325765bkb.50.1320646538929; Sun, 06 Nov 2011 22:15:38 -0800 (PST) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id r12sm16427624bkw.5.2011.11.06.22.15.35 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 06 Nov 2011 22:15:36 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Christoph Hellwig Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 7/9] xfstests: add a new quota test that runs fsstress under ENOSPC conditions In-Reply-To: <20111103193935.GA23014@infradead.org> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> <1320342953-7973-7-git-send-email-dmonakhov@openvz.org> <20111103193935.GA23014@infradead.org> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Mon, 07 Nov 2011 10:15:33 +0400 Message-ID: <871utkmpa2.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320646540 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.82 X-Barracuda-Spam-Status: No, SCORE=-1.82 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA590, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79551 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.20 BSF_SC0_SA590 Custom Rule SA590 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --=-=-= On Thu, 3 Nov 2011 15:39:35 -0400, Christoph Hellwig wrote: > On Thu, Nov 03, 2011 at 09:55:51PM +0400, Dmitry Monakhov wrote: > > - Same as 264 but with quota enabled. > > 269 is the new 270 > > > - IO performed from $qa_user user > > - fsstress granted with CAP_CHOWN capability. > > This fails in my test VM because it doesn't have setcap. If you > want to use additional non-standard tools please do a _notrun if > they aren't present. > > I don't quite understand why it would need CAP_CHOWN - chowns > from owned files do not require it, and the test runs fine with > the setcap line commented out. Yes, but fsstress want chown to random uid/gid int inode_change_ok(..) ... /* Make sure a caller can chown. */ if ((ia_valid & ATTR_UID) && (current_fsuid() != inode->i_uid || attr->ia_uid != inode->i_uid) && !capable(CAP_CHOWN)) return -EPERM; >From my experience fssstess with chown enabled is the most valuable quota accounting test. W/o chown this is just an useless time consumer. > > I'll commit it as-is for now, but unless I get some feedback on > why the setcap is require I'll add another commit to remove it ASAP. --=-=-= Content-Disposition: inline; filename=0001-xfstests-270-th-test-want-use-setcap-8.patch >From f8611f3bbc5ea2ac3672ee6c88d2d43d69351d49 Mon Sep 17 00:00:00 2001 From: Dmitry Monakhov Date: Mon, 7 Nov 2011 08:07:57 +0300 Subject: [PATCH] xfstests: 270'th test want use setcap(8) It wants to grant CAP_CHOWN to fsstress, without this capability chown(2) will fail due to EPERM. Since chown(2) is one of the most error prone places of quota accounting this makes 270'th test almost useless, so it is better to simply skip it. Signed-off-by: Dmitry Monakhov --- 270 | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/270 b/270 index ed1e31e..90e24f5 100755 --- a/270 +++ b/270 @@ -52,6 +52,9 @@ _workout() echo "fsstress $args" >> $here/$seq.full # Grant chown capability cp $FSSTRESS_PROG $tmp.fsstress.bin + if [ "`whereis setcap`" == "setcap:" ]; then + _notrun "setcap not installed." + fi setcap cap_chown=epi $tmp.fsstress.bin (su $qa_user -c "$tmp.fsstress.bin $args" &) > /dev/null 2>&1 -- 1.7.1 --=-=-= > > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html --=-=-=-- From BATV+b5242edae5d5de85ed24+2997+infradead.org+hch@bombadil.srs.infradead.org Mon Nov 7 01:38:13 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA77cDsA045625 for ; Mon, 7 Nov 2011 01:38:13 -0600 X-ASG-Debug-ID: 1320651491-137d03010000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 801DE162222E for ; Sun, 6 Nov 2011 23:38:11 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id Guk0gsxmYRUmuBjZ for ; Sun, 06 Nov 2011 23:38:11 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNJmS-0003j0-8l; Mon, 07 Nov 2011 07:38:08 +0000 Date: Mon, 7 Nov 2011 02:38:08 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Eryu Guan , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] xfstests: 075 112 redirect stderr of fsx to /dev/null too Subject: Re: [PATCH] xfstests: 075 112 redirect stderr of fsx to /dev/null too Message-ID: <20111107073808.GA14283@infradead.org> References: <1319941555-4405-1-git-send-email-guaneryu@gmail.com> <20111106230631.GD5534@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111106230631.GD5534@dastard> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320651492 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79558 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 07, 2011 at 10:06:31AM +1100, Dave Chinner wrote: > If this hasn't been committed, the correct fix is to pass the "-q" > option to fsx to tell it not to emit these messages. We still really > want any other output to stderr to cause a test failure.... I already commited it. I'll also take care of fixing it up. From BATV+b5242edae5d5de85ed24+2997+infradead.org+hch@bombadil.srs.infradead.org Mon Nov 7 04:21:28 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7ALQsI053349 for ; Mon, 7 Nov 2011 04:21:28 -0600 X-ASG-Debug-ID: 1320661281-393501150000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 848B11CEA842 for ; Mon, 7 Nov 2011 02:21:22 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id sJFcmeK8KbFyYZqr for ; Mon, 07 Nov 2011 02:21:22 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNMKP-0002fP-J0; Mon, 07 Nov 2011 10:21:21 +0000 Date: Mon, 7 Nov 2011 05:21:21 -0500 From: Christoph Hellwig To: Martin Rusko Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: Online TRIM/discard performance impact Subject: Re: Online TRIM/discard performance impact Message-ID: <20111107102121.GA2891@infradead.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320661283 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79569 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Nov 05, 2011 at 09:07:05PM +0100, Martin Rusko wrote: > SSD drive in question is one of the latest with SF-2281 chipset. I > expected, that TRIM function will just schedule sectors for garbage > collection, which happens some time later (during which drive can be > potentially slower). Trying the same tests with ext4 filesystem, it > got following numbers. That's unfortunately not how it works - the TRIM command always drains the whole queue of outstanding requests, and thus has a fairly severe impact on performance. We could reduce it a bit by using vectored trims like Windows does, but the Linux blocklayer doesn't have the infrastructure for that yet. Your workload will remove a lot of inode clusters on XFS, while ext4 never deletes inodes. Using the ikeep mount option for XFS (can't be set during remount) should speed the performance with discard for your workload up a lot. I would still recommand to do a periodic fstrim run instead, though. From BATV+b5242edae5d5de85ed24+2997+infradead.org+hch@bombadil.srs.infradead.org Mon Nov 7 04:30:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7AUddN053651 for ; Mon, 7 Nov 2011 04:30:40 -0600 X-ASG-Debug-ID: 1320661837-36ed01090000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 362E4101E25B for ; Mon, 7 Nov 2011 02:30:37 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id OmHtYEpLHU7wAq9i for ; Mon, 07 Nov 2011 02:30:37 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNMTN-0005MY-E9 for xfs@oss.sgi.com; Mon, 07 Nov 2011 10:30:37 +0000 Date: Mon, 7 Nov 2011 05:30:37 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: xfs_getbmap assert Subject: xfs_getbmap assert Message-ID: <20111107103037.GA16213@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320661839 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79568 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean With Dmitris fsstress updates I can hit the following assert fairly regularly: [11904.943956] XFS: Assertion failed: ((iflags & BMV_IF_DELALLOC) != 0) || (map[i].br_startblock != DELAYSTARTBLOCK), file: /home/hch/work/linux-2.6/fs/xfs/xfs_bmap.c, line: 5604 which means we get an delalloc extent back from FIEMAP/GETBMAP despite asking for a flush beforehand. While we hold the iolock over the call and thus exclude new buffered writers from appearing that doesn't prevent shared writeable mmaps from creating new delalloc extents. I don't think this actually is a real issue, and a workaround would be extremly hard. For now I've just remove the assert in my tree: Index: xfs/fs/xfs/xfs_bmap.c =================================================================== --- xfs.orig/fs/xfs/xfs_bmap.c 2011-11-05 09:02:24.410021945 +0100 +++ xfs/fs/xfs/xfs_bmap.c 2011-11-05 09:03:54.036494235 +0100 @@ -5600,8 +5600,6 @@ xfs_getbmap( XFS_FSB_TO_BB(mp, map[i].br_blockcount); out[cur_ext].bmv_unused1 = 0; out[cur_ext].bmv_unused2 = 0; - ASSERT(((iflags & BMV_IF_DELALLOC) != 0) || - (map[i].br_startblock != DELAYSTARTBLOCK)); if (map[i].br_startblock == HOLESTARTBLOCK && whichfork == XFS_ATTR_FORK) { /* came to the end of attribute fork */ From dave@twin.jikos.cz Mon Nov 7 07:25:42 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7DPgWX059962 for ; Mon, 7 Nov 2011 07:25:42 -0600 X-ASG-Debug-ID: 1320672339-345700bf0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from twin.jikos.cz (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5027A1CB8A9D; Mon, 7 Nov 2011 05:25:40 -0800 (PST) Received: from twin.jikos.cz (twin.jikos.cz [89.185.236.188]) by cuda.sgi.com with ESMTP id Cc1HeRDQbPpl198m; Mon, 07 Nov 2011 05:25:40 -0800 (PST) Received: from twin.jikos.cz (dave@localhost [127.0.0.1]) by twin.jikos.cz (8.13.6/8.13.6) with ESMTP id pA7DPYrg029368 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Nov 2011 14:25:34 +0100 Received: (from dave@localhost) by twin.jikos.cz (8.13.6/8.13.6/Submit) id pA7DPXdQ029367; Mon, 7 Nov 2011 14:25:33 +0100 Date: Mon, 7 Nov 2011 14:25:33 +0100 From: David Sterba To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111107132533.GP12759@twin.jikos.cz> Reply-To: dave@jikos.cz Mail-Followup-To: Dmitry Monakhov , linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> <1320342953-7973-6-git-send-email-dmonakhov@openvz.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320342953-7973-6-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.4.2.2i X-Barracuda-Connect: twin.jikos.cz[89.185.236.188] X-Barracuda-Start-Time: 1320672341 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79580 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi, I saw this while running the newly added tests on btrfs: On Thu, Nov 03, 2011 at 09:55:50PM +0400, Dmitry Monakhov wrote: > --- /dev/null > +++ b/269 > +_workout() > +{ > + echo "" > + echo "Run fsstress" > + echo "" > + num_iterations=10 > + enospc_time=2 > + out=$SCRATCH_MNT/fsstress.$$ > + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" > + echo "fsstress $args" >> $here/$seq.full > + $FSSTRESS_PROG $args > /dev/null 2>&1 & > + pid=$! > + echo "Run dd writers in parallel" > + for ((i=0; i < num_iterations; i++)) > + do > + # File will be opened with O_TRUNC each time > + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ > + > /dev/null 2>&1 > + sleep $enospc_time > + done > + kill $pid 269 [failed, exit status 1] - output mismatch (see 269.out.bad) --- 269.out 2011-11-07 13:26:23.000000000 +0100 +++ 269.out.bad 2011-11-07 14:07:44.000000000 +0100 @@ -3,3 +3,4 @@ Run fsstress Run dd writers in parallel +./269: line 60: kill: (9754) - No such process > + wait $pid > +} and the test is reported to fail. Seems that the dd commands finish in less 2 seconds. david From rjevskiy@gmail.com Mon Nov 7 07:31:17 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7DVHuB060211 for ; Mon, 7 Nov 2011 07:31:17 -0600 X-ASG-Debug-ID: 1320672675-0fc600050000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 731FB22765D for ; Mon, 7 Nov 2011 05:31:15 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id Rm1XsS1mWfTGg4fq for ; Mon, 07 Nov 2011 05:31:15 -0800 (PST) Received: by bkaq10 with SMTP id q10so2007896bka.26 for ; Mon, 07 Nov 2011 05:31:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=zx467+4cE1CDeWfyReun8eSzIk711GVPi1TSoFXaQqY=; b=KDox3KaOTbDrHDp6bVyWqokFrt6ZrWYJRTN8bGVPF4dK7o5441Dctw8TkxHq+cqwU1 lOuY/CHrzvGlRAivAeWmooAr+wbSXMOeoDRCt1QqZRnWI8VHMUZBHwIPr9zbiqlXl/Ym ZtxgMM6CT52jujulAF1nVWW/voLPIO8xAiWTI= Received: by 10.204.140.215 with SMTP id j23mr9577846bku.52.1320672675103; Mon, 07 Nov 2011 05:31:15 -0800 (PST) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id z13sm14221038bkw.8.2011.11.07.05.31.12 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 07 Nov 2011 05:31:13 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: dave@jikos.cz Cc: linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions In-Reply-To: <20111107132533.GP12759@twin.jikos.cz> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> <1320342953-7973-6-git-send-email-dmonakhov@openvz.org> <20111107132533.GP12759@twin.jikos.cz> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Mon, 07 Nov 2011 17:31:11 +0400 Message-ID: <87sjm0kqjk.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320672676 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79580 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 7 Nov 2011 14:25:33 +0100, David Sterba wrote: > Hi, > > I saw this while running the newly added tests on btrfs: > > On Thu, Nov 03, 2011 at 09:55:50PM +0400, Dmitry Monakhov wrote: > > --- /dev/null > > +++ b/269 > > +_workout() > > +{ > > + echo "" > > + echo "Run fsstress" > > + echo "" > > + num_iterations=10 > > + enospc_time=2 > > + out=$SCRATCH_MNT/fsstress.$$ > > + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" > > + echo "fsstress $args" >> $here/$seq.full > > + $FSSTRESS_PROG $args > /dev/null 2>&1 & > > + pid=$! > > + echo "Run dd writers in parallel" > > + for ((i=0; i < num_iterations; i++)) > > + do > > + # File will be opened with O_TRUNC each time > > + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ > > + > /dev/null 2>&1 > > + sleep $enospc_time > > + done > > + kill $pid > > 269 [failed, exit status 1] - output mismatch (see 269.out.bad) > --- 269.out 2011-11-07 13:26:23.000000000 +0100 > +++ 269.out.bad 2011-11-07 14:07:44.000000000 +0100 > @@ -3,3 +3,4 @@ > Run fsstress > > Run dd writers in parallel > +./269: line 60: kill: (9754) - No such process > > > + wait $pid > > +} > > and the test is reported to fail. Seems that the dd commands finish in less 2 > seconds. Have you recompile fsstress via make? test explicitly want use new feature "-f setattr=1" > > > david > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From martin.rusko@gmail.com Mon Nov 7 08:31:47 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7EVlfi061915 for ; Mon, 7 Nov 2011 08:31:47 -0600 X-ASG-Debug-ID: 1320676305-547400400000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-qw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2604D16D049B for ; Mon, 7 Nov 2011 06:31:45 -0800 (PST) Received: from mail-qw0-f53.google.com (mail-qw0-f53.google.com [209.85.216.53]) by cuda.sgi.com with ESMTP id IX69CaWX9rNjl2oO for ; Mon, 07 Nov 2011 06:31:45 -0800 (PST) Received: by qadz32 with SMTP id z32so626677qad.26 for ; Mon, 07 Nov 2011 06:31:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Wo6W8YezsyZCFkdEIWrrt4V3tnkaz0ihIddYj0WOFpw=; b=Kmz5s4x4mW5Iq7P+jjOoWzpny+nwkAGcmh+VjQ0vRZXfrihkxyheMGt1qYwlMMuy3+ o/+90j1/eRSp/VhkDp2t//WYdBFU47Uviubzz19mAWQZvIkl8C3fMFrL59CGz5E6XNiH L7wOYUyXIXBcWuU/6CoboHhv1suXoSQ6awLF8= MIME-Version: 1.0 Received: by 10.42.155.74 with SMTP id t10mr47768082icw.49.1320676304907; Mon, 07 Nov 2011 06:31:44 -0800 (PST) Received: by 10.42.240.70 with HTTP; Mon, 7 Nov 2011 06:31:44 -0800 (PST) In-Reply-To: <20111107102121.GA2891@infradead.org> References: <20111107102121.GA2891@infradead.org> Date: Mon, 7 Nov 2011 15:31:44 +0100 Message-ID: X-ASG-Orig-Subj: Re: Online TRIM/discard performance impact Subject: Re: Online TRIM/discard performance impact From: Martin Rusko To: Christoph Hellwig Cc: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-qw0-f53.google.com[209.85.216.53] X-Barracuda-Start-Time: 1320676306 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79583 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 7, 2011 at 11:21 AM, Christoph Hellwig wrot= e: > On Sat, Nov 05, 2011 at 09:07:05PM +0100, Martin Rusko wrote: >> SSD drive in question is one of the latest with SF-2281 chipset. I >> expected, that TRIM function will just schedule sectors for garbage >> collection, which happens some time later (during which drive can be >> potentially slower). Trying the same tests with ext4 filesystem, it >> got following numbers. > > That's unfortunately not how it works - the TRIM command always drains > the whole queue of outstanding requests, and thus has a fairly severe > impact on performance. =A0We could reduce it a bit by using vectored > trims like Windows does, but the Linux blocklayer doesn't have the > infrastructure for that yet. > > Your workload will remove a lot of inode clusters on XFS, while ext4 > never deletes inodes. Using the ikeep mount option for XFS (can't be > set during remount) should speed the performance with discard for > your workload up a lot. =A0I would still recommand to do a periodic > fstrim run instead, though. > As per man page ... ikeep|noikeep When inode clusters are emptied of inodes, keep them around on the disk (ikeep) - this is the traditional XFS behaviour and is still the default for now. Using the noikeep option, inode clusters are returned to the free space pool. ... which would imply, that it should be actually rather fast by default. Even if I had set this explicitly, it didn't really help ... # mount -o noatime,discard,ikeep /dev/data/test-xfs /mnt/disk /dev/mapper/data-test--xfs on /mnt/disk type xfs (rw,noatime,discard,ikeep) # time rm -rf linux-3.0.8 real 4m47.897s user 0m0.032s sys 0m0.992s ... normally it would be deleted in a second or even faster. I'm fine to use periodic fstrim, let say once a day. But this is such a big difference, that it makes me curious if it is expected behavior or if I'm doing something terribly wrong. Any idea? Thanks! Martin From alexandernaumann@gmx.de Mon Nov 7 08:36:57 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.3 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,J_CHICKENPOX_33,J_CHICKENPOX_43,T_TO_NO_BRKTS_FREEMAIL autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7EauHB062068 for ; Mon, 7 Nov 2011 08:36:57 -0600 X-ASG-Debug-ID: 1320676613-7c30004b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mailout-de.gmx.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with SMTP id 372211CEB4F3 for ; Mon, 7 Nov 2011 06:36:54 -0800 (PST) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by cuda.sgi.com with SMTP id yH4TfrR9SqsCX1FT for ; Mon, 07 Nov 2011 06:36:54 -0800 (PST) Received: (qmail invoked by alias); 07 Nov 2011 14:36:52 -0000 Received: from p4FDC0643.dip0.t-ipconnect.de (EHLO [192.168.15.21]) [79.220.6.67] by mail.gmx.net (mp006) with SMTP; 07 Nov 2011 15:36:52 +0100 X-Authenticated: #2220048 X-Provags-ID: V01U2FsdGVkX18RoPk5s6sRHE1ch/C6WW3s0/yn4lujgKl3jAPsrx fXY9hz3mOUtBrO Message-ID: <1320676611.3192.36.camel@AlfLaptop> X-ASG-Orig-Subj: Crash with XFS Subject: Crash with XFS From: Alexander Naumann To: xfs@oss.sgi.com Date: Mon, 07 Nov 2011 15:36:51 +0100 Content-Type: multipart/alternative; boundary="=-BD+yAaNRJCDVYep6RroT" X-Mailer: Evolution 3.2.0- Mime-Version: 1.0 X-Y-GMX-Trusted: 0 X-Barracuda-Connect: mailout-de.gmx.net[213.165.64.22] X-Barracuda-Start-Time: 1320676615 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79584 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --=-BD+yAaNRJCDVYep6RroT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Hi! I would be glad if anybody can give me any hint on the following subject. I have a linux server running formatted with XFS. Afer a couple of days (about 6 or 7) I get the following crash: Nov 6 15:44:03 archive kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000044 Nov 6 15:44:03 archive kernel: IP: [] xfs_inode_ag_iterator+0x4a/0xce Nov 6 15:44:03 archive kernel: PGD 0 Nov 6 15:44:03 archive kernel: Oops: 0000 [#1] SMP Nov 6 15:44:03 archive kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:08:00.1/host2/rport-2:0-0/target2:0:0/fc_transport/target2:0:0/port_name Nov 6 15:44:03 archive kernel: CPU 13 Nov 6 15:44:03 archive kernel: Modules linked in: iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi wcte11xp wctc4xxp wct4xxp wct1xxp wcte12xp dahdi_voicebus dahdi_transcode dahdi dm_round_robin qla2xxx scsi_dh_rdac scsi_dh_emc scsi_ dh_alua scsi_dh_hp_sw af_packet ipt_REDIRECT iptable_nat nf_nat ipt_REJECT xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables ipmi_si ipmi_watchdog ipmi_devintf ipmi_msghandler fan ac ipv6 fuse dm_multipath scsi_dh psmouse usbhid hid evdev ehci_hcd uhci_hcd iTCO_wdt rtc_cmos pcspkr serio_raw iTCO_vendor_support usbcore thermal bnx2 rtc_core rtc_lib button processor thermal_sys unix Nov 6 15:44:03 archive kernel: Nov 6 15:44:03 archive kernel: Pid: 659, comm: kswapd0 Not tainted 2.6.34.7-64bit #9 0P658H/PowerEdge R910 Nov 6 15:44:03 archive kernel: RIP: 0010:[] [] xfs_inode_ag_iterator+0x4a/0xce Nov 6 15:44:03 archive kernel: RSP: 0018:ffff88085eb1bcb0 EFLAGS: 00010282 Nov 6 15:44:03 archive kernel: RAX: 0000000000000000 RBX: ffff88085841b800 RCX: 0000000000000000 Nov 6 15:44:03 archive kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000 Nov 6 15:44:03 archive kernel: RBP: ffff88085eb1bd10 R08: 0000000000000001 R09: ffff88085eb1bd24 Nov 6 15:44:03 archive kernel: R10: ffffffffff000000 R11: ffff88047d42b0e8 R12: ffff88085eb1bd24 Nov 6 15:44:03 archive kernel: R13: 0000000000000000 R14: ffff88085eb1bd24 R15: 0000000000000000 Nov 6 15:44:03 archive kernel: FS: 0000000000000000(0000) GS:ffff8800023a0000(0000) knlGS:0000000000000000 Nov 6 15:44:03 archive kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Nov 6 15:44:03 archive kernel: CR2: 0000000000000044 CR3: 00000000016cb000 CR4: 00000000000006a0 Nov 6 15:44:03 archive kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 6 15:44:03 archive kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Nov 6 15:44:03 archive kernel: Process kswapd0 (pid: 659, threadinfo ffff88085eb1a000, task ffff88085f4706b0) Nov 6 15:44:03 archive kernel: Stack: Nov 6 15:44:03 archive kernel: ffff88085eb1bce4 000000018102914a 0000000000000000 ffffffff811d268f Nov 6 15:44:03 archive kernel: <0> ffff88085841b800 0000000000000000 0000005500000283 ffff88085841b800 Nov 6 15:44:03 archive kernel: <0> ffff88085eb1bd24 00000000ffffffff 00000000000002bc 00000000000000d0 Nov 6 15:44:03 archive kernel: Call Trace: Nov 6 15:44:03 archive kernel: [] ? xfs_reclaim_inode+0x0/0x212 Nov 6 15:44:03 archive kernel: [] xfs_reclaim_inode_shrink+0x61/0x123 Nov 6 15:44:03 archive kernel: [] shrink_slab +0xd8/0x148 Nov 6 15:44:03 archive kernel: [] kswapd+0x625/0x89d Nov 6 15:44:03 archive kernel: [] ? isolate_pages_global+0x0/0x23f Nov 6 15:44:03 archive kernel: [] ? autoremove_wake_function+0x0/0x38 Nov 6 15:44:03 archive kernel: [] ? kswapd+0x0/0x89d Nov 6 15:44:03 archive kernel: [] kthread+0x7d/0x85 Nov 6 15:44:03 archive kernel: [] kernel_thread_helper+0x4/0x10 Nov 6 15:44:03 archive kernel: [] ? kthread+0x0/0x85 Nov 6 15:44:03 archive kernel: [] ? kernel_thread_helper+0x0/0x10 Nov 6 15:44:03 archive kernel: Code: c0 48 89 75 b8 89 55 b4 89 4d b0 44 89 45 ac 74 03 41 8b 01 89 45 d4 45 31 ff 45 31 ed eb 69 48 8b 7d c0 44 89 ee e8 2d d1 fe ff <83> 78 44 00 49 89 c4 75 0a 48 89 c7 e8 ad c2 fe ff eb 47 44 8b Nov 6 15:44:03 archive kernel: RIP [] xfs_inode_ag_iterator+0x4a/0xce Nov 6 15:44:03 archive kernel: RSP Nov 6 15:44:03 archive kernel: CR2: 0000000000000044 Nov 6 15:44:03 archive kernel: ---[ end trace 3bcf38b06227bae0 ]--- Or like this: Oct 22 08:30:05 archive kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000044 Oct 22 08:30:05 archive kernel: IP: [] xfs_reclaim_inode_shrink+0xc3/0x123 Oct 22 08:30:05 archive kernel: [] shrink_slab +0xcb/0x148 Oct 22 08:30:05 archive kernel: [] kswapd+0x625/0x89d Oct 22 08:30:05 archive kernel: [] ? isolate_pages_global+0x0/0x23f Oct 22 08:30:05 archive kernel: [] ? autoremove_wake_function+0x0/0x38 Oct 22 08:30:05 archive kernel: [] ? kswapd+0x0/0x89d Oct 22 08:30:05 archive kernel: [] kthread+0x7d/0x85 Oct 22 08:30:05 archive kernel: [] kernel_thread_helper+0x4/0x10 Oct 22 08:30:05 archive kernel: [] ? kthread+0x0/0x85 Oct 22 08:30:05 archive kernel: [] ? kernel_thread_helper+0x0/0x10 The system is a Dell R910 (Intek Xeon CPU E7530, 24 cores (with hyperthreading). 32GB Ram, Raid Controller is Perc H700, SAS discs, Raid 5 with 1.7TB, formatted with XFS. Kernel version 2.6.34.7 is running (64bit kernel on a 32bit system, Debian packages). There is a multipathing fibrechannel conenction to an external storage, which partition is also formatted with XFS. xfs-tools are version 3.0.4. Host 2 is one of the FC connections. Does anybody has a hint to this crash? I could not find any solution in any bugtracker so I am not sure if it is already fixed. The system itself is under load (Load-Average is about 20 / 18 / 17). Are any other informations needed? Local filesystem informations: xfs_info / meta-data=/dev/root isize=256 agcount=32, agsize=13683646 blks = sectsz=512 attr=0 data = bsize=4096 blocks=437876672, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=32768, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=65536 blocks=0, rtextents=0 Thanks in advance Alex --=-BD+yAaNRJCDVYep6RroT Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit Hi!

I would be glad if anybody can give me any hint on the following subject.
I have a linux server running formatted with XFS. Afer a couple of days (about 6 or 7) I get the following crash:


Nov  6 15:44:03 archive kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000044
Nov  6 15:44:03 archive kernel: IP: [<ffffffff811d3248>] xfs_inode_ag_iterator+0x4a/0xce
Nov  6 15:44:03 archive kernel: PGD 0
Nov  6 15:44:03 archive kernel: Oops: 0000 [#1] SMP
Nov  6 15:44:03 archive kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:1c.0/0000:08:00.1/host2/rport-2:0-0/target2:0:0/fc_transport/target2:0:0/port_name
Nov  6 15:44:03 archive kernel: CPU 13
Nov  6 15:44:03 archive kernel: Modules linked in: iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi wcte11xp wctc4xxp wct4xxp wct1xxp wcte12xp dahdi_voicebus dahdi_transcode dahdi dm_round_robin qla2xxx scsi_dh_rdac scsi_dh_emc scsi_
dh_alua scsi_dh_hp_sw af_packet ipt_REDIRECT iptable_nat nf_nat ipt_REJECT xt_tcpudp nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack iptable_filter ip_tables x_tables ipmi_si ipmi_watchdog ipmi_devintf ipmi_msghandler fan ac ipv6
fuse dm_multipath scsi_dh psmouse usbhid hid evdev ehci_hcd uhci_hcd iTCO_wdt rtc_cmos pcspkr serio_raw iTCO_vendor_support usbcore thermal bnx2 rtc_core rtc_lib button processor thermal_sys unix
Nov  6 15:44:03 archive kernel:
Nov  6 15:44:03 archive kernel: Pid: 659, comm: kswapd0 Not tainted 2.6.34.7-64bit #9 0P658H/PowerEdge R910
Nov  6 15:44:03 archive kernel: RIP: 0010:[<ffffffff811d3248>]  [<ffffffff811d3248>] xfs_inode_ag_iterator+0x4a/0xce
Nov  6 15:44:03 archive kernel: RSP: 0018:ffff88085eb1bcb0  EFLAGS: 00010282
Nov  6 15:44:03 archive kernel: RAX: 0000000000000000 RBX: ffff88085841b800 RCX: 0000000000000000
Nov  6 15:44:03 archive kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000000
Nov  6 15:44:03 archive kernel: RBP: ffff88085eb1bd10 R08: 0000000000000001 R09: ffff88085eb1bd24
Nov  6 15:44:03 archive kernel: R10: ffffffffff000000 R11: ffff88047d42b0e8 R12: ffff88085eb1bd24
Nov  6 15:44:03 archive kernel: R13: 0000000000000000 R14: ffff88085eb1bd24 R15: 0000000000000000
Nov  6 15:44:03 archive kernel: FS:  0000000000000000(0000) GS:ffff8800023a0000(0000) knlGS:0000000000000000
Nov  6 15:44:03 archive kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Nov  6 15:44:03 archive kernel: CR2: 0000000000000044 CR3: 00000000016cb000 CR4: 00000000000006a0
Nov  6 15:44:03 archive kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov  6 15:44:03 archive kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Nov  6 15:44:03 archive kernel: Process kswapd0 (pid: 659, threadinfo ffff88085eb1a000, task ffff88085f4706b0)
Nov  6 15:44:03 archive kernel: Stack:
Nov  6 15:44:03 archive kernel:  ffff88085eb1bce4 000000018102914a 0000000000000000 ffffffff811d268f
Nov  6 15:44:03 archive kernel: <0> ffff88085841b800 0000000000000000 0000005500000283 ffff88085841b800
Nov  6 15:44:03 archive kernel: <0> ffff88085eb1bd24 00000000ffffffff 00000000000002bc 00000000000000d0
Nov  6 15:44:03 archive kernel: Call Trace:
Nov  6 15:44:03 archive kernel:  [<ffffffff811d268f>] ? xfs_reclaim_inode+0x0/0x212
Nov  6 15:44:03 archive kernel:  [<ffffffff811d332d>] xfs_reclaim_inode_shrink+0x61/0x123
Nov  6 15:44:03 archive kernel:  [<ffffffff81075f45>] shrink_slab+0xd8/0x148
Nov  6 15:44:03 archive kernel:  [<ffffffff810765da>] kswapd+0x625/0x89d
Nov  6 15:44:03 archive kernel:  [<ffffffff8107445f>] ? isolate_pages_global+0x0/0x23f
Nov  6 15:44:03 archive kernel:  [<ffffffff81040882>] ? autoremove_wake_function+0x0/0x38
Nov  6 15:44:03 archive kernel:  [<ffffffff81075fb5>] ? kswapd+0x0/0x89d
Nov  6 15:44:03 archive kernel:  [<ffffffff81040472>] kthread+0x7d/0x85
Nov  6 15:44:03 archive kernel:  [<ffffffff81002c74>] kernel_thread_helper+0x4/0x10
Nov  6 15:44:03 archive kernel:  [<ffffffff810403f5>] ? kthread+0x0/0x85
Nov  6 15:44:03 archive kernel:  [<ffffffff81002c70>] ? kernel_thread_helper+0x0/0x10
Nov  6 15:44:03 archive kernel: Code: c0 48 89 75 b8 89 55 b4 89 4d b0 44 89 45 ac 74 03 41 8b 01 89 45 d4 45 31 ff 45 31 ed eb 69 48 8b 7d c0 44 89 ee e8 2d d1 fe ff <83> 78 44 00 49 89 c4 75 0a 48 89 c7 e8 ad c2 fe ff eb 47 44 8b
Nov  6 15:44:03 archive kernel: RIP  [<ffffffff811d3248>] xfs_inode_ag_iterator+0x4a/0xce
Nov  6 15:44:03 archive kernel:  RSP <ffff88085eb1bcb0>
Nov  6 15:44:03 archive kernel: CR2: 0000000000000044
Nov  6 15:44:03 archive kernel: ---[ end trace 3bcf38b06227bae0 ]---


Or like this:
Oct 22 08:30:05 archive kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000044
Oct 22 08:30:05 archive kernel: IP: [<ffffffff811d338f>] xfs_reclaim_inode_shrink+0xc3/0x123

Oct 22 08:30:05 archive kernel:  [<ffffffff81075f38>] shrink_slab+0xcb/0x148
Oct 22 08:30:05 archive kernel:  [<ffffffff810765da>] kswapd+0x625/0x89d
Oct 22 08:30:05 archive kernel:  [<ffffffff8107445f>] ? isolate_pages_global+0x0/0x23f
Oct 22 08:30:05 archive kernel:  [<ffffffff81040882>] ? autoremove_wake_function+0x0/0x38
Oct 22 08:30:05 archive kernel:  [<ffffffff81075fb5>] ? kswapd+0x0/0x89d
Oct 22 08:30:05 archive kernel:  [<ffffffff81040472>] kthread+0x7d/0x85
Oct 22 08:30:05 archive kernel:  [<ffffffff81002c74>] kernel_thread_helper+0x4/0x10
Oct 22 08:30:05 archive kernel:  [<ffffffff810403f5>] ? kthread+0x0/0x85
Oct 22 08:30:05 archive kernel:  [<ffffffff81002c70>] ? kernel_thread_helper+0x0/0x10

The system is a Dell R910 (Intek Xeon CPU E7530, 24 cores (with hyperthreading).
32GB Ram, Raid Controller is Perc H700, SAS discs, Raid 5 with 1.7TB, formatted with XFS.
Kernel version 2.6.34.7 is running (64bit kernel on a 32bit system, Debian packages).
There is a multipathing fibrechannel conenction to an external storage, which partition is also formatted with XFS.
xfs-tools are version 3.0.4.
Host 2 is one of the FC connections.

Does anybody has a hint to this crash?
I could not find any solution in any bugtracker so I am not sure if it is already fixed.

The system itself is under load (Load-Average is about 20 / 18 / 17).

Are any other informations needed?

Local filesystem informations:
xfs_info  /
meta-data=/dev/root              isize=256    agcount=32, agsize=13683646 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=437876672, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=65536  blocks=0, rtextents=0



Thanks in advance
Alex

--=-BD+yAaNRJCDVYep6RroT-- From martin.rusko@gmail.com Mon Nov 7 08:44:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Ei8Ob062530 for ; Mon, 7 Nov 2011 08:44:08 -0600 X-ASG-Debug-ID: 1320677047-0f6c01a00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-iy0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E6604227CAF for ; Mon, 7 Nov 2011 06:44:07 -0800 (PST) Received: from mail-iy0-f181.google.com (mail-iy0-f181.google.com [209.85.210.181]) by cuda.sgi.com with ESMTP id damAx9YADGZJpm8o for ; Mon, 07 Nov 2011 06:44:07 -0800 (PST) Received: by iaen33 with SMTP id n33so512051iae.26 for ; Mon, 07 Nov 2011 06:44:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=FfRvNL6mxLHz9bo5CwVTikELu5d/EIeeRqjSGcJ52lU=; b=aqaOCFeWTPs/lqocY/PcN4z5RMzZQ70zXIOmJJmFZ8lDH2mQ2g6Zb3LSZ+B5o6QjYD ulYTrMUp9sYMYhkCZgMI43RRel3FWJNa5/WDiCs63nXf019ixoQecPOJy4mzIxxxvF4I tKV/vv6Z2uE696Q70+hTIpunK7Ymjm6iyzAvc= MIME-Version: 1.0 Received: by 10.42.151.196 with SMTP id f4mr47974078icw.17.1320677047354; Mon, 07 Nov 2011 06:44:07 -0800 (PST) Received: by 10.42.240.70 with HTTP; Mon, 7 Nov 2011 06:44:07 -0800 (PST) In-Reply-To: References: Date: Mon, 7 Nov 2011 15:44:07 +0100 Message-ID: X-ASG-Orig-Subj: Re: Advanced format disk Subject: Re: Advanced format disk From: Martin Rusko To: Zhu Han Cc: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-iy0-f181.google.com[209.85.210.181] X-Barracuda-Start-Time: 1320677047 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79584 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 7, 2011 at 3:12 AM, Zhu Han wrote: > Hi, > > I got several advanced format disk, whose physical size is 4096 bytes, bu= t > its logical size is 512 bytes: > $ sudo fdisk -l /dev/sdb > > Disk /dev/sdb: 750.2 GB, 750156374016 bytes > 255 heads, 63 sectors/track, 91201 cylinders > Units =3D cylinders of 16065 * 512 =3D 8225280 bytes > Sector size (logical/physical): 512 bytes / 4096 bytes > I/O size (minimum/optimal): 4096 bytes / 4096 bytes > Disk identifier: 0x00000000 > > Disk /dev/sdb doesn't contain a valid partition table > > Is there any special tuning knob I should notice before formating it? IMH= O, > set the sector size as 4096 bytes is enough. The default block size is 40= 96 > bytes. > $ sudo xfs_info /dev/sdb > meta-data=3D/dev/sdb=A0=A0=A0=A0=A0=A0=A0=A0 isize=3D256=A0=A0=A0 agcount= =3D4, agsize=3D45785912 blks > =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 attr=3D2 > data=A0=A0=A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D183143646, > imaxpct=3D25 > =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sunit=3D0=A0=A0=A0=A0=A0 swidth=3D0= blks > naming=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 version 2=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 bsiz= e=3D4096 > ascii-ci=3D0 > log=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3Dinternal=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D89425, version=3D2 > =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 sunit=3D1 bl= ks, > lazy-count=3D1 > realtime=A0=A0 =3Dnone=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 extsz=3D4096=A0=A0 blocks=3D0, rtextents=3D0 > > One thing is to make sure, that your partitions are aligned to physical sector size. Recent fdisk will do it properly. See options '-c' (or 'c' in interactive mode) which sets DOS compatibility mode and '-u' (or 'u' in interactive mode) which sets units which fdisk uses. You want no DOS compatibility and units of sectors. Then first partition starts on 2048 sector (so 1MiB is available for GRUB for example) and it's gets things nicely aligned ... 2048 logical sectors =3D 256 physical sectors. If you are creating more than one partition, use something like +34G while specifying end of the partition (so the next one is aligned as well). Martin From schumi.han@gmail.com Mon Nov 7 08:51:19 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_00,FREEMAIL_FROM, HTML_MESSAGE,T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7EpJnv062877 for ; Mon, 7 Nov 2011 08:51:19 -0600 X-ASG-Debug-ID: 1320677477-018f00730000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-gy0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2E7871F2F883 for ; Mon, 7 Nov 2011 06:51:17 -0800 (PST) Received: from mail-gy0-f181.google.com (mail-gy0-f181.google.com [209.85.160.181]) by cuda.sgi.com with ESMTP id bzALsiieLA9vp3HF for ; Mon, 07 Nov 2011 06:51:17 -0800 (PST) Received: by gyd10 with SMTP id 10so7530228gyd.26 for ; Mon, 07 Nov 2011 06:51:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Sj0Bgh8QS59kxp1HOtZn4OzagZskcTLU0S8Av6Poo58=; b=UFpXh88UEl8ovDASM8VUmp84fy5fpXDH+urdiLldPUsd/ECYR8NKvAiazRHfMCC2TV N9/ZiNCuNeHi5K9Udg/hsKRxI5m/Cq6pa38AvFHOq/1RGh2D/dRThTZ97dqMpusZwIRn cRBGvYhGjZTaAwGr1cGvfkMnX/rLjJ3n5XeDw= MIME-Version: 1.0 Received: by 10.236.179.105 with SMTP id g69mr33711516yhm.130.1320677477526; Mon, 07 Nov 2011 06:51:17 -0800 (PST) Received: by 10.236.29.196 with HTTP; Mon, 7 Nov 2011 06:51:17 -0800 (PST) In-Reply-To: References: Date: Mon, 7 Nov 2011 22:51:17 +0800 Message-ID: X-ASG-Orig-Subj: Re: Advanced format disk Subject: Re: Advanced format disk From: Zhu Han To: Martin Rusko Cc: xfs@oss.sgi.com Content-Type: multipart/alternative; boundary=20cf303a2d13e92e1704b1262fba X-Barracuda-Connect: mail-gy0-f181.google.com[209.85.160.181] X-Barracuda-Start-Time: 1320677478 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79586 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.00 HTML_MESSAGE BODY: HTML included in message 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --20cf303a2d13e92e1704b1262fba Content-Type: text/plain; charset=ISO-8859-1 On Mon, Nov 7, 2011 at 10:44 PM, Martin Rusko wrote: > On Mon, Nov 7, 2011 at 3:12 AM, Zhu Han wrote: > > Hi, > > > > I got several advanced format disk, whose physical size is 4096 bytes, > but > > its logical size is 512 bytes: > > $ sudo fdisk -l /dev/sdb > > > > Disk /dev/sdb: 750.2 GB, 750156374016 bytes > > 255 heads, 63 sectors/track, 91201 cylinders > > Units = cylinders of 16065 * 512 = 8225280 bytes > > Sector size (logical/physical): 512 bytes / 4096 bytes > > I/O size (minimum/optimal): 4096 bytes / 4096 bytes > > Disk identifier: 0x00000000 > > > > Disk /dev/sdb doesn't contain a valid partition table > > > > Is there any special tuning knob I should notice before formating it? > IMHO, > > set the sector size as 4096 bytes is enough. The default block size is > 4096 > > bytes. > > $ sudo xfs_info /dev/sdb > > meta-data=/dev/sdb isize=256 agcount=4, agsize=45785912 blks > > = sectsz=4096 attr=2 > > data = bsize=4096 blocks=183143646, > > imaxpct=25 > > = sunit=0 swidth=0 blks > > naming = version 2 bsize=4096 > > ascii-ci=0 > > log =internal bsize=4096 blocks=89425, > version=2 > > = sectsz=4096 sunit=1 blks, > > lazy-count=1 > > realtime =none extsz=4096 blocks=0, rtextents=0 > > > > > > One thing is to make sure, that your partitions are aligned to > physical sector size. Recent fdisk will do it properly. See options > '-c' (or 'c' in interactive mode) which sets DOS compatibility mode > and '-u' (or 'u' in interactive mode) which sets units which fdisk > uses. You want no DOS compatibility and units of sectors. Then first > partition starts on 2048 sector (so 1MiB is available for GRUB for > example) and it's gets things nicely aligned ... 2048 logical sectors > = 256 physical sectors. > If the whole disk is used for the file system (it is not a bootable disk so no partition is created), can I ignore these settings safely? > > If you are creating more than one partition, use something like +34G > while specifying end of the partition (so the next one is aligned as > well). > > Martin > --20cf303a2d13e92e1704b1262fba Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable


On Mon, Nov 7, 2011 at 10:44 PM, Martin = Rusko <marti= n.rusko@gmail.com> wrote:
On Mon, Nov 7, 2011 at 3:12 AM, Zhu Han &= lt;schumi.han@gmail.com> wro= te:
> Hi,
>
> I got several advanced format disk, whose physical size is 4096 bytes,= but
> its logical size is 512 bytes:
> $ sudo fdisk -l /dev/sdb
>
> Disk /dev/sdb: 750.2 GB, 750156374016 bytes
> 255 heads, 63 sectors/track, 91201 cylinders
> Units =3D cylinders of 16065 * 512 =3D 8225280 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk identifier: 0x00000000
>
> Disk /dev/sdb doesn't contain a valid partition table
>
> Is there any special tuning knob I should notice before formating it? = IMHO,
> set the sector size as 4096 bytes is enough. The default block size is= 4096
> bytes.
> $ sudo xfs_info /dev/sdb
> meta-data=3D/dev/sdb=A0=A0=A0=A0=A0=A0=A0=A0 isize=3D256=A0=A0=A0 agco= unt=3D4, agsize=3D45785912 blks
> =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 attr=3D2
> data=A0=A0=A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D183143646,
> imaxpct=3D25
> =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sunit=3D0=A0=A0=A0=A0=A0 swidth= =3D0 blks
> naming=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 version 2=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 b= size=3D4096
> ascii-ci=3D0
> log=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3Dinternal=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D89425, version=3D2
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 sunit=3D1= blks,
> lazy-count=3D1
> realtime=A0=A0 =3Dnone=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 extsz=3D4096=A0=A0 blocks=3D0, rtextents=3D0
>
>

One thing is to make sure, that your partitions are aligned to<= br> physical sector size. Recent fdisk will do it properly. See options
'-c' (or 'c' in interactive mode) which sets DOS compatibil= ity mode
and '-u' (or 'u' in interactive mode) which sets units whic= h fdisk
uses. You want no DOS compatibility and units of sectors. Then first
partition starts on 2048 sector (so 1MiB is available for GRUB for
example) and it's gets things nicely aligned ... 2048 logical sectors =3D 256 physical sectors.

If the whole disk is use= d for the file system (it is not a bootable disk so no partition is created= ), can I ignore these settings safely?
=A0

If you are creating more than one partition, use something like +34G
while specifying end of the partition (so the next one is aligned as
well).

Martin

--20cf303a2d13e92e1704b1262fba-- From martin.rusko@gmail.com Mon Nov 7 09:14:03 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7FE2Qg063664 for ; Mon, 7 Nov 2011 09:14:03 -0600 X-ASG-Debug-ID: 1320678841-547000e70000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-gx0-f181.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 389361301B3A for ; Mon, 7 Nov 2011 07:14:01 -0800 (PST) Received: from mail-gx0-f181.google.com (mail-gx0-f181.google.com [209.85.161.181]) by cuda.sgi.com with ESMTP id VM4idM7E5FPVavnA for ; Mon, 07 Nov 2011 07:14:01 -0800 (PST) Received: by ggnv2 with SMTP id v2so698881ggn.26 for ; Mon, 07 Nov 2011 07:14:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=Xl4N3rCjfbfl4mYPxdprwuY78R0waFjcVi2Kw1M+FwI=; b=R0+2FjP43rHq7vh+8vUdL2AHqVDsUWB0THU9Z8R4o9ZmjF7nsR3ZDHQ9CKRIMPJ8sX BE+ZVxpnOPTH1lNmweF1XtZTeUgX/cHy/zLAeNc3Dr/RguTPNy3OrufPCNrGpXsKo4ob zaQPQZBRto1lBffZYQ3sW4ADZNUhv/Mt8Cfi8= MIME-Version: 1.0 Received: by 10.42.41.143 with SMTP id p15mr48260154ice.9.1320678840516; Mon, 07 Nov 2011 07:14:00 -0800 (PST) Received: by 10.42.240.70 with HTTP; Mon, 7 Nov 2011 07:14:00 -0800 (PST) In-Reply-To: References: Date: Mon, 7 Nov 2011 16:14:00 +0100 Message-ID: X-ASG-Orig-Subj: Re: Advanced format disk Subject: Re: Advanced format disk From: Martin Rusko To: Zhu Han Cc: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-gx0-f181.google.com[209.85.161.181] X-Barracuda-Start-Time: 1320678842 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79587 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 7, 2011 at 3:51 PM, Zhu Han wrote: > > > > On Mon, Nov 7, 2011 at 10:44 PM, Martin Rusko > wrote: >> >> On Mon, Nov 7, 2011 at 3:12 AM, Zhu Han wrote: >> > Hi, >> > >> > I got several advanced format disk, whose physical size is 4096 bytes, >> > but >> > its logical size is 512 bytes: >> > $ sudo fdisk -l /dev/sdb >> > >> > Disk /dev/sdb: 750.2 GB, 750156374016 bytes >> > 255 heads, 63 sectors/track, 91201 cylinders >> > Units =3D cylinders of 16065 * 512 =3D 8225280 bytes >> > Sector size (logical/physical): 512 bytes / 4096 bytes >> > I/O size (minimum/optimal): 4096 bytes / 4096 bytes >> > Disk identifier: 0x00000000 >> > >> > Disk /dev/sdb doesn't contain a valid partition table >> > >> > Is there any special tuning knob I should notice before formating it? >> > IMHO, >> > set the sector size as 4096 bytes is enough. The default block size is >> > 4096 >> > bytes. >> > $ sudo xfs_info /dev/sdb >> > meta-data=3D/dev/sdb=A0=A0=A0=A0=A0=A0=A0=A0 isize=3D256=A0=A0=A0 agco= unt=3D4, agsize=3D45785912 blks >> > =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 attr=3D2 >> > data=A0=A0=A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D183143646, >> > imaxpct=3D25 >> > =A0=A0=A0=A0=A0=A0=A0 =A0 =A0 =A0 =A0 =A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sunit=3D0=A0=A0=A0=A0=A0 swidth= =3D0 blks >> > naming=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0 version 2=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 b= size=3D4096 >> > ascii-ci=3D0 >> > log=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3Dinternal=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 bsize=3D4096=A0=A0 blocks=3D89425, >> > version=3D2 >> > =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =3D=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 sectsz=3D4096=A0 sunit=3D1= blks, >> > lazy-count=3D1 >> > realtime=A0=A0 =3Dnone=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 extsz=3D4096=A0=A0 blocks=3D0, rtextents=3D0 >> > >> > >> >> One thing is to make sure, that your partitions are aligned to >> physical sector size. Recent fdisk will do it properly. See options >> '-c' (or 'c' in interactive mode) which sets DOS compatibility mode >> and '-u' (or 'u' in interactive mode) which sets units which fdisk >> uses. You want no DOS compatibility and units of sectors. Then first >> partition starts on 2048 sector (so 1MiB is available for GRUB for >> example) and it's gets things nicely aligned ... 2048 logical sectors >> =3D 256 physical sectors. > > If the whole disk is used for the file system (it is not a bootable disk = so > no partition is created), can I ignore these settings safely? > I've heard about some WD drives with jumper switch, which could be used to shift logical sectors by one to be aligned somehow with dos-compatible partitioning. If you don't have this jumper on or you don't have drive with it, you can ignore it. >> >> If you are creating more than one partition, use something like +34G >> while specifying end of the partition (so the next one is aligned as >> well). >> From wkendall@sgi.com Mon Nov 7 09:14:38 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7FEble063713 for ; Mon, 7 Nov 2011 09:14:38 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id B4E578F806F; Mon, 7 Nov 2011 07:14:34 -0800 (PST) Received: from [128.162.233.117] (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 9098B70006EC; Mon, 7 Nov 2011 09:14:34 -0600 (CST) Message-ID: <4EB7F5DA.5040103@sgi.com> Date: Mon, 07 Nov 2011 09:14:34 -0600 From: Bill Kendall User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110921 Thunderbird/3.1.15 MIME-Version: 1.0 To: Mike Frysinger Cc: xfs@oss.sgi.com Subject: Re: [PATCH] path: fix 1 byte overflow with empty lists References: <1320554817-2594-1-git-send-email-vapier@gentoo.org> In-Reply-To: <1320554817-2594-1-git-send-email-vapier@gentoo.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 11/05/2011 11:46 PM, Mike Frysinger wrote: > If pap->pa_cnt is 0, then the local buffer is allocated as 1 byte, > but the code then writes two bytes to it '/' and '\0'. > > Signed-off-by: Mike Frysinger Looks good, Reviewed-by: Bill Kendall From aelder@sgi.com Mon Nov 7 10:10:29 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7GATTX066088 for ; Mon, 7 Nov 2011 10:10:29 -0600 Received: from xmail.sgi.com (pv-excas3-dc21.corp.sgi.com [137.38.102.206]) by relay3.corp.sgi.com (Postfix) with ESMTP id BF203AC003; Mon, 7 Nov 2011 08:10:25 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 10:10:24 -0600 Message-ID: <1320682224.3186.5.camel@doink> Subject: [PATCH, updated] xfs: Fix possible memory corruption in xfs_readlink From: Alex Elder Reply-To: To: XFS Mailing List CC: Carlos Maiolino , Ben Hutchings Date: Mon, 7 Nov 2011 10:10:24 -0600 In-Reply-To: <1320156842.30281.28.camel@deadeye> References: <1320156842.30281.28.camel@deadeye> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean From: Carlos Maiolino Fixes a possible memory corruption when the link is larger than MAXPATHLEN and XFS_DEBUG is not enabled. This also remove the S_ISLNK assert, since the inode mode is checked previously in xfs_readlink_by_handle() and via VFS. Updated to address concerns raised by Ben Hutchings about the loose attention paid to 32- vs 64-bit values, and the lack of handling a potentially negative pathlen value: - Changed type of "pathlen" to be xfs_fsize_t, to match that of ip->i_d.di_size - Added checking for a negative pathlen to the too-long pathlen test, and generalized the message that gets reported in that case to reflect the change As a result, if a negative pathlen were encountered, this function would return EFSCORRUPTED (and would fail an assertion for a debug build)--just as would a too-long pathlen. Signed-off-by Alex Elder --- fs/xfs/xfs_vnodeops.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) Index: b/fs/xfs/xfs_vnodeops.c =================================================================== --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -112,7 +112,7 @@ xfs_readlink( char *link) { xfs_mount_t *mp = ip->i_mount; - int pathlen; + xfs_fsize_t pathlen; int error = 0; trace_xfs_readlink(ip); @@ -122,13 +122,19 @@ xfs_readlink( xfs_ilock(ip, XFS_ILOCK_SHARED); - ASSERT(S_ISLNK(ip->i_d.di_mode)); - ASSERT(ip->i_d.di_size <= MAXPATHLEN); - pathlen = ip->i_d.di_size; if (!pathlen) goto out; + if (pathlen < 0 || pathlen > MAXPATHLEN) { + xfs_alert(mp, "%s: inode (%llu) bad symlink length (%lld)", + __func__, (unsigned long long) ip->i_ino, + (long long) pathlen); + ASSERT(0); + return XFS_ERROR(EFSCORRUPTED); + } + + if (ip->i_df.if_flags & XFS_IFINLINE) { memcpy(link, ip->i_df.if_u1.if_data, pathlen); link[pathlen] = '\0'; From cmaiolino@redhat.com Mon Nov 7 10:32:34 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7GWXJv066969 for ; Mon, 7 Nov 2011 10:32:34 -0600 X-ASG-Debug-ID: 1320683552-193403d30000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 324D5102ABB1; Mon, 7 Nov 2011 08:32:32 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id RwaN0dvV9JnMfQFV; Mon, 07 Nov 2011 08:32:32 -0800 (PST) X-ASG-Whitelist: Barracuda Reputation Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id pA7GWOB7018518 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 7 Nov 2011 11:32:25 -0500 Received: from andromeda.usersys.redhat.com (ovpn-113-26.phx2.redhat.com [10.3.113.26]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id pA7GWKxs027207 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 7 Nov 2011 11:32:23 -0500 Date: Mon, 7 Nov 2011 14:31:08 -0200 From: Carlos Maiolino To: Alex Elder Cc: XFS Mailing List , Ben Hutchings X-ASG-Orig-Subj: Re: [PATCH, updated] xfs: Fix possible memory corruption in xfs_readlink Subject: Re: [PATCH, updated] xfs: Fix possible memory corruption in xfs_readlink Message-ID: <20111107163108.GA21214@andromeda.usersys.redhat.com> References: <1320156842.30281.28.camel@deadeye> <1320682224.3186.5.camel@doink> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320682224.3186.5.camel@doink> User-Agent: Mutt/1.5.20 (2009-12-10) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1320683553 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean ACK, below my signoff On Mon, Nov 07, 2011 at 10:10:24AM -0600, Alex Elder wrote: > From: Carlos Maiolino > > Fixes a possible memory corruption when the link is larger than > MAXPATHLEN and XFS_DEBUG is not enabled. This also remove the > S_ISLNK assert, since the inode mode is checked previously in > xfs_readlink_by_handle() and via VFS. > > Updated to address concerns raised by Ben Hutchings about the loose > attention paid to 32- vs 64-bit values, and the lack of handling a > potentially negative pathlen value: > - Changed type of "pathlen" to be xfs_fsize_t, to match that of > ip->i_d.di_size > - Added checking for a negative pathlen to the too-long pathlen > test, and generalized the message that gets reported in that case > to reflect the change > As a result, if a negative pathlen were encountered, this function > would return EFSCORRUPTED (and would fail an assertion for a debug > build)--just as would a too-long pathlen. > > Signed-off-by Alex Elder Signed-off-by: Carlos Maiolino > > --- > fs/xfs/xfs_vnodeops.c | 14 ++++++++++---- > 1 file changed, 10 insertions(+), 4 deletions(-) > > Index: b/fs/xfs/xfs_vnodeops.c > =================================================================== > --- a/fs/xfs/xfs_vnodeops.c > +++ b/fs/xfs/xfs_vnodeops.c > @@ -112,7 +112,7 @@ xfs_readlink( > char *link) > { > xfs_mount_t *mp = ip->i_mount; > - int pathlen; > + xfs_fsize_t pathlen; > int error = 0; > > trace_xfs_readlink(ip); > @@ -122,13 +122,19 @@ xfs_readlink( > > xfs_ilock(ip, XFS_ILOCK_SHARED); > > - ASSERT(S_ISLNK(ip->i_d.di_mode)); > - ASSERT(ip->i_d.di_size <= MAXPATHLEN); > - > pathlen = ip->i_d.di_size; > if (!pathlen) > goto out; > > + if (pathlen < 0 || pathlen > MAXPATHLEN) { > + xfs_alert(mp, "%s: inode (%llu) bad symlink length (%lld)", > + __func__, (unsigned long long) ip->i_ino, > + (long long) pathlen); > + ASSERT(0); > + return XFS_ERROR(EFSCORRUPTED); > + } > + > + > if (ip->i_df.if_flags & XFS_IFINLINE) { > memcpy(link, ip->i_df.if_u1.if_data, pathlen); > link[pathlen] = '\0'; > > > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs -- --Carlos From wkendall@sgi.com Mon Nov 7 10:58:29 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7GwTmV068171 for ; Mon, 7 Nov 2011 10:58:29 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id 0CFD9AC004; Mon, 7 Nov 2011 08:58:28 -0800 (PST) Received: from [128.162.233.117] (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id AE90F700076B; Mon, 7 Nov 2011 10:58:28 -0600 (CST) Message-ID: <4EB80E34.9070704@sgi.com> Date: Mon, 07 Nov 2011 10:58:28 -0600 From: Bill Kendall User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110921 Thunderbird/3.1.15 MIME-Version: 1.0 To: Bill Kendall Cc: xfs@oss.sgi.com Subject: Re: [PATCH] xfstests: add 059 and 060 for multi-stream xfsdump References: <1320322464-6559-1-git-send-email-wkendall@sgi.com> In-Reply-To: <1320322464-6559-1-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On 11/03/2011 07:14 AM, Bill Kendall wrote: > Implement the multi-stream placeholder tests 059 and 060. The comments > imply that these tests existed on IRIX, but I was unable to find them. > > Test 059 does a simple 4-way dump and restore. Test 060 does a 4-way > dump, then restores each dump file individually (in a cumulative > fashion). > > The tests are skipped if the installed xfsdump does not support > multi-stream output. > > Signed-off-by: Bill Kendall > --- > 059 | 23 ++++++++++++++--------- > 060 | 29 ++++++++++++++++++++--------- > common.dump | 40 ++++++++++++++++++++++++++++++++++++++-- > group | 4 ++-- > 4 files changed, 74 insertions(+), 22 deletions(-) I realized the .out files were not included in my patch. I'll repost shortly... Bill From wkendall@sgi.com Mon Nov 7 11:22:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_42, J_CHICKENPOX_64,T_LOTS_OF_MONEY autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7HMNM0068717 for ; Mon, 7 Nov 2011 11:22:23 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id 3AB8E8F8084; Mon, 7 Nov 2011 09:22:20 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 2A8A470016D4; Mon, 7 Nov 2011 11:22:20 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id DC68720DF332; Mon, 7 Nov 2011 11:22:19 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2] xfstests: add 059 and 060 for multi-stream xfsdump Date: Mon, 7 Nov 2011 11:22:16 -0600 Message-Id: <1320686536-1399-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Implement the multi-stream placeholder tests 059 and 060. The comments imply that these tests existed on IRIX, but I was unable to find them. Test 059 does a simple 4-way dump and restore. Test 060 does a 4-way dump, then restores each dump file individually (in a cumulative fashion). The tests are skipped if the installed xfsdump does not support multi-stream output. Signed-off-by: Bill Kendall --- Changed for v2: - Included the expected-output files. 059 | 23 +++++++---- 059.out | 76 +++++++++++++++++++++++++++++++++++++ 060 | 29 ++++++++++---- 060.out | 120 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ common.dump | 40 +++++++++++++++++++- group | 4 +- 6 files changed, 270 insertions(+), 22 deletions(-) create mode 100644 059.out create mode 100644 060.out diff --git a/059 b/059 index 6348f0e..1f3f8e3 100755 --- a/059 +++ b/059 @@ -1,10 +1,10 @@ #! /bin/bash # FS QA Test No. 059 # -# place holder for IRIX 059 test for xfsdump/xfsrestore multi streams +# Test multi-stream xfsdump/xfsrestore. # #----------------------------------------------------------------------- -# Copyright (c) 2000-2001 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2000-2001, 2011 SGI. All Rights Reserved. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License as @@ -22,26 +22,31 @@ #----------------------------------------------------------------------- # # creator -owner=ivanr@sgi.com +owner=wkendall@sgi.com seq=`basename $0` echo "QA output created by $seq" here=`pwd` tmp=/tmp/$$ -status=1 # failure is the default! +status=0 # success is the default! trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc -. ./common.filter - -_notrun "Place holder for IRIX test 059" +. ./common.dump # real QA test starts here _supported_fs xfs -_supported_os IRIX +_supported_os Linux + +_require_multi_stream + +_create_dumpdir_fill_multi +_do_dump_multi_file --multi 4 +_do_restore_multi_file --multi 4 +_ls_compare_sub +_diff_compare # success, all done -status=0 exit diff --git a/059.out b/059.out new file mode 100644 index 0000000..7d670e1 --- /dev/null +++ b/059.out @@ -0,0 +1,76 @@ +QA output created by 059 +Creating directory system to dump using src/fill. +Setup ....................................... +Dumping to files... +xfsdump -f DUMP_FILE.0 -M stress_tape_media.0 -f DUMP_FILE.1 -M stress_tape_media.1 -f DUMP_FILE.2 -M stress_tape_media.2 -f DUMP_FILE.3 -M stress_tape_media.3 -L stress_059 SCRATCH_MNT +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: level 0 dump of HOSTNAME:SCRATCH_MNT +xfsdump: dump date: DATE +xfsdump: session id: ID +xfsdump: session label: "stress_059" +xfsdump: ino map +xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 3: ino INO offset NUM to end +xfsdump: ino map construction complete +xfsdump: estimated dump size: NUM bytes +xfsdump: estimated dump size per stream: NUM bytes +xfsdump: /var/xfsdump/inventory created +xfsdump: dump size (non-dir files) : NUM bytes +xfsdump: dump complete: SECS seconds elapsed +xfsdump: Dump Status: SUCCESS +Restoring from file... +xfsrestore -f DUMP_FILE.0 -f DUMP_FILE.1 -f DUMP_FILE.2 -f DUMP_FILE.3 -L stress_059 RESTORE_DIR +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: restore complete: SECS seconds elapsed +xfsrestore: Restore Status: SUCCESS +Comparing listing of dump directory with restore directory +Files TMP.dump_dir and TMP.restore_dir are identical +Comparing dump directory with restore directory +Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical +Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical +Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical +Files DUMP_DIR/large222 and RESTORE_DIR/DUMP_SUBDIR/large222 are identical +Files DUMP_DIR/small and RESTORE_DIR/DUMP_SUBDIR/small are identical +Files DUMP_DIR/sub/a and RESTORE_DIR/DUMP_SUBDIR/sub/a are identical +Files DUMP_DIR/sub/a00 and RESTORE_DIR/DUMP_SUBDIR/sub/a00 are identical +Files DUMP_DIR/sub/a000 and RESTORE_DIR/DUMP_SUBDIR/sub/a000 are identical +Files DUMP_DIR/sub/b and RESTORE_DIR/DUMP_SUBDIR/sub/b are identical +Files DUMP_DIR/sub/b00 and RESTORE_DIR/DUMP_SUBDIR/sub/b00 are identical +Files DUMP_DIR/sub/big and RESTORE_DIR/DUMP_SUBDIR/sub/big are identical +Files DUMP_DIR/sub/c and RESTORE_DIR/DUMP_SUBDIR/sub/c are identical +Files DUMP_DIR/sub/c00 and RESTORE_DIR/DUMP_SUBDIR/sub/c00 are identical +Files DUMP_DIR/sub/d and RESTORE_DIR/DUMP_SUBDIR/sub/d are identical +Files DUMP_DIR/sub/d00 and RESTORE_DIR/DUMP_SUBDIR/sub/d00 are identical +Files DUMP_DIR/sub/e and RESTORE_DIR/DUMP_SUBDIR/sub/e are identical +Files DUMP_DIR/sub/e00 and RESTORE_DIR/DUMP_SUBDIR/sub/e00 are identical +Files DUMP_DIR/sub/e000 and RESTORE_DIR/DUMP_SUBDIR/sub/e000 are identical +Files DUMP_DIR/sub/f and RESTORE_DIR/DUMP_SUBDIR/sub/f are identical +Files DUMP_DIR/sub/f00 and RESTORE_DIR/DUMP_SUBDIR/sub/f00 are identical +Files DUMP_DIR/sub/g and RESTORE_DIR/DUMP_SUBDIR/sub/g are identical +Files DUMP_DIR/sub/g00 and RESTORE_DIR/DUMP_SUBDIR/sub/g00 are identical +Files DUMP_DIR/sub/h and RESTORE_DIR/DUMP_SUBDIR/sub/h are identical +Files DUMP_DIR/sub/h00 and RESTORE_DIR/DUMP_SUBDIR/sub/h00 are identical +Files DUMP_DIR/sub/h000 and RESTORE_DIR/DUMP_SUBDIR/sub/h000 are identical +Files DUMP_DIR/sub/i and RESTORE_DIR/DUMP_SUBDIR/sub/i are identical +Files DUMP_DIR/sub/i00 and RESTORE_DIR/DUMP_SUBDIR/sub/i00 are identical +Files DUMP_DIR/sub/j and RESTORE_DIR/DUMP_SUBDIR/sub/j are identical +Files DUMP_DIR/sub/j00 and RESTORE_DIR/DUMP_SUBDIR/sub/j00 are identical +Files DUMP_DIR/sub/k and RESTORE_DIR/DUMP_SUBDIR/sub/k are identical +Files DUMP_DIR/sub/k00 and RESTORE_DIR/DUMP_SUBDIR/sub/k00 are identical +Files DUMP_DIR/sub/k000 and RESTORE_DIR/DUMP_SUBDIR/sub/k000 are identical +Files DUMP_DIR/sub/l and RESTORE_DIR/DUMP_SUBDIR/sub/l are identical +Files DUMP_DIR/sub/l00 and RESTORE_DIR/DUMP_SUBDIR/sub/l00 are identical +Files DUMP_DIR/sub/m and RESTORE_DIR/DUMP_SUBDIR/sub/m are identical +Files DUMP_DIR/sub/m00 and RESTORE_DIR/DUMP_SUBDIR/sub/m00 are identical +Files DUMP_DIR/sub/n and RESTORE_DIR/DUMP_SUBDIR/sub/n are identical +Files DUMP_DIR/sub/n00 and RESTORE_DIR/DUMP_SUBDIR/sub/n00 are identical +Files DUMP_DIR/sub/small and RESTORE_DIR/DUMP_SUBDIR/sub/small are identical +Only in SCRATCH_MNT: RESTORE_SUBDIR diff --git a/060 b/060 index 6e58e5b..4adc00f 100755 --- a/060 +++ b/060 @@ -1,10 +1,10 @@ #! /bin/bash # FS QA Test No. 060 # -# place holder for IRIX 060 test for xfsdump/xfsrestore multi streams +# Test multi-stream xfsdump and restoring one stream at a time. # #----------------------------------------------------------------------- -# Copyright (c) 2000-2001 Silicon Graphics, Inc. All Rights Reserved. +# Copyright (c) 2000-2001, 2011 SGI. All Rights Reserved. # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License as @@ -22,26 +22,37 @@ #----------------------------------------------------------------------- # # creator -owner=ivanr@sgi.com +owner=wkendall@sgi.com seq=`basename $0` echo "QA output created by $seq" here=`pwd` tmp=/tmp/$$ -status=1 # failure is the default! +status=0 # success is the default! trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc -. ./common.filter - -_notrun "Place holder for IRIX test 060" +. ./common.dump # real QA test starts here _supported_fs xfs -_supported_os IRIX +_supported_os Linux + +_require_multi_stream + +_create_dumpdir_fill_multi +_do_dump_multi_file --multi 4 +_prepare_restore_dir +base=$dump_file +opts="" +for x in 0 1 2 3; do + _do_restore_file_cum -f $base.$x $opts + opts=-R +done +_ls_compare_sub +_diff_compare # success, all done -status=0 exit diff --git a/060.out b/060.out new file mode 100644 index 0000000..29a1c5e --- /dev/null +++ b/060.out @@ -0,0 +1,120 @@ +QA output created by 060 +Creating directory system to dump using src/fill. +Setup ....................................... +Dumping to files... +xfsdump -f DUMP_FILE.0 -M stress_tape_media.0 -f DUMP_FILE.1 -M stress_tape_media.1 -f DUMP_FILE.2 -M stress_tape_media.2 -f DUMP_FILE.3 -M stress_tape_media.3 -L stress_060 SCRATCH_MNT +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: using file dump (drive_simple) strategy +xfsdump: level 0 dump of HOSTNAME:SCRATCH_MNT +xfsdump: dump date: DATE +xfsdump: session id: ID +xfsdump: session label: "stress_060" +xfsdump: ino map +xfsdump: stream 0: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 1: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 2: ino INO offset NUM to ino INO offset NUM +xfsdump: stream 3: ino INO offset NUM to end +xfsdump: ino map construction complete +xfsdump: estimated dump size: NUM bytes +xfsdump: estimated dump size per stream: NUM bytes +xfsdump: /var/xfsdump/inventory created +xfsdump: dump size (non-dir files) : NUM bytes +xfsdump: dump complete: SECS seconds elapsed +xfsdump: Dump Status: SUCCESS +Restoring cumumlative from file... +xfsrestore -f DUMP_FILE -r RESTORE_DIR +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: searching media for dump +xfsrestore: examining media file 0 +xfsrestore: dump description: +xfsrestore: hostname: HOSTNAME +xfsrestore: mount point: SCRATCH_MNT +xfsrestore: volume: SCRATCH_DEV +xfsrestore: session time: TIME +xfsrestore: level: 0 +xfsrestore: session label: "stress_060" +xfsrestore: media label: "stress_tape_media.0" +xfsrestore: file system ID: ID +xfsrestore: session id: ID +xfsrestore: media ID: ID +xfsrestore: using online session inventory +xfsrestore: searching media for directory dump +xfsrestore: reading directories +xfsrestore: 3 directories and 41 entries processed +xfsrestore: directory post-processing +xfsrestore: restoring non-directory files +xfsrestore: NOTE: restore interrupted: SECS seconds elapsed: may resume later using -R option +xfsrestore: Restore Status: INCOMPLETE +Restoring cumumlative from file... +xfsrestore -R -f DUMP_FILE -r RESTORE_DIR +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: resuming restore previously begun DATE +xfsrestore: examining media file 0 +xfsrestore: seeking past media file directory dump +xfsrestore: restoring non-directory files +xfsrestore: NOTE: restore interrupted: SECS seconds elapsed: may resume later using -R option +xfsrestore: Restore Status: INCOMPLETE +Restoring cumumlative from file... +xfsrestore -R -f DUMP_FILE -r RESTORE_DIR +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: resuming restore previously begun DATE +xfsrestore: examining media file 0 +xfsrestore: seeking past media file directory dump +xfsrestore: restoring non-directory files +xfsrestore: NOTE: restore interrupted: SECS seconds elapsed: may resume later using -R option +xfsrestore: Restore Status: INCOMPLETE +Restoring cumumlative from file... +xfsrestore -R -f DUMP_FILE -r RESTORE_DIR +xfsrestore: using file dump (drive_simple) strategy +xfsrestore: resuming restore previously begun DATE +xfsrestore: examining media file 0 +xfsrestore: seeking past media file directory dump +xfsrestore: restoring non-directory files +xfsrestore: restore complete: SECS seconds elapsed +xfsrestore: Restore Status: SUCCESS +Comparing listing of dump directory with restore directory +Files TMP.dump_dir and TMP.restore_dir are identical +Comparing dump directory with restore directory +Files DUMP_DIR/big and RESTORE_DIR/DUMP_SUBDIR/big are identical +Files DUMP_DIR/large000 and RESTORE_DIR/DUMP_SUBDIR/large000 are identical +Files DUMP_DIR/large111 and RESTORE_DIR/DUMP_SUBDIR/large111 are identical +Files DUMP_DIR/large222 and RESTORE_DIR/DUMP_SUBDIR/large222 are identical +Files DUMP_DIR/small and RESTORE_DIR/DUMP_SUBDIR/small are identical +Files DUMP_DIR/sub/a and RESTORE_DIR/DUMP_SUBDIR/sub/a are identical +Files DUMP_DIR/sub/a00 and RESTORE_DIR/DUMP_SUBDIR/sub/a00 are identical +Files DUMP_DIR/sub/a000 and RESTORE_DIR/DUMP_SUBDIR/sub/a000 are identical +Files DUMP_DIR/sub/b and RESTORE_DIR/DUMP_SUBDIR/sub/b are identical +Files DUMP_DIR/sub/b00 and RESTORE_DIR/DUMP_SUBDIR/sub/b00 are identical +Files DUMP_DIR/sub/big and RESTORE_DIR/DUMP_SUBDIR/sub/big are identical +Files DUMP_DIR/sub/c and RESTORE_DIR/DUMP_SUBDIR/sub/c are identical +Files DUMP_DIR/sub/c00 and RESTORE_DIR/DUMP_SUBDIR/sub/c00 are identical +Files DUMP_DIR/sub/d and RESTORE_DIR/DUMP_SUBDIR/sub/d are identical +Files DUMP_DIR/sub/d00 and RESTORE_DIR/DUMP_SUBDIR/sub/d00 are identical +Files DUMP_DIR/sub/e and RESTORE_DIR/DUMP_SUBDIR/sub/e are identical +Files DUMP_DIR/sub/e00 and RESTORE_DIR/DUMP_SUBDIR/sub/e00 are identical +Files DUMP_DIR/sub/e000 and RESTORE_DIR/DUMP_SUBDIR/sub/e000 are identical +Files DUMP_DIR/sub/f and RESTORE_DIR/DUMP_SUBDIR/sub/f are identical +Files DUMP_DIR/sub/f00 and RESTORE_DIR/DUMP_SUBDIR/sub/f00 are identical +Files DUMP_DIR/sub/g and RESTORE_DIR/DUMP_SUBDIR/sub/g are identical +Files DUMP_DIR/sub/g00 and RESTORE_DIR/DUMP_SUBDIR/sub/g00 are identical +Files DUMP_DIR/sub/h and RESTORE_DIR/DUMP_SUBDIR/sub/h are identical +Files DUMP_DIR/sub/h00 and RESTORE_DIR/DUMP_SUBDIR/sub/h00 are identical +Files DUMP_DIR/sub/h000 and RESTORE_DIR/DUMP_SUBDIR/sub/h000 are identical +Files DUMP_DIR/sub/i and RESTORE_DIR/DUMP_SUBDIR/sub/i are identical +Files DUMP_DIR/sub/i00 and RESTORE_DIR/DUMP_SUBDIR/sub/i00 are identical +Files DUMP_DIR/sub/j and RESTORE_DIR/DUMP_SUBDIR/sub/j are identical +Files DUMP_DIR/sub/j00 and RESTORE_DIR/DUMP_SUBDIR/sub/j00 are identical +Files DUMP_DIR/sub/k and RESTORE_DIR/DUMP_SUBDIR/sub/k are identical +Files DUMP_DIR/sub/k00 and RESTORE_DIR/DUMP_SUBDIR/sub/k00 are identical +Files DUMP_DIR/sub/k000 and RESTORE_DIR/DUMP_SUBDIR/sub/k000 are identical +Files DUMP_DIR/sub/l and RESTORE_DIR/DUMP_SUBDIR/sub/l are identical +Files DUMP_DIR/sub/l00 and RESTORE_DIR/DUMP_SUBDIR/sub/l00 are identical +Files DUMP_DIR/sub/m and RESTORE_DIR/DUMP_SUBDIR/sub/m are identical +Files DUMP_DIR/sub/m00 and RESTORE_DIR/DUMP_SUBDIR/sub/m00 are identical +Files DUMP_DIR/sub/n and RESTORE_DIR/DUMP_SUBDIR/sub/n are identical +Files DUMP_DIR/sub/n00 and RESTORE_DIR/DUMP_SUBDIR/sub/n00 are identical +Files DUMP_DIR/sub/small and RESTORE_DIR/DUMP_SUBDIR/sub/small are identical +Only in SCRATCH_MNT: RESTORE_SUBDIR +Only in RESTORE_DIR: xfsrestorehousekeepingdir diff --git a/common.dump b/common.dump index 076517d..31a75c5 100644 --- a/common.dump +++ b/common.dump @@ -75,6 +75,14 @@ done have_mtvariable=false [ `uname` = "Linux" ] && have_mtvariable=true + +_require_multi_stream() +{ + $XFSDUMP_PROG -JF -f /dev/null -f /dev/null 2> /dev/null | + grep -q "too many -f arguments" && + _notrun "xfsdump multi-stream support required" +} + # # do a remote/local mt # @@ -536,6 +544,18 @@ _do_create_dumpdir_fill() cd $here } +_mk_fillconfig_multi() +{ + _mk_fillconfig1 + cat <>$tmp.config +# pathname size in bytes +# +large000 8874368 $nobody $nobody +large111 2582912 $nobody $nobody +large222 7825792 $nobody $nobody +End-of-File +} + _create_dumpdir_largefile() { _wipe_fs @@ -580,6 +600,17 @@ _create_dumpdir_fill_ea() _stable_fs } +# +# Create enough files, and a few large enough files, so that +# some files are likely to be split across streams. +# +_create_dumpdir_fill_multi() +{ + _wipe_fs + _mk_fillconfig_multi + _do_create_dumpdir_fill + _stable_fs +} # # Append a subset of the fill'ed files @@ -802,9 +833,11 @@ _dump_filter_main() -e 's/time:[ ].*/time: TIME/' \ -e 's/date:[ ].*/date: DATE/' \ -e 's/dump begun .*/dump begun DATE/' \ + -e 's/previously begun .*/previously begun DATE/' \ -e 's/[0-9][0-9]* seconds/SECS seconds/' \ -e 's/restore.[0-9][0-9]*/restore.PID/' \ - -e 's/ino [0-9][0-9]*/ino INO/' \ + -e 's/ino [0-9][0-9]*/ino INO/g' \ + -e '/stream [0-9]:/s/offset [0-9][0-9]*/offset NUM/g' \ -e '/: dump size/s/[0-9][0-9]*/NUM/' \ -e '/dump size:/s/[0-9][0-9]*/NUM/' \ -e '/dump size per stream:/s/[0-9][0-9]*/NUM/' \ @@ -862,7 +895,7 @@ _invutil_filter() _dir_filter() { sed \ - -e "s#$dump_file#DUMP_FILE#" \ + -e "s#$dump_file#DUMP_FILE#g" \ -e "s#$SCRATCH_DEV#SCRATCH_DEV#" \ -e "s#$SCRATCH_RAWDEV#SCRATCH_DEV#" \ -e "s#$dumptape#TAPE_DEV#" \ @@ -959,6 +992,9 @@ _parse_restore_args() --no-check-quota) do_quota_check=false ;; + -R) + restore_args="$restore_args $1" + ;; *) _fail "invalid argument to common.dump function: $1" ;; diff --git a/group b/group index 616cbdd..79b8c8c 100644 --- a/group +++ b/group @@ -169,8 +169,8 @@ deprecated 056 dump ioctl auto quick 057 acl auto 058 acl auto -059 dump ioctl -060 dump ioctl +059 dump ioctl auto quick +060 dump ioctl auto quick 061 dump ioctl auto quick 062 attr udf auto quick 063 dump attr auto quick -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 11:50:59 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7HoxdQ069543 for ; Mon, 7 Nov 2011 11:50:59 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id 64ADB8F8049; Mon, 7 Nov 2011 09:50:59 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 56F9970006E6; Mon, 7 Nov 2011 11:50:59 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 1859A20DF332; Mon, 7 Nov 2011 11:50:58 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH] xfstests: fix permissions on non-executable tests Date: Mon, 7 Nov 2011 11:50:53 -0600 Message-Id: <1320688253-2432-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Tests that are not executable are made executable when they are run, causing git to report them as modified. Fix this by changing non-executable tests to have permissions of 0755. Signed-off-by: Bill Kendall --- 0 files changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 255 mode change 100644 => 100755 256 mode change 100644 => 100755 257 mode change 100644 => 100755 258 mode change 100644 => 100755 259 mode change 100644 => 100755 260 mode change 100644 => 100755 261 mode change 100644 => 100755 262 mode change 100644 => 100755 263 mode change 100644 => 100755 266 mode change 100644 => 100755 267 mode change 100644 => 100755 268 diff --git a/255 b/255 old mode 100644 new mode 100755 diff --git a/256 b/256 old mode 100644 new mode 100755 diff --git a/257 b/257 old mode 100644 new mode 100755 diff --git a/258 b/258 old mode 100644 new mode 100755 diff --git a/259 b/259 old mode 100644 new mode 100755 diff --git a/260 b/260 old mode 100644 new mode 100755 diff --git a/261 b/261 old mode 100644 new mode 100755 diff --git a/262 b/262 old mode 100644 new mode 100755 diff --git a/263 b/263 old mode 100644 new mode 100755 diff --git a/266 b/266 old mode 100644 new mode 100755 diff --git a/267 b/267 old mode 100644 new mode 100755 diff --git a/268 b/268 old mode 100644 new mode 100755 -- 1.7.0.4 From bpm@sgi.com Mon Nov 7 14:04:51 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7K4oDZ074446 for ; Mon, 7 Nov 2011 14:04:50 -0600 Received: from whiskey.americas.sgi.com (whiskey.americas.sgi.com [128.162.233.19]) by relay2.corp.sgi.com (Postfix) with ESMTP id 73560304059; Mon, 7 Nov 2011 12:04:47 -0800 (PST) Received: by whiskey.americas.sgi.com (Postfix, from userid 4600) id 09AD5431F74; Mon, 7 Nov 2011 14:05:04 -0600 (CST) Date: Mon, 7 Nov 2011 14:05:03 -0600 From: Ben Myers To: Chandra Seetharaman Cc: xfs@oss.sgi.com Subject: Re: [RFC PATCH 0/4] Rearrange code to make the code more readable Message-ID: <20111107200503.GH20464@sgi.com> References: <20111027220523.2638.12351.sendpatchset@chandra-lucid.austin.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111027220523.2638.12351.sendpatchset@chandra-lucid.austin.ibm.com> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hey Chandra, On Thu, Oct 27, 2011 at 05:05:23PM -0500, Chandra Seetharaman wrote: > These are the split up patches from patch 1/5 that I posted today, as > suggested by Christoph. > > These define some new inline functions and macros, rearrange > the code for easier reading, simplification of code paths > and prepare for the changes to support pquota and gquota > simultaneously. All four of these patches look good to me. Reviewed-by: Ben Myers Regards, Ben From david@fromorbit.com Mon Nov 7 14:15:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KFBmt074789 for ; Mon, 7 Nov 2011 14:15:11 -0600 X-ASG-Debug-ID: 1320696908-1a7801670000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1A10B22675C for ; Mon, 7 Nov 2011 12:15:08 -0800 (PST) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id qUtqJWoC9rGPYxxy for ; Mon, 07 Nov 2011 12:15:08 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAOY5uE55LIil/2dsb2JhbABDqXCBBoFyAQEEAScTHCMFCwgDDgouFCUDIROIArgmEog2YwSdEokP Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail05.adl6.internode.on.net with ESMTP; 08 Nov 2011 06:45:07 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNVaz-0003pN-HY; Tue, 08 Nov 2011 07:15:05 +1100 Date: Tue, 8 Nov 2011 07:15:05 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfs_getbmap assert Subject: Re: xfs_getbmap assert Message-ID: <20111107201505.GE5534@dastard> References: <20111107103037.GA16213@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111107103037.GA16213@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1320696910 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79606 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 07, 2011 at 05:30:37AM -0500, Christoph Hellwig wrote: > With Dmitris fsstress updates I can hit the following assert fairly > regularly: > > [11904.943956] XFS: Assertion failed: ((iflags & BMV_IF_DELALLOC) != 0) || > (map[i].br_startblock != DELAYSTARTBLOCK), file: > /home/hch/work/linux-2.6/fs/xfs/xfs_bmap.c, line: 5604 > > which means we get an delalloc extent back from FIEMAP/GETBMAP despite > asking for a flush beforehand. While we hold the iolock over the call > and thus exclude new buffered writers from appearing that doesn't > prevent shared writeable mmaps from creating new delalloc extents. Possibly. However, we might actually be tripping over speculative delalloc regions beyond EOF - flushing will not convert those at all. There's already a comment earlier on like this: /* * even after flushing the inode, there can still be delalloc * blocks on the inode beyond EOF due to speculative * preallocation. These are not removed until the release * function is called or the inode is inactivated. Hence we * cannot assert here that ip->i_delayed_blks == 0. */ Can you see if that is the case that is being triggered? > I don't think this actually is a real issue, and a workaround would > be extremly hard. For now I've just remove the assert in my tree: Probably the right thing to do, anyway. Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Nov 7 14:18:20 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KIK0x074880 for ; Mon, 7 Nov 2011 14:18:20 -0600 X-ASG-Debug-ID: 1320697098-1a8301580000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C69BB2267A3 for ; Mon, 7 Nov 2011 12:18:18 -0800 (PST) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id QKCcB86YrBsmW1TW for ; Mon, 07 Nov 2011 12:18:18 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAOY5uE55LIil/2dsb2JhbABDqXCBBoFyAQEEATocIwULCAMYLhQlAyETiAK4JhKINmMEpiE Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail05.adl6.internode.on.net with ESMTP; 08 Nov 2011 06:48:17 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNVe4-0003q6-TD; Tue, 08 Nov 2011 07:18:16 +1100 Date: Tue, 8 Nov 2011 07:18:16 +1100 From: Dave Chinner To: Alexander Naumann Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: Crash with XFS Subject: Re: Crash with XFS Message-ID: <20111107201816.GF5534@dastard> References: <1320676611.3192.36.camel@AlfLaptop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320676611.3192.36.camel@AlfLaptop> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1320697099 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0266 1.0000 -1.8487 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.85 X-Barracuda-Spam-Status: No, SCORE=-1.85 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79606 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Nov 07, 2011 at 03:36:51PM +0100, Alexander Naumann wrote: > Hi! > > I would be glad if anybody can give me any hint on the following > subject. > I have a linux server running formatted with XFS. Afer a couple of days > (about 6 or 7) I get the following crash: > > > Nov 6 15:44:03 archive kernel: BUG: unable to handle kernel NULL > pointer dereference at 0000000000000044 > Nov 6 15:44:03 archive kernel: IP: [] > xfs_inode_ag_iterator+0x4a/0xce ..... > The system is a Dell R910 (Intek Xeon CPU E7530, 24 cores (with > hyperthreading). 32GB Ram, Raid Controller is Perc H700, SAS > discs, Raid 5 with 1.7TB, formatted with XFS. Kernel version > 2.6.34.7 is running (64bit kernel on a 32bit system, Debian ^^^^^^^^ Upgrade. There were problems in .34 to do with inode reclaim that were fixed in .35 and .36.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Nov 7 14:52:56 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_43 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Kqums075883 for ; Mon, 7 Nov 2011 14:52:56 -0600 X-ASG-Debug-ID: 1320699173-5b1c00040000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id AC6C81488C15 for ; Mon, 7 Nov 2011 12:52:53 -0800 (PST) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id Nt1iUohuRAJfvTyE for ; Mon, 07 Nov 2011 12:52:53 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAHBEuE55LIil/2dsb2JhbABEqXKBBoFyAQEFOhwjEAgDDgouFCUDIRPAQBKJGQSdEokP Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail05.adl6.internode.on.net with ESMTP; 08 Nov 2011 07:22:52 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNWBW-0003yp-K9; Tue, 08 Nov 2011 07:52:50 +1100 Date: Tue, 8 Nov 2011 07:52:50 +1100 From: Dave Chinner To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 1/9] xfstests: fsstress add command line style output for show_opts Subject: Re: [PATCH 1/9] xfstests: fsstress add command line style output for show_opts Message-ID: <20111107205250.GG5534@dastard> References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1320699175 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0207 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79609 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 09:55:45PM +0400, Dmitry Monakhov wrote: > Once some combination of seed+fs_ops result in regression it is > reasonable to document that combination. It is usefull to dump > that configuration in command line style. Later this line may be > simply hardcoded in to regression test. ..... > @@ -345,7 +345,10 @@ int main(int argc, char **argv) > zero_freq(); > break; > case 'S': > - show_ops(0, NULL); > + i = 0; > + if (optarg[0] == 'c') > + i = 1; > + show_ops(1, NULL); I don't think that does what it is supposed to do... Cheers, Dave. -- Dave Chinner david@fromorbit.com From wkendall@sgi.com Mon Nov 7 14:59:40 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KxeOT076016 for ; Mon, 7 Nov 2011 14:59:40 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id 5BEA0304048 for ; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 48E0A70016E1 for ; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 005F120E36FD; Mon, 7 Nov 2011 14:59:39 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Subject: [PATCH v2 0/8] xfsdump: enable support for multiple streams Date: Mon, 7 Nov 2011 14:58:23 -0600 Message-Id: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Version 2 of this series changes only patch #3. POSIX semaphores are now used instead of rolling our own, as suggested by Christoph. We lose the ability to count the number of waiting threads, which was used in some asserts. There was adequate coverage in related asserts, so this was deemed acceptable. This series resurrects the IRIX multi-stream support for splitting a backup among several output files/tapes. This offers some nice performance improvements, particularly in xfsrestore where a single stream often cannot keep the filesystem/disks busy. I've observed a 1.7x improvement on a backup and a 5x improvement on restore. I have a couple of xfstests for this, and will submit those once a few outstanding xfsdump test patches have been reviewed. There's a bit more work to do: - Now that xfsdump has threads once again, the tape I/O ring buffer support can be enabled. This series leaves it disabled so that more testing can be done in that area. - Currently the stream split points are determined by doing an extra inode scan. This is unchanged from how it was done on IRIX. I'd like to change this so that enough info is kept from the initial inode scan to determine appropriate split points without an additional scan. - You may notice in the last patch of this series that a lot of "miniroot" references also look at whether or not the dump is to a pipe. Now that the "miniroot" checks are gone, it's possible to clean up the pipe-related code too. I've got a separate patch series for that which I'll submit later. From wkendall@sgi.com Mon Nov 7 14:59:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Kxec1076025 for ; Mon, 7 Nov 2011 14:59:41 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id C4FB230406A; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 7D0DC70016E3; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 432C120E36F7; Mon, 7 Nov 2011 14:59:39 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 3/8] xfsdump: implement lock abstraction with pthreads Date: Mon, 7 Nov 2011 14:58:26 -0600 Message-Id: <1320699511-12281-4-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch implements xfsdump's locking abstraction using pthread locking primitives. The obsolete IRIX locking code has been removed. The POSIX semaphore implementation on Linux does not provide a count of threads waiting to decrement the semaphore, so qsemPblocked() has been removed. It was called from a few asserts, but the current count of the semaphore was also checked in adjacent asserts, so it was deemed acceptable to remove the qsemPblocked() asserts. A few debug-level log calls had to be changed to use the "no lock" flag to prevent locks from being acquired out of order. This is used in various debug messages throughout xfsdump. Signed-off-by: Bill Kendall --- Changes in v2: - Use POSIX semaphores (sem_t) instead of rolling our own with a pthread mutex and condition, as suggested by Christoph. common/main.c | 2 +- common/qlock.c | 355 +++++++++++------------------------------------------ common/qlock.h | 13 +-- common/ring.c | 4 - restore/content.c | 17 ++- 5 files changed, 88 insertions(+), 303 deletions(-) diff --git a/common/main.c b/common/main.c index 08be16e..be8a921 100644 --- a/common/main.c +++ b/common/main.c @@ -359,7 +359,7 @@ main( int argc, char *argv[] ) /* initialize the spinlock allocator */ - ok = qlock_init( miniroot ); + ok = qlock_init( ); if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } diff --git a/common/qlock.c b/common/qlock.c index 6c9ee16..adaa7dd 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -19,6 +19,9 @@ #include #include +#include +#include + #include "types.h" #include "qlock.h" #include "mlog.h" @@ -27,24 +30,13 @@ struct qlock { ix_t ql_ord; /* ordinal position of this lock */ - pid_t ql_owner; - /* who owns this lock - */ -#ifdef HIDDEN - ulock_t ql_uslockh; - /* us lock handle - */ -#endif /* HIDDEN */ + pthread_mutex_t ql_mutex; }; typedef struct qlock qlock_t; /* internal qlock */ -#define QLOCK_SPINS 0x1000 - /* how many times to spin on lock before sleeping for it - */ - #define QLOCK_THRDCNTMAX 256 /* arbitrary limit on number of threads supported */ @@ -67,15 +59,13 @@ static ordmap_t qlock_ordalloced; */ struct thrddesc { - pid_t td_pid; + pthread_t td_tid; ordmap_t td_ordmap; }; typedef struct thrddesc thrddesc_t; -#ifdef HIDDEN static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* holds the ordmap for each thread */ -#endif #define QLOCK_ORDMAP_SET( ordmap, ord ) ( ordmap |= 1U << ord ) /* sets the ordinal bit in an ordmap @@ -93,21 +83,6 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* checks if any bits less than ord are set in the ordmap */ -#ifdef HIDDEN -static usptr_t *qlock_usp; -#else -static void *qlock_usp; -#endif /* HIDDEN */ - - /* pointer to shared arena from which locks are allocated - */ - -#ifdef HIDDEN -static char *qlock_arenaroot = "xfsrestoreqlockarena"; - /* shared arena file name root - */ -#endif - /* REFERENCED */ static bool_t qlock_inited = BOOL_FALSE; /* to sanity check initialization @@ -115,22 +90,13 @@ static bool_t qlock_inited = BOOL_FALSE; /* forward declarations */ -#ifdef HIDDEN -static void qlock_ordmap_add( pid_t pid ); -static ordmap_t *qlock_ordmapp_get( pid_t pid ); -static ix_t qlock_thrdix_get( pid_t pid ); -#endif +static void qlock_ordmap_add( pthread_t tid ); +static ordmap_t *qlock_ordmapp_get( pthread_t tid ); +static ix_t qlock_thrdix_get( pthread_t tid ); bool_t -qlock_init( bool_t miniroot ) +qlock_init( void ) { -#ifdef HIDDEN - char arenaname[ 100 ]; - /* REFERENCED */ - intgen_t nwritten; - intgen_t rval; -#endif - /* sanity checks */ ASSERT( ! qlock_inited ); @@ -143,44 +109,6 @@ qlock_init( bool_t miniroot ) */ qlock_ordalloced = 0; - /* if miniroot, fake it - */ - if ( miniroot ) { - qlock_inited = BOOL_TRUE; - qlock_usp = 0; - return BOOL_TRUE; - } -#ifdef HIDDEN - - /* generate the arena name - */ - nwritten = sprintf( arenaname, - "/tmp/%s.%d", - qlock_arenaroot, - get_pid() ); - ASSERT( nwritten > 0 ); - ASSERT( ( size_t )nwritten < sizeof( arenaname )); - - /* configure shared arenas to automatically unlink on last close - */ - rval = usconfig( CONF_ARENATYPE, ( u_intgen_t )US_SHAREDONLY ); - if ( rval ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to configure shared arena for auto unlink: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } - - /* allocate a shared arena for the locks - */ - qlock_usp = usinit( arenaname ); - if ( ! qlock_usp ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to allocate shared arena for thread locks: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } - /* now say we are initialized */ qlock_inited = BOOL_TRUE; @@ -191,7 +119,6 @@ qlock_init( bool_t miniroot ) qlock_inited = BOOL_FALSE; return BOOL_FALSE; } -#endif /* HIDDEN */ return BOOL_TRUE; } @@ -199,28 +126,13 @@ qlock_init( bool_t miniroot ) bool_t qlock_thrdinit( void ) { -#ifdef HIDDEN - intgen_t rval; - /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - /* add thread to shared arena - */ - rval = usadd( qlock_usp ); - if ( rval ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to add thread to shared arena: %s\n"), - strerror( errno )); - return BOOL_FALSE; - } /* add thread to ordmap list */ - qlock_ordmap_add( get_pid() ); -#endif /* HIDDEN */ + qlock_ordmap_add( pthread_self() ); return BOOL_TRUE; } @@ -244,14 +156,9 @@ qlock_alloc( ix_t ord ) qlockp = ( qlock_t * )calloc( 1, sizeof( qlock_t )); ASSERT( qlockp ); -#ifdef HIDDEN - /* allocate a us lock: bypass if miniroot + /* initialize the mutex */ - if ( qlock_usp ) { - qlockp->ql_uslockh = usnewlock( qlock_usp ); - ASSERT( qlockp->ql_uslockh ); - } -#endif /* HIDDEN */ + pthread_mutex_init( &qlockp->ql_mutex, NULL ); /* assign the ordinal position */ @@ -263,44 +170,34 @@ qlock_alloc( ix_t ord ) void qlock_lock( qlockh_t qlockh ) { -#ifdef HIDDEN qlock_t *qlockp = ( qlock_t * )qlockh; - pid_t pid; + pthread_t tid; ix_t thrdix; ordmap_t *ordmapp; /* REFERENCED */ - bool_t lockacquired; -#endif + intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - /* bypass if miniroot - */ - if ( ! qlock_usp ) { - return; - } - -#ifdef HIDDEN - - /* get the caller's pid and thread index + /* get the caller's tid and thread index */ - pid = get_pid(); + tid = pthread_self(); - thrdix = qlock_thrdix_get( pid ); + thrdix = qlock_thrdix_get( tid ); /* get the ordmap for this thread */ - ordmapp = qlock_ordmapp_get( pid ); + ordmapp = qlock_ordmapp_get( tid ); - /* assert that this lock not already held + /* assert that this lock not already held by this thread */ if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock already held: thrd %d pid %d ord %d map %x\n"), + _("lock already held: thrd %d tid %lu ord %d map %x\n"), thrdix, - pid, + tid, qlockp->ql_ord, *ordmapp ); } @@ -310,278 +207,176 @@ qlock_lock( qlockh_t qlockh ) */ if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock ordinal violation: thrd %d pid %d ord %d map %x\n"), + _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"), thrdix, - pid, + tid, qlockp->ql_ord, *ordmapp ); } ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )); - /* acquire the us lock + /* acquire the lock */ - lockacquired = uswsetlock( qlockp->ql_uslockh, QLOCK_SPINS ); - ASSERT( lockacquired ); - - /* verify lock is not already held - */ - ASSERT( ! qlockp->ql_owner ); + rval = pthread_mutex_lock( &qlockp->ql_mutex ); + ASSERT( !rval ); /* add ordinal to this threads ordmap */ QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord ); - - /* indicate the lock's owner - */ - qlockp->ql_owner = pid; -#endif /* HIDDEN */ } void qlock_unlock( qlockh_t qlockh ) { -#ifdef HIDDEN qlock_t *qlockp = ( qlock_t * )qlockh; - pid_t pid; ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; -#endif /* sanity checks */ ASSERT( qlock_inited ); - /* bypass if miniroot - */ - if ( ! qlock_usp ) { - return; - } - -#ifdef HIDDEN - /* get the caller's pid - */ - pid = get_pid(); - /* get the ordmap for this thread */ - ordmapp = qlock_ordmapp_get( pid ); + ordmapp = qlock_ordmapp_get( pthread_self() ); /* verify lock is held by this thread */ ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); - ASSERT( qlockp->ql_owner == pid ); - - /* clear lock owner - */ - qlockp->ql_owner = 0; /* clear lock's ord from thread's ord map */ QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord ); - /* release the us lock + /* release the lock */ - rval = usunsetlock( qlockp->ql_uslockh ); + rval = pthread_mutex_unlock( &qlockp->ql_mutex ); ASSERT( ! rval ); -#endif /* HIDDEN */ } qsemh_t qsem_alloc( ix_t cnt ) { -#ifdef HIDDEN - usema_t *usemap; + sem_t *semp; + intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* allocate a us semaphore + /* allocate a semaphore */ - usemap = usnewsema( qlock_usp, ( intgen_t )cnt ); - ASSERT( usemap ); + semp = ( sem_t * )calloc( 1, sizeof( sem_t )); + ASSERT( semp ); - return ( qsemh_t )usemap; -#else - return 0; -#endif /* HIDDEN */ + /* initialize the semaphore + */ + rval = sem_init( semp, 0, cnt ); + ASSERT( !rval ); + + return ( qsemh_t )semp; } void qsem_free( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + sem_t *semp = ( sem_t * )qsemh; + intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* free the us semaphore + /* destroy the mutex and condition + */ + rval = sem_destroy( semp ); + ASSERT( !rval ); + + /* free the semaphore */ - usfreesema( usemap, qlock_usp ); -#endif /* HIDDEN */ + free( semp ); } void qsemP( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + sem_t *semp = ( sem_t * )qsemh; intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); /* "P" the semaphore */ - rval = uspsema( usemap ); - if ( rval != 1 ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to \"P\" semaphore: " - "rval == %d, errno == %d (%s)\n"), - rval, - errno, - strerror( errno )); - } - ASSERT( rval == 1 ); -#endif /* HIDDEN */ + rval = sem_wait( semp ); + ASSERT( !rval ); } void qsemV( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + sem_t *semp = ( sem_t * )qsemh; intgen_t rval; /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); /* "V" the semaphore */ - rval = usvsema( usemap ); - if ( rval != 0 ) { - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_NOLOCK, - _("unable to \"V\" semaphore: " - "rval == %d, errno == %d (%s)\n"), - rval, - errno, - strerror( errno )); - } - ASSERT( rval == 0 ); -#endif /* HIDDEN */ + rval = sem_post( semp ); + ASSERT( !rval ); } bool_t qsemPwouldblock( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + sem_t *semp = ( sem_t * )qsemh; + int count; intgen_t rval; - + /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* check the semaphore - */ - rval = ustestsema( usemap ); + rval = sem_getvalue( semp, &count ); + ASSERT( !rval ); - /* if equal to zero, no tokens left. if less than zero, other thread(s) - * currently waiting. - */ - if ( rval <= 0 ) { - return BOOL_TRUE; - } else { - return BOOL_FALSE; - } -#else -return BOOL_FALSE; -#endif /* HIDDEN */ + return count <= 0 ? BOOL_TRUE : BOOL_FALSE; } size_t qsemPavail( qsemh_t qsemh ) { -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; + sem_t *semp = ( sem_t * )qsemh; + int count; intgen_t rval; - - /* sanity checks - */ - ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - /* check the semaphore - */ - rval = ustestsema( usemap ); - - /* if greater or equal to zero, no one is blocked and that is the number - * of resources available. if less than zero, absolute value is the - * number of blocked threads. - */ - if ( rval < 0 ) { - return 0; - } else { - return ( size_t )rval; - } -#else -return 0; -#endif /* HIDDEN */ -} -size_t -qsemPblocked( qsemh_t qsemh ) -{ -#ifdef HIDDEN - usema_t *usemap = ( usema_t * )qsemh; - intgen_t rval; - /* sanity checks */ ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - /* check the semaphore - */ - rval = ustestsema( usemap ); + rval = sem_getvalue( semp, &count ); + ASSERT( !rval ); - /* if greater or equal to zero, no one is blocked. if less than zero, - * absolute value is the number of blocked threads. - */ - if ( rval < 0 ) { - return ( size_t )( 0 - rval ); - } else { - return 0; - } -#else -return 0; -#endif /* HIDDEN */ + return count < 0 ? 0 : count; } /* internal ordinal map abstraction */ -#ifdef HIDDEN static void -qlock_ordmap_add( pid_t pid ) +qlock_ordmap_add( pthread_t tid ) { ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX ); - qlock_thrddesc[ qlock_thrdcnt ].td_pid = pid; + qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid; qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0; qlock_thrdcnt++; } static thrddesc_t * -qlock_thrddesc_get( pid_t pid ) +qlock_thrddesc_get( pthread_t tid ) { thrddesc_t *p; thrddesc_t *endp; @@ -592,29 +387,27 @@ qlock_thrddesc_get( pid_t pid ) p < endp ; p++ ) { - if ( p->td_pid == pid ) { + if ( pthread_equal( p->td_tid, tid ) ) { return p; } } - ASSERT( 0 ); return 0; } static ordmap_t * -qlock_ordmapp_get( pid_t pid ) +qlock_ordmapp_get( pthread_t tid ) { thrddesc_t *p; - p = qlock_thrddesc_get( pid ); + p = qlock_thrddesc_get( tid ); return &p->td_ordmap; } static ix_t -qlock_thrdix_get( pid_t pid ) +qlock_thrdix_get( pthread_t tid ) { thrddesc_t *p; - p = qlock_thrddesc_get( pid ); + p = qlock_thrddesc_get( tid ); ASSERT( p >= &qlock_thrddesc[ 0 ] ); return ( ix_t )( p - &qlock_thrddesc[ 0 ] ); } -#endif diff --git a/common/qlock.h b/common/qlock.h index 1880aff..ae411bb 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -21,19 +21,13 @@ /* qlock - quick locks abstraction * * threads may allocate quick locks using qlock_alloc, and free them with - * qlock_free. the abstraction is initialized with qlock_init. the underlying - * mechanism is the IRIX us lock primitive. in order to use this, a temporary - * shared arena is created in /tmp. this will be automatically unlinked - * when the last thread exits. + * qlock_free. the abstraction is initialized with qlock_init. * * deadlock detection is accomplished by giving an ordinal number to each * lock allocated, and record all locks held by each thread. locks may not * be acquired out of order. that is, subsequently acquired locks must have * a lower ordinal than all locks currently held. for convenience, the ordinals * of all locks to be allocated will be defined in this file. - * - * ADDITION: added counting semaphores. simpler to do here since same - * shared arena can be used. */ #define QLOCK_ORD_CRIT 0 @@ -54,7 +48,7 @@ typedef void *qlockh_t; /* opaque handle */ -extern bool_t qlock_init( bool_t miniroot ); +extern bool_t qlock_init( void ); /* called by main to initialize abstraction. returns FALSE if * utility should abort. */ @@ -98,8 +92,5 @@ extern bool_t qsemPwouldblock( qsemh_t qsemh ); extern size_t qsemPavail( qsemh_t qsemh ); /* number of resources available */ -extern size_t qsemPblocked( qsemh_t qsemh ); - /* number of threads currently blocked on this semaphore - */ #endif /* QLOCK_H */ diff --git a/common/ring.c b/common/ring.c index e1dddba..b6074d1 100644 --- a/common/ring.c +++ b/common/ring.c @@ -258,9 +258,7 @@ ring_reset( ring_t *ringp, ring_msg_t *msgp ) /* re-initialize the ring */ ASSERT( qsemPavail( ringp->r_ready_qsemh ) == 0 ); - ASSERT( qsemPblocked( ringp->r_ready_qsemh ) == 0 ); ASSERT( qsemPavail( ringp->r_active_qsemh ) == 0 ); - ASSERT( qsemPblocked( ringp->r_active_qsemh ) <= 1 ); ringp->r_ready_in_ix = 0; ringp->r_ready_out_ix = 0; ringp->r_active_in_ix = 0; @@ -277,9 +275,7 @@ ring_reset( ring_t *ringp, ring_msg_t *msgp ) qsemV( ringp->r_ready_qsemh ); } ASSERT( qsemPavail( ringp->r_ready_qsemh ) == ringp->r_len ); - ASSERT( qsemPblocked( ringp->r_ready_qsemh ) == 0 ); ASSERT( qsemPavail( ringp->r_active_qsemh ) == 0 ); - ASSERT( qsemPblocked( ringp->r_active_qsemh ) <= 1 ); } void diff --git a/restore/content.c b/restore/content.c index 2228a7f..8dfa456 100644 --- a/restore/content.c +++ b/restore/content.c @@ -8911,7 +8911,8 @@ partial_reg( ix_t d_index, /* If not found, find a free one, fill it in and return */ if ( ! isptr ) { - mlog(MLOG_NITTY, "partial_reg: no entry found for %llu\n", ino); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: no entry found for %llu\n", ino); /* find a free one */ for (i=0; i < partialmax; i++ ) { if (persp->a.parrest[i].is_ino == 0) { @@ -8950,8 +8951,10 @@ found: /* no existing entry for this drive, fill in the values */ bsptr->offset = offset; bsptr->endoffset = endoffset; - mlog(MLOG_NITTY, "partial_reg: update entry [%d]: " - "\n", d_index, offset, endoffset); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: update entry [%d]: " + "\n", + d_index, offset, endoffset); } else { bool_t ret; @@ -8960,9 +8963,11 @@ found: */ bsptr->endoffset = endoffset; ret = partial_check2(isptr, fsize); - mlog(MLOG_NITTY, "partial_reg: extend entry [%d]: " - "\n", d_index, endoffset); - mlog(MLOG_NITTY, "partial_reg: partial_check returns: %d\n", ret); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: extend entry [%d]: \n", + d_index, endoffset); + mlog(MLOG_NITTY | MLOG_NOLOCK, + "partial_reg: partial_check returns: %d\n", ret); } pi_unlock(); -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Kxfni076029 for ; Mon, 7 Nov 2011 14:59:41 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id EE0E8304048; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id B290870016E8; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 7060720E3700; Mon, 7 Nov 2011 14:59:40 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 8/8] xfsdump: enable multiple streams Date: Mon, 7 Nov 2011 14:58:31 -0600 Message-Id: <1320699511-12281-9-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean IRIX contained an environment referred to as "miniroot" where sproc threads were either not available, or at least not used in xfsdump. Throughout xfsdump there's a "miniroot" variable which indicates whether or not thread support is enabled. On Linux this variable has always been false in order to disable support for multiple streams. Now that the threading infracstructure has been converted over to pthreads, this patch removes the "miniroot" variable and enables the option of using multiple streams. Note that another feature in xfsdump, using a ring buffer for I/O to tapes, also depends on thread support. I'm leaving that disabled for now until more testing has been done. Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- common/drive.c | 20 +------ common/drive.h | 8 +-- common/drive_minrmt.c | 15 +++--- common/drive_scsitape.c | 15 +++--- common/drive_simple.c | 8 ++-- common/main.c | 124 +++++----------------------------------------- dump/content.c | 35 ++++++------- dump/getopt.h | 4 +- man/man8/xfsdump.8 | 8 +-- man/man8/xfsrestore.8 | 4 +- restore/content.c | 23 ++++----- restore/getopt.h | 4 +- 12 files changed, 74 insertions(+), 194 deletions(-) diff --git a/common/drive.c b/common/drive.c index 227fecd..32a7191 100644 --- a/common/drive.c +++ b/common/drive.c @@ -81,7 +81,7 @@ static drive_strategy_t *strategypp[] = { * specified on the command line. */ bool_t -drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) +drive_init1( int argc, char *argv[ ] ) { intgen_t c; ix_t driveix; @@ -103,18 +103,6 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) } } - /* validate drive count - */ - if ( singlethreaded && drivecnt > 1 ) { - mlog( MLOG_NORMAL, _( - "too many -%c arguments: " - "maximum is %d when running in miniroot\n"), - GETOPT_DUMPDEST, - 1 ); - usage( ); - return BOOL_FALSE; - } - /* allocate an array to hold ptrs to drive descriptors */ if (drivecnt > 0) { @@ -221,8 +209,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) intgen_t score; score = ( * sp->ds_match )( argc, argv, - drivep, - singlethreaded ); + drivep ); if ( ! bestsp || score > bestscore ) { bestsp = sp; bestscore = score; @@ -237,8 +224,7 @@ drive_init1( int argc, char *argv[ ], bool_t singlethreaded ) bestsp->ds_description ); ok = ( * bestsp->ds_instantiate )( argc, argv, - drivep, - singlethreaded ); + drivep ); if ( ! ok ) { return BOOL_FALSE; } diff --git a/common/drive.h b/common/drive.h index 4dc4270..f693976 100644 --- a/common/drive.h +++ b/common/drive.h @@ -140,15 +140,13 @@ struct drive_strategy { */ intgen_t ( * ds_match )( intgen_t argc, char *argv[ ], - struct drive *drivep, - bool_t singlethreaded ); + struct drive *drivep ); /* returns degree of match. drivep has been pre-allocated * and initialized with generic info. */ bool_t ( * ds_instantiate )( intgen_t argc, char *argv[ ], - struct drive *drivep, - bool_t singlethreaded ); + struct drive *drivep ); /* creates a drive manager instance, by filling in the * blanks of the pre-allocated drive descriptor * returns FALSE on failure. @@ -648,7 +646,7 @@ extern size_t partialmax; * * Returns FALSE if utility should be aborted. */ -extern bool_t drive_init1( int argc, char *argv[], bool_t singlethreaded ); +extern bool_t drive_init1( int argc, char *argv[] ); /* drive_init2 - allocate and initialize read and write hdr buffers, diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c index 3ff4d0f..5f5f96c 100644 --- a/common/drive_minrmt.c +++ b/common/drive_minrmt.c @@ -257,8 +257,8 @@ extern int rmtwrite( int, const void *, uint); /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* manager operations */ @@ -404,7 +404,7 @@ static u_int32_t cmdlineblksize = 0; */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { intgen_t fd; intgen_t c; @@ -464,7 +464,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; intgen_t c; @@ -487,9 +487,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) ASSERT( contextp ); memset( ( void * )contextp, 0, sizeof( *contextp )); - /* transfer indication of singlethreadedness to context + /* do not enable a separate I/O thread, + * more testing to be done first... */ - contextp->dc_singlethreadedpr = singlethreaded; + contextp->dc_singlethreadedpr = BOOL_TRUE; /* scan the command line for the I/O buffer ring length * and record checksum request @@ -580,7 +581,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ - if ( singlethreaded ) { + if ( contextp->dc_singlethreadedpr ) { contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ ); ASSERT( contextp->dc_bufp ); } else { diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c index f24d604..71b809f 100644 --- a/common/drive_scsitape.c +++ b/common/drive_scsitape.c @@ -299,8 +299,8 @@ extern int rmtwrite( int, const void *, uint); /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* manager operations */ @@ -506,7 +506,7 @@ is_scsi_driver(char *pathname) */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { struct mtget mt_stat; intgen_t fd; @@ -544,7 +544,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; intgen_t c; @@ -567,9 +567,10 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) ASSERT( contextp ); memset( ( void * )contextp, 0, sizeof( *contextp )); - /* transfer indication of singlethreadedness to context + /* do not enable a separate I/O thread, + * more testing to be done first... */ - contextp->dc_singlethreadedpr = singlethreaded; + contextp->dc_singlethreadedpr = BOOL_TRUE; /* scan the command line for the I/O buffer ring length * and record checksum request @@ -670,7 +671,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ - if ( singlethreaded ) { + if ( contextp->dc_singlethreadedpr ) { contextp->dc_bufp = ( char * )memalign( PGSZ, STAPE_MAX_RECSZ ); ASSERT( contextp->dc_bufp ); } else { diff --git a/common/drive_simple.c b/common/drive_simple.c index cb88aa8..086d162 100644 --- a/common/drive_simple.c +++ b/common/drive_simple.c @@ -98,8 +98,8 @@ extern size_t pgsz; /* strategy functions */ -static intgen_t ds_match( int, char *[], drive_t *, bool_t ); -static intgen_t ds_instantiate( int, char *[], drive_t *, bool_t ); +static intgen_t ds_match( int, char *[], drive_t * ); +static intgen_t ds_instantiate( int, char *[], drive_t * ); /* declare manager operators */ @@ -178,7 +178,7 @@ static drive_ops_t drive_ops = { */ /* ARGSUSED */ static intgen_t -ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_match( int argc, char *argv[], drive_t *drivep ) { bool_t isrmtpr; struct stat64 statbuf; @@ -221,7 +221,7 @@ ds_match( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) */ /*ARGSUSED*/ static bool_t -ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) +ds_instantiate( int argc, char *argv[], drive_t *drivep ) { drive_context_t *contextp; diff --git a/common/main.c b/common/main.c index 38b3889..5880723 100644 --- a/common/main.c +++ b/common/main.c @@ -87,10 +87,6 @@ bool_t preemptchk( int ); static bool_t loadoptfile( int *argcp, char ***argvp ); static char * stripquotes( char *p ); static void shiftleftby1( char *p, char *endp ); -static bool_t in_miniroot_heuristic( void ); -#ifdef HIDDEN -static void mrh_sighandler( int ); -#endif static void sighandler( int ); static int childmain( void * ); static bool_t sigint_dialog( void ); @@ -111,11 +107,6 @@ intgen_t version = 3; intgen_t subversion = 0; char *progname = 0; /* used in all error output */ char *homedir = 0; /* directory invoked from */ -#ifdef HIDDEN -bool_t miniroot = BOOL_FALSE; -#else -bool_t miniroot = BOOL_TRUE; -#endif /* HIDDEN */ bool_t pipeline = BOOL_FALSE; bool_t stdoutpiped = BOOL_FALSE; pthread_t parenttid; @@ -215,16 +206,11 @@ main( int argc, char *argv[] ) if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } - /* scan the command line for the miniroot, info, progress + /* scan the command line for the info, progress * report options, and stacksz. */ minstacksz = MINSTACKSZ; maxstacksz = MAXSTACKSZ; -#ifdef HIDDEN - miniroot = BOOL_FALSE; -#else - miniroot = BOOL_TRUE; -#endif /* HIDDEN */ infoonly = BOOL_FALSE; progrpt_enabledpr = BOOL_FALSE; optind = 1; @@ -275,9 +261,6 @@ main( int argc, char *argv[] ) } maxstacksz = tmpstacksz; break; - case GETOPT_MINIROOT: - miniroot = BOOL_TRUE; - break; case GETOPT_HELP: infoonly = BOOL_TRUE; mlog_exit_hint(RV_USAGE); @@ -349,13 +332,6 @@ main( int argc, char *argv[] ) return mlog_exit(EXIT_ERROR, RV_INIT); } - /* perform an experiment to determine if we are in the miniroot. - * various features will be disallowed if in miniroot. - */ - if ( ! miniroot && in_miniroot_heuristic( )) { - miniroot = BOOL_TRUE; - } - /* initialize message logging (stage 2) - allocate the message lock */ ok = mlog_init2( ); @@ -473,7 +449,7 @@ main( int argc, char *argv[] ) * terribly time-consuming here. A second initialization pass * will be done shortly. */ - ok = drive_init1( argc, argv, miniroot ); + ok = drive_init1( argc, argv ); if ( ! ok ) { return mlog_exit(EXIT_ERROR, RV_INIT); } @@ -503,11 +479,7 @@ main( int argc, char *argv[] ) VERSION, version, subversion ); - if ( miniroot ) { - mlog( MLOG_VERBOSE | MLOG_BARE, _( - " - " - "Running single-threaded\n") ); - } else if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) { + if ( ! pipeline && ! stdoutpiped && sistr && dlog_allowed( )) { mlog( MLOG_VERBOSE | MLOG_BARE, _( " - " "type %s for status and control\n"), @@ -531,11 +503,10 @@ main( int argc, char *argv[] ) */ mlog_tell_streamcnt( drivecnt ); - /* initialize the state of signal processing. if miniroot or - * pipeline, just want to exit when a signal is received. otherwise, - * hold signals so they don't interfere with sys calls; they will - * be released at pre-emption points and upon pausing in the main - * loop. + /* initialize the state of signal processing. if in a pipeline, just + * want to exit when a signal is received. otherwise, hold signals so + * they don't interfere with sys calls; they will be released at + * pre-emption points and upon pausing in the main loop. */ sigfillset(&sa.sa_mask); @@ -551,7 +522,7 @@ main( int argc, char *argv[] ) sigaction( SIGPIPE, &sa, NULL ); sigaction( SIGCHLD, &sa, NULL ); - if ( ! miniroot && ! pipeline ) { + if ( ! pipeline ) { sigset_t blocked_set; stop_in_progress = BOOL_FALSE; @@ -594,10 +565,9 @@ main( int argc, char *argv[] ) return mlog_exit(EXIT_ERROR, RV_INIT); } - /* if miniroot or a pipeline, go single-threaded - * with just one stream. + /* if in a pipeline, go single-threaded with just one stream. */ - if ( miniroot || pipeline ) { + if ( pipeline ) { intgen_t exitcode; sa.sa_handler = sighandler; @@ -977,9 +947,6 @@ usage( void ) ULO(_("(show verbosity in messages)"), GETOPT_SHOWLOGLEVEL ); #endif /* REVEAL */ ULO(_(""), GETOPT_RINGLEN ); -#ifdef REVEAL - ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT ); -#endif /* REVEAL */ ULN(_("- (stdout)") ); ULN(_("") ); #endif /* DUMP */ @@ -1030,9 +997,6 @@ usage( void ) #endif /* REVEAL */ ULO(_(" ..."), GETOPT_NOSUBTREE ); ULO(_(""), GETOPT_RINGLEN ); -#ifdef REVEAL - ULO(_("(miniroot restrictions)"), GETOPT_MINIROOT ); -#endif /* REVEAL */ ULN(_("- (stdin)") ); ULN(_("") ); #endif /* RESTORE */ @@ -1081,9 +1045,9 @@ preemptchk( int flg ) return BOOL_FALSE; } - /* signals not caught in these cases + /* signals not caught if in a pipeline */ - if ( miniroot || pipeline ) { + if ( pipeline ) { return BOOL_FALSE; } @@ -1400,66 +1364,6 @@ loadoptfile( intgen_t *argcp, char ***argvp ) return BOOL_TRUE; } -#ifdef HIDDEN -static pid_t mrh_cid; -#endif - -static bool_t -in_miniroot_heuristic( void ) -{ - return BOOL_TRUE; - -#ifdef HIDDEN - SIG_PF prev_handler_hup; - SIG_PF prev_handler_term; - SIG_PF prev_handler_int; - SIG_PF prev_handler_quit; - SIG_PF prev_handler_cld; - bool_t in_miniroot; - - /* attempt to call sproc. - */ - prev_handler_hup = sigset( SIGHUP, SIG_IGN ); - prev_handler_term = sigset( SIGTERM, SIG_IGN ); - prev_handler_int = sigset( SIGINT, SIG_IGN ); - prev_handler_quit = sigset( SIGQUIT, SIG_IGN ); - prev_handler_cld = sigset( SIGCLD, mrh_sighandler ); - ( void )sighold( SIGCLD ); - mrh_cid = ( pid_t )sproc( ( void ( * )( void * ))exit, PR_SALL, 0 ); - if ( mrh_cid < 0 ) { - in_miniroot = BOOL_TRUE; - } else { - while ( mrh_cid >= 0 ) { - ( void )sigpause( SIGCLD ); - } - in_miniroot = BOOL_FALSE; - } - ( void )sigset( SIGHUP, prev_handler_hup ); - ( void )sigset( SIGTERM, prev_handler_term ); - ( void )sigset( SIGINT, prev_handler_int ); - ( void )sigset( SIGQUIT, prev_handler_quit ); - ( void )sigset( SIGCLD, prev_handler_cld ); - - return in_miniroot; -#endif /* HIDDEN */ -} - -#ifdef HIDDEN -static void -mrh_sighandler( int signo ) -{ - if ( signo == SIGCLD ) { - pid_t cid; - intgen_t stat; - - cid = wait( &stat ); - if ( cid == mrh_cid ) { - mrh_cid = -1; - } - } -} -#endif - /* parent and children share this handler. */ static void @@ -1470,9 +1374,9 @@ sighandler( int signo ) if ( dlog_sighandler( signo ) ) return; - /* if in miniroot, don't do anything risky. just quit. + /* if in pipeline, don't do anything risky. just quit. */ - if ( miniroot || pipeline ) { + if ( pipeline ) { intgen_t rval; mlog( MLOG_TRACE | MLOG_NOTE | MLOG_NOLOCK | MLOG_PROC, diff --git a/dump/content.c b/dump/content.c index 33f1166..3a7f508 100644 --- a/dump/content.c +++ b/dump/content.c @@ -244,7 +244,6 @@ typedef struct pds pds_t; extern void usage( void ); extern bool_t preemptchk( int ); extern char *homedir; -extern bool_t miniroot; extern bool_t pipeline; extern bool_t stdoutpiped; extern char *sistr; @@ -2352,7 +2351,7 @@ content_stream_dump( ix_t strmix ) inomap_next_nondir, inomap_contextp, ( intgen_t * )&rv, - ( miniroot || pipeline ) ? + pipeline ? (bool_t (*)(int))preemptchk : 0, bstatbufp, bstatbuflen ); @@ -2586,24 +2585,22 @@ decision_more: * from all streams have been registered. */ if ( drivep->d_capabilities & DRIVE_CAP_FILES ) { - if ( ! miniroot ) { - if ( stream_cnt( ) > 1 ) { - mlog( MLOG_VERBOSE, _( - "waiting for synchronized " - "session inventory dump\n") ); - sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC; - } + if ( stream_cnt( ) > 1 ) { + mlog( MLOG_VERBOSE, _( + "waiting for synchronized " + "session inventory dump\n") ); + sc_stat_pds[ strmix ].pds_phase = PDS_INVSYNC; + } - /* first be sure all threads have begun - */ - while ( sc_thrdsarrivedcnt < drivecnt ) { - sleep( 1 ); - } - /* now wait for survivors to checkin - */ - while ( sc_thrdsdonecnt < stream_cnt( )) { - sleep( 1 ); - } + /* first be sure all threads have begun + */ + while ( sc_thrdsarrivedcnt < drivecnt ) { + sleep( 1 ); + } + /* now wait for survivors to checkin + */ + while ( sc_thrdsdonecnt < stream_cnt( )) { + sleep( 1 ); } /* proceeed */ diff --git a/dump/getopt.h b/dump/getopt.h index 8a55631..ba26c93 100644 --- a/dump/getopt.h +++ b/dump/getopt.h @@ -27,7 +27,7 @@ * facilitating easy changes. */ -#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:Z" +#define GETOPT_CMDSTRING "ab:c:d:ef:hl:mop:qs:t:v:z:AB:CDEFG:H:I:JL:M:NO:PRSTUVWY:" #define GETOPT_DUMPASOFFLINE 'a' /* dump DMF dualstate files as offline */ #define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */ @@ -80,6 +80,6 @@ #define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */ /* 'X' */ #define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */ -#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */ +/* 'Z' used on irix for miniroot */ #endif /* GETOPT_H */ diff --git a/man/man8/xfsdump.8 b/man/man8/xfsdump.8 index 46e6bd5..5360378 100644 --- a/man/man8/xfsdump.8 +++ b/man/man8/xfsdump.8 @@ -66,10 +66,6 @@ option. Other techniques, such as making a second copy of the dump image, provide more protection against media failures than multiple media files will. .P -However, the current implementation in Linux only supports one destination -and running single threaded. Therefore, the above comments -regarding multiple streams describe the possible future capabilities. -.P .I xfsdump maintains an online dump inventory in \f2/var/lib/xfsdump/inventory\f1. The @@ -391,8 +387,8 @@ Specify I/O buffer ring length. uses a ring of output buffers to achieve maximum throughput when dumping to tape drives. The default ring length is 3. -However, this is only supported when running multi-threaded -which has not been done for Linux yet - making this option benign. +However, this is not currently enabled on +Linux yet, making this option benign. .TP 5 .B \- A lone diff --git a/man/man8/xfsrestore.8 b/man/man8/xfsrestore.8 index 8e33ee9..aad97fa 100644 --- a/man/man8/xfsrestore.8 +++ b/man/man8/xfsrestore.8 @@ -426,8 +426,8 @@ Specify I/O buffer ring length. uses a ring of input buffers to achieve maximum throughput when restoring from tape drives. The default ring length is 3. -However, this is only supported when running multi-threaded -which has not been done for Linux yet - making this option benign. +However, this is not currently enabled on +Linux yet, making this option benign. .TP 5 .B \- A lone diff --git a/restore/content.c b/restore/content.c index 0108a40..a9e0b20 100644 --- a/restore/content.c +++ b/restore/content.c @@ -660,7 +660,6 @@ typedef struct tran tran_t; extern void usage( void ); extern bool_t preemptchk( void ); extern char *homedir; -extern bool_t miniroot; extern bool_t pipeline; extern bool_t stdoutpiped; extern char *sistr; @@ -2544,14 +2543,12 @@ content_stream_restore( ix_t thrdix ) } tranp->t_sync5 = SYNC_BUSY; unlock( ); - if ( ! miniroot ) { - if ( drivecnt > 1 ) { - mlog( MLOG_TRACE, - "waiting for other streams to exit\n" ); - } - while ( cldmgr_otherstreamsremain( thrdix )) { - sleep( 1 ); - } + if ( drivecnt > 1 ) { + mlog( MLOG_TRACE, + "waiting for other streams to exit\n" ); + } + while ( cldmgr_otherstreamsremain( thrdix )) { + sleep( 1 ); } mlog( MLOG_DEBUG, @@ -3006,10 +3003,10 @@ applydirdump( drive_t *drivep, return RV_INTR; } - /* if miniroot or pipeline , call preemptchk( ) to + /* if in a pipeline , call preemptchk( ) to * print status reports */ - if ( miniroot || pipeline ) + if ( pipeline ) { mlog( MLOG_DEBUG , "preemptchk( )\n"); @@ -3543,10 +3540,10 @@ applynondirdump( drive_t *drivep, fhdrp->fh_offset ); } - /* if miniroot or pipeline , call preemptchk( ) to + /* if in a pipeline , call preemptchk( ) to * print status reports */ - if ( miniroot || pipeline ) + if ( pipeline ) { mlog( MLOG_DEBUG , "preemptchk( )\n"); diff --git a/restore/getopt.h b/restore/getopt.h index 4ee865d..63568de 100644 --- a/restore/getopt.h +++ b/restore/getopt.h @@ -26,7 +26,7 @@ * purpose is to contain that command string. */ -#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:Z" +#define GETOPT_CMDSTRING "a:b:c:def:himn:op:qrs:tv:wABCDEFG:H:I:JL:M:NO:PQRS:TUVWX:Y:" #define GETOPT_WORKSPACE 'a' /* workspace dir (content.c) */ #define GETOPT_BLOCKSIZE 'b' /* blocksize for rmt */ @@ -79,6 +79,6 @@ #define GETOPT_SHOWLOGLEVEL 'W' /* show level of log messages */ #define GETOPT_NOSUBTREE 'X' /* subtree to exclude */ #define GETOPT_RINGLEN 'Y' /* specify I/O buffer ring length */ -#define GETOPT_MINIROOT 'Z' /* apply miniroot restrictions */ +/* 'Z' used on irix for miniroot */ #endif /* GETOPT_H */ -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KxiJG076057 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id D05FCAC003; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 77C6070016E1; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 2868121E23C8; Mon, 7 Nov 2011 14:59:39 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 1/8] xfsdump: link with libpthread Date: Mon, 7 Nov 2011 14:58:24 -0600 Message-Id: <1320699511-12281-2-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch changes xfsdump/xfsrestore to link with libpthread in preparation for enabling multi-stream support. The libpthread checks are the same as those used in xfsprogs. Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- configure.in | 3 +++ dump/Makefile | 2 +- include/builddefs.in | 1 + m4/Makefile | 1 + m4/package_pthread.m4 | 19 +++++++++++++++++++ restore/Makefile | 2 +- 6 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 m4/package_pthread.m4 diff --git a/configure.in b/configure.in index b7ea1eb..8d4a256 100644 --- a/configure.in +++ b/configure.in @@ -67,6 +67,9 @@ AC_MULTILIB($enable_lib64) AC_PACKAGE_NEED_UUID_H AC_PACKAGE_NEED_UUIDCOMPARE +AC_PACKAGE_NEED_PTHREAD_H +AC_PACKAGE_NEED_PTHREADMUTEXINIT + AC_PACKAGE_NEED_NCURSES_H AC_PACKAGE_WANT_WORKING_LIBNCURSES diff --git a/dump/Makefile b/dump/Makefile index ce964a0..d193f68 100644 --- a/dump/Makefile +++ b/dump/Makefile @@ -89,7 +89,7 @@ HFILES = $(LOCALINCL) LHFILES = $(COMMINCL) $(INVINCL) LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON) LDIRT = $(LINKS) -LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) +LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD) LTDEPENDENCIES = $(LIBRMT) LCFLAGS = -DDUMP diff --git a/include/builddefs.in b/include/builddefs.in index f69c038..d5cc8d3 100644 --- a/include/builddefs.in +++ b/include/builddefs.in @@ -16,6 +16,7 @@ CFLAGS = @CFLAGS@ LIBRMT = $(TOPDIR)/librmt/librmt.la LIBXFS = @libxfs@ LIBATTR = @libattr@ +LIBPTHREAD = @libpthread@ LIBUUID = @libuuid@ LIBCURSES = @libcurses@ LIBHANDLE = @libhdl@ diff --git a/m4/Makefile b/m4/Makefile index 8ce63f8..9a35056 100644 --- a/m4/Makefile +++ b/m4/Makefile @@ -17,6 +17,7 @@ LSRCFILES = \ package_attrdev.m4 \ package_globals.m4 \ package_ncurses.m4 \ + package_pthread.m4 \ package_utilies.m4 \ package_uuiddev.m4 \ package_xfslibs.m4 \ diff --git a/m4/package_pthread.m4 b/m4/package_pthread.m4 new file mode 100644 index 0000000..be21d29 --- /dev/null +++ b/m4/package_pthread.m4 @@ -0,0 +1,19 @@ +AC_DEFUN([AC_PACKAGE_NEED_PTHREAD_H], + [ AC_CHECK_HEADERS(pthread.h) + if test $ac_cv_header_pthread_h = no; then + AC_CHECK_HEADERS(pthread.h,, [ + echo + echo 'FATAL ERROR: could not find a valid pthread header.' + exit 1]) + fi + ]) + +AC_DEFUN([AC_PACKAGE_NEED_PTHREADMUTEXINIT], + [ AC_CHECK_LIB(pthread, pthread_mutex_init,, [ + echo + echo 'FATAL ERROR: could not find a valid pthread library.' + exit 1 + ]) + libpthread=-lpthread + AC_SUBST(libpthread) + ]) diff --git a/restore/Makefile b/restore/Makefile index 003ea16..ce3d6b4 100644 --- a/restore/Makefile +++ b/restore/Makefile @@ -99,7 +99,7 @@ HFILES = $(LOCALINCL) LHFILES = $(COMMINCL) $(INVINCL) LINKS = $(COMMINCL) $(COMMON) $(INVINCL) $(INVCOMMON) LDIRT = $(LINKS) -LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) +LLDLIBS = $(LIBUUID) $(LIBHANDLE) $(LIBATTR) $(LIBRMT) $(LIBPTHREAD) LTDEPENDENCIES = $(LIBRMT) LCFLAGS = -DRESTORE -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Kxiak076055 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id F035B8F8096; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 9C50670016E7; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 681C120E36FF; Mon, 7 Nov 2011 14:59:40 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 7/8] xfsdump: path lookup cache must be thread specific Date: Mon, 7 Nov 2011 14:58:30 -0600 Message-Id: <1320699511-12281-8-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The xfsrestore path lookup routine (maps an ino + generation number to a pathname) contains a cache for fast lookups of consecutive files in the same parent directory. This cache either needs to be protected by a lock or a cache is needed per thread. I took the latter approach since it doesn't add lock contention and it increases the cache hit rate (it's relatively unlikely that each stream is processing files in the same directory). Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- restore/tree.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/restore/tree.c b/restore/tree.c index 7bddddf..9e4e83c 100644 --- a/restore/tree.c +++ b/restore/tree.c @@ -3441,7 +3441,7 @@ Node2path( nh_t nh, char *path, char *errmsg ) static intgen_t Node2path_recurse( nh_t nh, char *buf, intgen_t bufsz, intgen_t level ) { - static path_cache_t cache = { NH_NULL, 0, "" }; + static __thread path_cache_t cache = { NH_NULL, 0, "" }; node_t *np; nh_t parh; xfs_ino_t ino; -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:45 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KxiXO076060 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id 11664AC008; Mon, 7 Nov 2011 12:59:41 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 9B32A70016E6; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 5FE3220E36FE; Mon, 7 Nov 2011 14:59:40 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 6/8] xfsdump: process thread exit status Date: Mon, 7 Nov 2011 14:58:29 -0600 Message-Id: <1320699511-12281-7-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When IRIX sprocs were in use, the main thread was notified of a thread exit just as if a child process exited -- it received SIGCHLD. The main thread would grab the pid and exit status, then call cldmgr_died() to inform it that the child was gone so the slot in the child array could be freed up for reuse. This patch implements a similar mechanism for pthreads. The "c_busy" field in struct cld has been replaced with a "c_state" field that indicates whether the array slot is free (C_AVAIL), in use (C_ALIVE), or is waiting to be joined (C_EXITED). Additionally a "c_exit_code" field has been added to store the thread's exit value. Normally this is set when the thread entry function returns, but it is initialized to EXIT_INTERRUPT in case the thread is cancelled or calls pthread_exit() rather than returning (neither of which happens in the code today). When the child thread starts, it registers a pthread cleanup handler which takes care of marking the child as C_EXITED and notifies the main thread that a child is gone. Doing this in a cleanup handler ensures that it's done regardless of how the thread exits. The main thread's loop is based around sigsuspsend(), so the notification is done by sending SIGUSR1. The main thread will then call cldmgr_join() to join all exited threads and obtain their exit status. Additional changes: * cldmgr_findbypid() has been removed, it's no longer referenced. * stream_dead() no longer grabs the lock(), because it's called only from cldmgr_join() which already holds the lock(). Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- common/cldmgr.c | 88 +++++++++++++++++++++++++++++++++++------------------- common/cldmgr.h | 7 +++- common/main.c | 33 +++++--------------- common/stream.c | 3 +- common/stream.h | 1 + 5 files changed, 73 insertions(+), 59 deletions(-) diff --git a/common/cldmgr.c b/common/cldmgr.c index 4574834..be7de34 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -26,6 +26,7 @@ #include #include +#include "exit.h" #include "types.h" #include "lock.h" #include "qlock.h" @@ -36,8 +37,12 @@ extern size_t pgsz; #define CLD_MAX ( STREAM_SIMMAX * 2 ) + +typedef enum { C_AVAIL, C_ALIVE, C_EXITED } state_t; + struct cld { - bool_t c_busy; + state_t c_state; + intgen_t c_exit_code; pthread_t c_tid; ix_t c_streamix; int ( * c_entry )( void *arg1 ); @@ -50,8 +55,8 @@ static cld_t cld[ CLD_MAX ]; static bool_t cldmgr_stopflag; static cld_t *cldmgr_getcld( void ); -static cld_t * cldmgr_findbytid( pthread_t ); static void *cldmgr_entry( void * ); +static void cldmgr_cleanup( void * ); /* REFERENCED */ static pthread_t cldmgr_parenttid; @@ -87,6 +92,7 @@ cldmgr_create( int ( * entry )( void *arg1 ), return BOOL_FALSE; } + cldp->c_exit_code = EXIT_INTERRUPT; cldp->c_streamix = streamix; cldp->c_entry = entry; cldp->c_arg1 = arg1; @@ -117,18 +123,37 @@ cldmgr_stop( void ) cldmgr_stopflag = BOOL_TRUE; } -void -cldmgr_died( pthread_t tid ) +intgen_t +cldmgr_join( void ) { - cld_t *cldp = cldmgr_findbytid( tid ); + cld_t *p = cld; + cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); + intgen_t xc = EXIT_NORMAL; - if ( ! cldp ) { - return; - } - cldp->c_busy = BOOL_FALSE; - if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_dead( tid ); + lock(); + for ( ; p < ep ; p++ ) { + if ( p->c_state == C_EXITED ) { + if ( ( intgen_t )( p->c_streamix ) >= 0 ) { + stream_dead( p->c_tid ); + } + pthread_join( p->c_tid, NULL ); + if ( p->c_exit_code != EXIT_NORMAL && xc != EXIT_FAULT ) + xc = p->c_exit_code; + if ( p->c_exit_code != EXIT_NORMAL ) { + mlog( MLOG_DEBUG | MLOG_PROC | MLOG_NOLOCK, + "child (thread %lu) requested stop: " + "exit code %d (%s)\n", + p->c_tid, p->c_exit_code, + exit_codestring( p->c_exit_code )); + } + + // reinit this child for reuse + memset( ( void * )p, 0, sizeof( cld_t )); + } } + unlock(); + + return xc; } bool_t @@ -147,7 +172,7 @@ cldmgr_remainingcnt( void ) cnt = 0; lock( ); for ( ; p < ep ; p++ ) { - if ( p->c_busy ) { + if ( p->c_state == C_ALIVE ) { cnt++; } } @@ -164,7 +189,7 @@ cldmgr_otherstreamsremain( ix_t streamix ) lock( ); for ( ; p < ep ; p++ ) { - if ( p->c_busy && p->c_streamix != streamix ) { + if ( p->c_state == C_ALIVE && p->c_streamix != streamix ) { unlock( ); return BOOL_TRUE; } @@ -182,8 +207,8 @@ cldmgr_getcld( void ) lock(); for ( ; p < ep ; p++ ) { - if ( ! p->c_busy ) { - p->c_busy = BOOL_TRUE; + if ( p->c_state == C_AVAIL ) { + p->c_state = C_ALIVE; break; } } @@ -192,27 +217,14 @@ cldmgr_getcld( void ) return ( p < ep ) ? p : 0; } -static cld_t * -cldmgr_findbytid( pthread_t tid ) -{ - cld_t *p = cld; - cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); - - for ( ; p < ep ; p++ ) { - if ( p->c_busy && pthread_equal( p->c_tid, tid )) { - break; - } - } - - return ( p < ep ) ? p : 0; -} - static void * cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; pthread_t tid = pthread_self( ); + pthread_cleanup_push( cldmgr_cleanup, arg1 ); + if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { stream_register( tid, ( intgen_t )cldp->c_streamix ); } @@ -220,7 +232,21 @@ cldmgr_entry( void *arg1 ) "thread %lu created for stream %d\n", tid, cldp->c_streamix ); + cldp->c_exit_code = ( * cldp->c_entry )( cldp->c_arg1 ); + + pthread_cleanup_pop( 1 ); - ( * cldp->c_entry )( cldp->c_arg1 ); return NULL; } + +static void +cldmgr_cleanup( void *arg1 ) +{ + cld_t *cldp = ( cld_t * )arg1; + + lock(); + cldp->c_state = C_EXITED; + // signal the main thread to look for exited threads + kill( getpid( ), SIGUSR1 ); + unlock(); +} diff --git a/common/cldmgr.h b/common/cldmgr.h index e393b82..1df0c0c 100644 --- a/common/cldmgr.h +++ b/common/cldmgr.h @@ -39,9 +39,12 @@ extern bool_t cldmgr_create( int ( * entry )( void *arg1 ), */ extern void cldmgr_stop( void ); -/* cldmgr_died - tells the child manager that the child died +/* cldmgr_join - join child threads that have exited. + * returns EXIT_NORMAL if all exited normally (or no threads have exited), + * EXIT_FAULT if any threads requested a core dump, or another EXIT_* + * value if any threads exited abnormally. */ -extern void cldmgr_died( pthread_t tid ); +extern intgen_t cldmgr_join( void ); /* cldmgr_stop_requested - returns TRUE if the child should gracefully * terminate. diff --git a/common/main.c b/common/main.c index d4dbe28..38b3889 100644 --- a/common/main.c +++ b/common/main.c @@ -137,10 +137,6 @@ static bool_t sighup_received; static bool_t sigterm_received; static bool_t sigquit_received; static bool_t sigint_received; -static size_t prbcld_cnt; -static pid_t prbcld_pid; -static intgen_t prbcld_xc; -static intgen_t prbcld_signo; /* REFERENCED */ static intgen_t sigstray_received; static bool_t progrpt_enabledpr; @@ -168,6 +164,8 @@ main( int argc, char *argv[] ) intgen_t exitcode; rlim64_t tmpstacksz; struct sigaction sa; + intgen_t prbcld_xc = EXIT_NORMAL; + intgen_t xc; bool_t ok; /* REFERENCED */ int rval; @@ -563,7 +561,6 @@ main( int argc, char *argv[] ) sigint_received = BOOL_FALSE; sigquit_received = BOOL_FALSE; sigstray_received = BOOL_FALSE; - prbcld_cnt = 0; alarm( 0 ); @@ -573,6 +570,7 @@ main( int argc, char *argv[] ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); + sigaddset( &blocked_set, SIGUSR1 ); pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); sa.sa_handler = sighandler; @@ -581,6 +579,7 @@ main( int argc, char *argv[] ) sigaction( SIGTERM, &sa, NULL ); sigaction( SIGQUIT, &sa, NULL ); sigaction( SIGALRM, &sa, NULL ); + sigaction( SIGUSR1, &sa, NULL ); } /* do content initialization. @@ -710,31 +709,16 @@ main( int argc, char *argv[] ) * stop. furthermore, note that core should be dumped if * the child explicitly exited with EXIT_FAULT. */ - if ( prbcld_cnt ) { - if ( prbcld_xc == EXIT_FAULT || prbcld_signo != 0 ) { + xc = cldmgr_join( ); + if ( xc ) { + if ( xc == EXIT_FAULT ) { coredump_requested = BOOL_TRUE; stop_timeout = ABORT_TIMEOUT; } else { stop_timeout = STOP_TIMEOUT; } + prbcld_xc = xc; stop_requested = BOOL_TRUE; - if ( prbcld_xc != EXIT_NORMAL ) { - mlog( MLOG_DEBUG | MLOG_PROC, - "child (pid %d) requested stop: " - "exit code %d (%s)\n", - prbcld_pid, - prbcld_xc, - exit_codestring( prbcld_xc )); - } else if ( prbcld_signo ) { - ASSERT( prbcld_signo ); - mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC, - _("child (pid %d) faulted: " - "signal number %d (%s)\n"), - prbcld_pid, - prbcld_signo, - sig_numstring( prbcld_signo )); - } - prbcld_cnt = 0; } /* all children died normally. break out. @@ -1528,6 +1512,7 @@ sighandler( int signo ) sigquit_received = BOOL_TRUE; break; case SIGALRM: + case SIGUSR1: break; default: sigstray_received = signo; diff --git a/common/stream.c b/common/stream.c index 48e25ee..6704661 100644 --- a/common/stream.c +++ b/common/stream.c @@ -86,19 +86,18 @@ stream_register( pthread_t tid, intgen_t streamix ) p->s_exit_hint = RV_NONE; } +/* NOTE: lock() must be held when calling stream_dead() */ void stream_dead( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); - lock(); for ( ; p < ep ; p++ ) if ( pthread_equal( p->s_tid, tid ) ) { p->s_state = S_ZOMBIE; break; } - unlock(); ASSERT( p < ep ); } diff --git a/common/stream.h b/common/stream.h index 292792e..4b3799f 100644 --- a/common/stream.h +++ b/common/stream.h @@ -43,6 +43,7 @@ typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t; extern void stream_init( void ); extern void stream_register( pthread_t tid, intgen_t streamix ); +/* NOTE: lock() must be held when calling stream_dead */ extern void stream_dead( pthread_t tid ); extern void stream_free( pthread_t tid ); extern int stream_find_all( stream_state_t states[], -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7Kxhbh076051 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay2.corp.sgi.com (Postfix) with ESMTP id B4BEE304067; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 77CD570016E2; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 36CF520E36F6; Mon, 7 Nov 2011 14:59:39 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 2/8] xfsdump: remove multi-stream synchronous dir dump Date: Mon, 7 Nov 2011 14:58:25 -0600 Message-Id: <1320699511-12281-3-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When doing multi-stream dumps, the directories are dumped by each stream in lock-step fashion. A stream backs up a given directory, then waits for all others to finish that directory, then they all move on to the next one. During restore a single stream is chosen to apply the directory dump. This patch changes xfsdump so that only one stream dumps the directories. The other streams begin dumping files immediately. This results in a faster backup, which is presumably why multiple streams are being used in the first place. It also removes the barrier routines in qlock.c as they were only used by the "SYNCDIR" code. Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- common/qlock.c | 37 ---------- common/qlock.h | 11 --- dump/content.c | 193 ++++++++++++----------------------------------------- restore/content.c | 33 +++++++++ 4 files changed, 75 insertions(+), 199 deletions(-) diff --git a/common/qlock.c b/common/qlock.c index fd0e98d..6c9ee16 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -568,43 +568,6 @@ return 0; #endif /* HIDDEN */ } -qbarrierh_t -qbarrier_alloc( void ) -{ -#ifdef HIDDEN - barrier_t *barrierp; - - /* sanity checks - */ - ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - /* allocate a us barrier - */ - barrierp = new_barrier( qlock_usp ); - ASSERT( barrierp ); - - return ( qbarrierh_t )barrierp; -#else -return 0; -#endif /* HIDDEN */ -} - -void -qbarrier( qbarrierh_t qbarrierh, size_t thrdcnt ) -{ -#ifdef HIDDEN - barrier_t *barrierp = ( barrier_t * )qbarrierh; - - /* sanity checks - */ - ASSERT( qlock_inited ); - ASSERT( qlock_usp ); - - barrier( barrierp, thrdcnt ); -#endif /* HIDDEN */ -} - /* internal ordinal map abstraction */ #ifdef HIDDEN diff --git a/common/qlock.h b/common/qlock.h index 54469ea..1880aff 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -102,15 +102,4 @@ extern size_t qsemPblocked( qsemh_t qsemh ); /* number of threads currently blocked on this semaphore */ -typedef void *qbarrierh_t; -#define QBARRIERH_NULL 0 - /* opaque handle - */ -extern qbarrierh_t qbarrier_alloc( void ); - /* allocates a rendezvous barrier - */ -extern void qbarrier( qbarrierh_t barrierh, size_t thrdcnt ); - /* causes thrdcnt threads to rendezvous - */ - #endif /* QLOCK_H */ diff --git a/dump/content.c b/dump/content.c index 54c97d1..64bfe54 100644 --- a/dump/content.c +++ b/dump/content.c @@ -64,9 +64,6 @@ #include "getdents.h" #include "arch_xlate.h" -#undef SYNCDIR -#define SYNCDIR - /* max "unsigned long long int" */ #define ULONGLONG_MAX 18446744073709551615LLU @@ -231,7 +228,6 @@ typedef struct extent_group_context extent_group_context_t; struct pds { enum { PDS_NULL, /* per-drive activity not begun */ PDS_INOMAP, /* dumping inomap */ - PDS_DIRRENDEZVOUS, /* waiting to dump dirs */ PDS_DIRDUMP, /* dumping dirs */ PDS_NONDIR, /* dumping nondirs */ PDS_INVSYNC, /* waiting for inventory */ @@ -263,9 +259,6 @@ static rv_t dump_dirs( ix_t strmix, xfs_bstat_t *bstatbufp, size_t bstatbuflen, void *inomap_contextp ); -#ifdef SYNCDIR -static rv_t dump_dirs_rendezvous( void ); -#endif /* SYNCDIR */ static rv_t dump_dir( ix_t strmix, jdm_fshandle_t *, intgen_t, @@ -485,12 +478,6 @@ static bool_t sc_dumpextattrpr = BOOL_TRUE; static bool_t sc_dumpasoffline = BOOL_FALSE; /* dump dual-residency HSM files as offline */ -#ifdef SYNCDIR -static size_t sc_thrdsdirdumpsynccnt = 0; -static size_t sc_thrdswaitingdirdumpsync1 = 0; -static size_t sc_thrdswaitingdirdumpsync2 = 0; -static qbarrierh_t sc_barrierh; -#endif /* SYNCDIR */ static bool_t sc_savequotas = BOOL_TRUE; /* save quota information in dump @@ -1466,14 +1453,13 @@ baseuuidbypass: var_skip( &fsid, inomap_skip ); /* fill in write header template content info. always produce - * an inomap and dir dump for each media file. + * an inomap for each media file. the dirdump flag will be set + * in content_stream_dump() for streams which dump the directories. */ ASSERT( sizeof( cwhdrtemplatep->ch_specific ) >= sizeof( *scwhdrtemplatep )); scwhdrtemplatep->cih_mediafiletype = CIH_MEDIAFILETYPE_DATA; scwhdrtemplatep->cih_level = ( int32_t )sc_level; - scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP - | - CIH_DUMPATTR_DIRDUMP; + scwhdrtemplatep->cih_dumpattr = CIH_DUMPATTR_INOMAP; if ( subtreecnt ) { scwhdrtemplatep->cih_dumpattr |= CIH_DUMPATTR_SUBTREE; } @@ -1714,22 +1700,6 @@ baseuuidbypass: } } -#ifdef SYNCDIR - /* allocate a barrier to synchronize directory dumping - */ - if ( drivecnt > 1 ) { - sc_barrierh = qbarrier_alloc( ); - } - - /* initialize the number of players in the synchronized dir dump. - * they drop out when last media file complete. MUST be modified - * under lock( ). - */ - sc_thrdsdirdumpsynccnt = drivecnt; - -#endif /* SYNCDIR */ - - return BOOL_TRUE; } @@ -1877,10 +1847,6 @@ content_statline( char **linespp[ ] ) strcat( statline[ statlinecnt ], "dumping inomap" ); break; - case PDS_DIRRENDEZVOUS: - strcat( statline[ statlinecnt ], - "waiting for synchronized directory dump" ); - break; case PDS_DIRDUMP: sprintf( &statline[ statlinecnt ] [ strlen( statline[ statlinecnt ] ) ], @@ -2157,6 +2123,11 @@ content_stream_dump( ix_t strmix ) scwhdrp->cih_endpt.sp_flags = STARTPT_FLAGS_END; } + // the first stream dumps the directories + if ( strmix == 0 ) { + scwhdrp->cih_dumpattr |= CIH_DUMPATTR_DIRDUMP; + } + /* fill in inomap fields of write hdr */ inomap_writehdr( scwhdrp ); @@ -2322,39 +2293,41 @@ content_stream_dump( ix_t strmix ) return mlog_exit(EXIT_FAULT, rv); } - /* now dump the directories. use the bigstat iterator - * capability to call my dump_dir function - * for each directory in the bitmap. + /* now dump the directories, if this is a stream that dumps + * directories. use the bigstat iterator capability to call + * my dump_dir function for each directory in the bitmap. */ - sc_stat_pds[ strmix ].pds_dirdone = 0; - rv = dump_dirs( strmix, - bstatbufp, - bstatbuflen, - inomap_contextp ); - if ( rv == RV_INTR ) { - stop_requested = BOOL_TRUE; - goto decision_more; - } - if ( rv == RV_EOM ) { - hit_eom = BOOL_TRUE; - goto decision_more; - } - if ( rv == RV_DRIVE ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_NORMAL, rv); - } - if ( rv == RV_ERROR ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_ERROR, rv); - } - if ( rv == RV_CORE ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_FAULT, rv); - } - ASSERT( rv == RV_OK ); - if ( rv != RV_OK ) { - free( ( void * )bstatbufp ); - return mlog_exit(EXIT_FAULT, rv); + if ( scwhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP ) { + sc_stat_pds[ strmix ].pds_dirdone = 0; + rv = dump_dirs( strmix, + bstatbufp, + bstatbuflen, + inomap_contextp ); + if ( rv == RV_INTR ) { + stop_requested = BOOL_TRUE; + goto decision_more; + } + if ( rv == RV_EOM ) { + hit_eom = BOOL_TRUE; + goto decision_more; + } + if ( rv == RV_DRIVE ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_NORMAL, rv); + } + if ( rv == RV_ERROR ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_ERROR, rv); + } + if ( rv == RV_CORE ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_FAULT, rv); + } + ASSERT( rv == RV_OK ); + if ( rv != RV_OK ) { + free( ( void * )bstatbufp ); + return mlog_exit(EXIT_FAULT, rv); + } } /* finally, dump the non-directory files beginning with this @@ -2527,20 +2500,6 @@ decision_more: */ done = all_nondirs_committed; -#ifdef SYNCDIR - /* drop out of the synchronous dump game if done - */ - if ( done ) { - /* REFERENCED */ - size_t tmpthrdsdirdumpsynccnt; - lock( ); - tmpthrdsdirdumpsynccnt = sc_thrdsdirdumpsynccnt; - sc_thrdsdirdumpsynccnt--; - unlock( ); - ASSERT( tmpthrdsdirdumpsynccnt > 0 ); - } -#endif /* SYNCDIR */ - /* tell the inventory about the media file */ if ( inv_stmt != INV_TOKEN_NULL ) { @@ -2833,22 +2792,6 @@ dump_dirs( ix_t strmix, __s32 buflenout; intgen_t rval; -#ifdef SYNCDIR - /* have all threads rendezvous - */ - if ( sc_thrdsdirdumpsynccnt > 1 && stream_cnt( ) > 1 ) { - rv_t rv; - mlog( bulkstatcallcnt == 0 ? MLOG_VERBOSE : MLOG_NITTY, - _("waiting for synchronized directory dump\n") ); - sc_stat_pds[ strmix ].pds_phase = PDS_DIRRENDEZVOUS; - rv = dump_dirs_rendezvous( ); - if ( rv == RV_INTR ) { - return RV_INTR; - } - ASSERT( rv == RV_OK ); - } -#endif /* SYNCDIR */ - if ( bulkstatcallcnt == 0 ) { mlog( MLOG_VERBOSE, _( "dumping directories\n") ); @@ -2948,58 +2891,6 @@ dump_dirs( ix_t strmix, /* NOTREACHED */ } -#ifdef SYNCDIR -static rv_t -dump_dirs_rendezvous( void ) -{ - static size_t localsync1; - static size_t localsync2; - - sc_thrdswaitingdirdumpsync2 = 0; - lock( ); - sc_thrdswaitingdirdumpsync1++; - localsync1 = sc_thrdswaitingdirdumpsync1; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - while ( localsync2 == 0 - && - localsync1 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) { - sleep( 1 ); - if ( cldmgr_stop_requested( )) { - lock( ); - sc_thrdswaitingdirdumpsync1--; - unlock( ); - return RV_INTR; - } - lock( ); - localsync1 = sc_thrdswaitingdirdumpsync1; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - } - lock( ); - sc_thrdswaitingdirdumpsync1--; - sc_thrdswaitingdirdumpsync2++; - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - while ( localsync2 < min( stream_cnt( ), sc_thrdsdirdumpsynccnt )) { - sleep( 1 ); - if ( cldmgr_stop_requested( )) { - return RV_INTR; - } - lock( ); - localsync2 = sc_thrdswaitingdirdumpsync2; - unlock( ); - } - if ( cldmgr_stop_requested( )) { - return RV_INTR; - } - - qbarrier( sc_barrierh, min( stream_cnt( ), sc_thrdsdirdumpsynccnt )); - - return RV_OK; -} -#endif /* SYNCDIR */ - static rv_t dump_dir( ix_t strmix, jdm_fshandle_t *fshandlep, diff --git a/restore/content.c b/restore/content.c index 34fc4a0..2228a7f 100644 --- a/restore/content.c +++ b/restore/content.c @@ -625,6 +625,9 @@ struct tran { intgen_t t_persfd; /* file descriptor of the persistent state file */ + size64_t t_dirdumps; + /* bitset of streams which contain a directory dump + */ sync_t t_sync1; /* to single-thread attempt to validate command line * selection of dump with online inventory @@ -1184,6 +1187,12 @@ content_init( intgen_t argc, char *argv[ ], size64_t vmsz ) return BOOL_FALSE; } + /* assume all streams contain a directory dump. streams will remove + * themselves from this bitset if they do not contain a directory dump. + */ + ASSERT( drivecnt <= sizeof(tranp->t_dirdumps) * NBBY ); + tranp->t_dirdumps = ( 1ULL << drivecnt ) - 1; + /* the user may specify stdin as the restore source stream, * by a single dash ('-') with no option letter. This must * appear between the last lettered argument and the destination @@ -2237,6 +2246,30 @@ content_stream_restore( ix_t thrdix ) unlock( ); continue; } + if ( !(scrhdrp->cih_dumpattr & CIH_DUMPATTR_DIRDUMP) ) { + /* if no streams have a directory dump, issue a + * message and exit. first set SYNC_BUSY to prevent + * other threads from coming through here and issuing + * the same message. + */ + tranp->t_dirdumps &= ~(1ULL << thrdix); + if ( !tranp->t_dirdumps ) { + tranp->t_sync3 = SYNC_BUSY; + } + unlock( ); + if ( !tranp->t_dirdumps ) { + mlog( MLOG_VERBOSE | MLOG_ERROR, _( + "no directory dump found\n") ); + Media_end( Mediap ); + return mlog_exit(EXIT_NORMAL, RV_ERROR); + } + sleep( 1 ); + if ( cldmgr_stop_requested( )) { + Media_end( Mediap ); + return mlog_exit(EXIT_NORMAL, RV_INTR); + } + continue; + } tranp->t_sync3 = SYNC_BUSY; unlock( ); if ( ! tranp->t_dirattrinitdonepr ) { -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:45 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64, LOCAL_GNU_PATCH autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KxiO1076063 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1B731AC009; Mon, 7 Nov 2011 12:59:41 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 8755E70016E5; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 53AA620E36FD; Mon, 7 Nov 2011 14:59:40 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 5/8] xfsdump: convert IRIX sproc threads to pthreads Date: Mon, 7 Nov 2011 14:58:28 -0600 Message-Id: <1320699511-12281-6-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The existing (disabled) threading code in xfsdump is based on IRIX sprocs. This patch converts the code to use pthreads. The threading code remains disabled at this point. Changes: - pid_t to pthread_t - getpid() to pthread_self() - "pid1 == pid2" to pthread_equal(tid1, tid2) - sigprocmask() to pthread_sigmask() - sproc() to pthread_create() Also the following are not referenced and have been removed: - cldmgr_pid2streamix() - PROCMAX - r_slavepid - sproc.c and sproc.h Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- common/Makefile | 2 +- common/cldmgr.c | 56 ++++++++++++++-------------- common/cldmgr.h | 8 +--- common/dlog.c | 4 +- common/drive_minrmt.c | 2 +- common/drive_scsitape.c | 2 +- common/main.c | 20 +++++----- common/mlog.c | 44 +++++++++++----------- common/ring.c | 9 +--- common/ring.h | 1 - common/sproc.c | 42 --------------------- common/sproc.h | 23 ----------- common/stream.c | 94 +++++++++++++++++++++-------------------------- common/stream.h | 22 +++++----- dump/Makefile | 4 +- dump/content.c | 4 +- restore/Makefile | 2 - restore/content.c | 5 +- 18 files changed, 127 insertions(+), 217 deletions(-) delete mode 100644 common/sproc.c delete mode 100644 common/sproc.h diff --git a/common/Makefile b/common/Makefile index 8d9d868..ad3d61a 100644 --- a/common/Makefile +++ b/common/Makefile @@ -13,7 +13,7 @@ LSRCFILES = arch_xlate.c arch_xlate.h \ hsmapi.c hsmapi.h inventory.c inventory.h lock.c lock.h \ main.c media.c media.h media_rmvtape.h mlog.c mlog.h \ openutil.c openutil.h path.c path.h qlock.c qlock.h \ - rec_hdr.h ring.c ring.h sproc.c sproc.h stream.c \ + rec_hdr.h ring.c ring.h stream.c \ stream.h timeutil.c timeutil.h ts_mtio.h types.h util.c util.h default install install-dev : diff --git a/common/cldmgr.c b/common/cldmgr.c index d327bab..4574834 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -24,6 +24,7 @@ #include #include #include +#include #include "types.h" #include "lock.h" @@ -31,14 +32,13 @@ #include "stream.h" #include "mlog.h" #include "cldmgr.h" -#include "sproc.h" extern size_t pgsz; #define CLD_MAX ( STREAM_SIMMAX * 2 ) struct cld { bool_t c_busy; - pid_t c_pid; + pthread_t c_tid; ix_t c_streamix; int ( * c_entry )( void *arg1 ); void * c_arg1; @@ -50,32 +50,31 @@ static cld_t cld[ CLD_MAX ]; static bool_t cldmgr_stopflag; static cld_t *cldmgr_getcld( void ); -static cld_t * cldmgr_findbypid( pid_t ); -static int cldmgr_entry( void * ); +static cld_t * cldmgr_findbytid( pthread_t ); +static void *cldmgr_entry( void * ); /* REFERENCED */ -static pid_t cldmgr_parentpid; +static pthread_t cldmgr_parenttid; bool_t cldmgr_init( void ) { ( void )memset( ( void * )cld, 0, sizeof( cld )); cldmgr_stopflag = BOOL_FALSE; - cldmgr_parentpid = getpid( ); + cldmgr_parenttid = pthread_self( ); return BOOL_TRUE; } bool_t cldmgr_create( int ( * entry )( void *arg1 ), - u_intgen_t inh, ix_t streamix, char *descstr, void *arg1 ) { cld_t *cldp; - pid_t cldpid; + intgen_t rval; - ASSERT( getpid( ) == cldmgr_parentpid ); + ASSERT( pthread_equal( pthread_self( ), cldmgr_parenttid ) ); cldp = cldmgr_getcld( ); if ( ! cldp ) { @@ -91,22 +90,22 @@ cldmgr_create( int ( * entry )( void *arg1 ), cldp->c_streamix = streamix; cldp->c_entry = entry; cldp->c_arg1 = arg1; - cldpid = ( pid_t )sproc( cldmgr_entry, inh, ( void * )cldp ); - if ( cldpid < 0 ) { + rval = pthread_create( &cldp->c_tid, NULL, cldmgr_entry, cldp ); + if ( rval ) { mlog( MLOG_NORMAL | MLOG_ERROR | MLOG_PROC, _( - "sproc failed creating %s thread for stream %u: %s\n"), + "failed creating %s thread for stream %u: %s\n"), descstr, streamix, - strerror( errno )); + strerror( rval )); } else { mlog( MLOG_NITTY | MLOG_PROC, - "%s thread created for stream %u: pid %d\n", + "%s thread created for stream %u: tid %lu\n", descstr, streamix, - cldpid ); + cldp->c_tid ); } - return cldpid < 0 ? BOOL_FALSE : BOOL_TRUE; + return rval ? BOOL_FALSE : BOOL_TRUE; } void @@ -119,16 +118,16 @@ cldmgr_stop( void ) } void -cldmgr_died( pid_t pid ) +cldmgr_died( pthread_t tid ) { - cld_t *cldp = cldmgr_findbypid( pid ); + cld_t *cldp = cldmgr_findbytid( tid ); if ( ! cldp ) { return; } cldp->c_busy = BOOL_FALSE; if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_dead( pid ); + stream_dead( tid ); } } @@ -194,13 +193,13 @@ cldmgr_getcld( void ) } static cld_t * -cldmgr_findbypid( pid_t pid ) +cldmgr_findbytid( pthread_t tid ) { cld_t *p = cld; cld_t *ep = cld + sizeof( cld ) / sizeof( cld[ 0 ] ); for ( ; p < ep ; p++ ) { - if ( p->c_busy && p->c_pid == pid ) { + if ( p->c_busy && pthread_equal( p->c_tid, tid )) { break; } } @@ -208,19 +207,20 @@ cldmgr_findbypid( pid_t pid ) return ( p < ep ) ? p : 0; } -static int +static void * cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; - pid_t pid = getpid( ); + pthread_t tid = pthread_self( ); - cldp->c_pid = pid; if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { - stream_register( pid, ( intgen_t )cldp->c_streamix ); + stream_register( tid, ( intgen_t )cldp->c_streamix ); } mlog( MLOG_DEBUG | MLOG_PROC, - "child %d created for stream %d\n", - pid, + "thread %lu created for stream %d\n", + tid, cldp->c_streamix ); - return ( * cldp->c_entry )( cldp->c_arg1 ); + + ( * cldp->c_entry )( cldp->c_arg1 ); + return NULL; } diff --git a/common/cldmgr.h b/common/cldmgr.h index bb3f612..e393b82 100644 --- a/common/cldmgr.h +++ b/common/cldmgr.h @@ -30,7 +30,6 @@ extern bool_t cldmgr_init( void ); * encountered */ extern bool_t cldmgr_create( int ( * entry )( void *arg1 ), - u_intgen_t inh, ix_t streamix, char *descstr, void *arg1 ); @@ -42,18 +41,13 @@ extern void cldmgr_stop( void ); /* cldmgr_died - tells the child manager that the child died */ -extern void cldmgr_died( pid_t pid ); +extern void cldmgr_died( pthread_t tid ); /* cldmgr_stop_requested - returns TRUE if the child should gracefully * terminate. */ extern bool_t cldmgr_stop_requested( void ); -/* cldmgr_pid2streamix - retrieves the stream index. returns -1 if - * not associated with any stream. - */ -extern intgen_t cldmgr_pid2streamix( pid_t pid ); - /* cldmgr_remainingcnt - returns number of children remaining */ extern size_t cldmgr_remainingcnt( void ); diff --git a/common/dlog.c b/common/dlog.c index 8cf9a65..51666cf 100644 --- a/common/dlog.c +++ b/common/dlog.c @@ -429,7 +429,7 @@ promptinput( char *buf, sigaddset( &dlog_registered_sigs, SIGQUIT ); } - sigprocmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set ); + pthread_sigmask( SIG_UNBLOCK, &dlog_registered_sigs, &orig_set ); /* wait for input, timeout, or interrupt. * note we come out of the select() frequently in order to @@ -455,7 +455,7 @@ promptinput( char *buf, /* restore signal handling */ - sigprocmask( SIG_SETMASK, &orig_set, NULL ); + pthread_sigmask( SIG_SETMASK, &orig_set, NULL ); sigemptyset( &dlog_registered_sigs ); /* check for timeout or interrupt diff --git a/common/drive_minrmt.c b/common/drive_minrmt.c index 836b663..3ff4d0f 100644 --- a/common/drive_minrmt.c +++ b/common/drive_minrmt.c @@ -577,7 +577,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) drivep->d_cap_est = -1; drivep->d_rate_est = -1; - /* if sproc not allowed, allocate a record buffer. otherwise + /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ if ( singlethreaded ) { diff --git a/common/drive_scsitape.c b/common/drive_scsitape.c index 8c1bd49..f24d604 100644 --- a/common/drive_scsitape.c +++ b/common/drive_scsitape.c @@ -667,7 +667,7 @@ ds_instantiate( int argc, char *argv[], drive_t *drivep, bool_t singlethreaded ) drivep->d_cap_est = -1; drivep->d_rate_est = -1; - /* if sproc not allowed, allocate a record buffer. otherwise + /* if threads not allowed, allocate a record buffer. otherwise * create a ring, from which buffers will be taken. */ if ( singlethreaded ) { diff --git a/common/main.c b/common/main.c index 25c0838..d4dbe28 100644 --- a/common/main.c +++ b/common/main.c @@ -32,6 +32,7 @@ #include #include #include +#include #include "exit.h" #include "types.h" @@ -117,7 +118,7 @@ bool_t miniroot = BOOL_TRUE; #endif /* HIDDEN */ bool_t pipeline = BOOL_FALSE; bool_t stdoutpiped = BOOL_FALSE; -pid_t parentpid; +pthread_t parenttid; char *sistr; size_t pgsz; size_t pgmask; @@ -195,10 +196,10 @@ main( int argc, char *argv[] ) */ mlog_init0(); - /* Get the parent's pid. will be used in signal handling + /* Get the parent's pthread id. will be used * to differentiate parent from children. */ - parentpid = getpid( ); + parenttid = pthread_self( ); rval = atexit(mlog_exit_flush); assert(rval == 0); @@ -395,11 +396,11 @@ main( int argc, char *argv[] ) ASSERT( ( intgen_t )pgsz > 0 ); pgmask = pgsz - 1; - /* report parent pid + /* report parent tid */ mlog( MLOG_DEBUG | MLOG_PROC, - "parent pid is %d\n", - parentpid ); + "parent tid is %lu\n", + parenttid ); /* get the current working directory: this is where we will dump * core, if necessary. some tmp files may be placed here as well. @@ -572,7 +573,7 @@ main( int argc, char *argv[] ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); - sigprocmask( SIG_SETMASK, &blocked_set, NULL ); + pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); sa.sa_handler = sighandler; sigaction( SIGINT, &sa, NULL ); @@ -676,7 +677,6 @@ main( int argc, char *argv[] ) if ( ! init_error ) { for ( stix = 0 ; stix < drivecnt ; stix++ ) { ok = cldmgr_create( childmain, - CLONE_VM, stix, "child", ( void * )stix ); @@ -895,7 +895,7 @@ main( int argc, char *argv[] ) if ( coredump_requested ) { mlog( MLOG_DEBUG | MLOG_PROC, "core dump requested, aborting (pid %d)\n", - parentpid ); + getpid() ); abort(); } @@ -1560,7 +1560,7 @@ childmain( void *arg1 ) drivep = drivepp[ stix ]; ( * drivep->d_opsp->do_quit )( drivep ); - exit( exitcode ); + return exitcode; } diff --git a/common/mlog.c b/common/mlog.c index 2265895..b0135b9 100644 --- a/common/mlog.c +++ b/common/mlog.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "types.h" #include "qlock.h" @@ -40,7 +41,7 @@ extern char *progname; extern void usage( void ); -extern pid_t parentpid; +extern pthread_t parenttid; #ifdef DUMP static FILE *mlog_fp = NULL; /* stderr */; @@ -385,7 +386,7 @@ mlog_va( intgen_t levelarg, char *fmt, va_list args ) if ( ! ( levelarg & MLOG_BARE )) { intgen_t streamix; - streamix = stream_getix( getpid() ); + streamix = stream_getix( pthread_self( ) ); if ( mlog_showss ) { sprintf( mlog_ssstr, ":%s", mlog_ss_names[ ss ] ); @@ -568,10 +569,10 @@ rv_getdesc(rv_t rv) int _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) { - pid_t pid; + pthread_t tid; const struct rv_map *rvp; - pid = getpid(); + tid = pthread_self(); rvp = rv_getdesc(rv); @@ -595,7 +596,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) * most accurate information about the termination condition. */ - if (pid == parentpid) { + if ( pthread_equal( tid, parenttid ) ) { if (mlog_main_exit_code == -1) { mlog_main_exit_code = exit_code; mlog_main_exit_return = rv; @@ -608,7 +609,7 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) int exit_code; rv_t exit_return, exit_hint; - if (stream_get_exit_status(pid, + if (stream_get_exit_status(tid, states, N(states), &state, @@ -618,8 +619,8 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) &exit_hint)) { if (exit_code == -1) { - stream_set_code(pid, exit_code); - stream_set_return(pid, rv); + stream_set_code(tid, exit_code); + stream_set_return(tid, rv); } } } @@ -630,10 +631,10 @@ _mlog_exit( const char *file, int line, int exit_code, rv_t rv ) void _mlog_exit_hint( const char *file, int line, rv_t rv ) { - pid_t pid; + pthread_t tid; const struct rv_map *rvp; - pid = getpid(); + tid = pthread_self(); rvp = rv_getdesc(rv); mlog( MLOG_DEBUG | MLOG_NOLOCK, @@ -655,10 +656,10 @@ _mlog_exit_hint( const char *file, int line, rv_t rv ) * information about the termination condition. */ - if (pid == parentpid) + if ( pthread_equal( tid, parenttid ) ) mlog_main_exit_hint = rv; else - stream_set_hint( pid, rv ); + stream_set_hint( tid, rv ); } @@ -670,10 +671,10 @@ mlog_get_hint( void ) bool_t ok; rv_t hint; - if (getpid() == parentpid) + if ( pthread_equal( pthread_self(), parenttid ) ) return mlog_main_exit_hint; - ok = stream_get_exit_status(getpid(), states, N(states), + ok = stream_get_exit_status(pthread_self(), states, N(states), NULL, NULL, NULL, NULL, &hint); ASSERT(ok); return hint; @@ -697,8 +698,8 @@ mlog_get_hint( void ) void mlog_exit_flush(void) { - pid_t pids[STREAM_SIMMAX]; - int i, npids; + pthread_t tids[STREAM_SIMMAX]; + int i, ntids; const struct rv_map *rvp; stream_state_t states[] = { S_RUNNING, S_ZOMBIE }; bool_t incomplete = BOOL_FALSE; @@ -713,13 +714,13 @@ mlog_exit_flush(void) if (mlog_main_exit_hint == RV_USAGE) return; - npids = stream_find_all(states, N(states), pids, STREAM_SIMMAX); - if (npids > 0) { + ntids = stream_find_all(states, N(states), tids, STREAM_SIMMAX); + if (ntids > 0) { /* print the state of all the streams */ fprintf(mlog_fp, _("%s: %s Summary:\n"), progname, PROGSTR_CAPS ); - for (i = 0; i < npids; i++) { + for (i = 0; i < ntids; i++) { stream_state_t state; intgen_t streamix; int exit_code; @@ -727,7 +728,7 @@ mlog_exit_flush(void) /* REFERENCED */ bool_t ok; - ok = stream_get_exit_status(pids[i], + ok = stream_get_exit_status(tids[i], states, N(states), &state, @@ -743,11 +744,10 @@ mlog_exit_flush(void) /* print status of this stream */ rvp = rv_getdesc(rv); fprintf(mlog_fp, - _("%s: stream %d (pid %d) %s " + _("%s: stream %d %s " "%s (%s)\n"), progname, streamix, - pids[i], drivepp[streamix]->d_pathname, rvp->rv_string, rvp->rv_desc); diff --git a/common/ring.c b/common/ring.c index b6074d1..0d2feb0 100644 --- a/common/ring.c +++ b/common/ring.c @@ -123,7 +123,6 @@ ring_create( size_t ringlen, /* kick off the slave thread */ ok = cldmgr_create( ring_slave_entry, - CLONE_VM, drive_index, _("slave"), ringp ); @@ -417,11 +416,7 @@ ring_slave_entry( void *ringctxp ) sigaddset( &blocked_set, SIGTERM ); sigaddset( &blocked_set, SIGQUIT ); sigaddset( &blocked_set, SIGALRM ); - sigprocmask( SIG_SETMASK, &blocked_set, NULL ); - - /* record slave pid to be used to kill slave - */ - ringp->r_slavepid = getpid( ); + pthread_sigmask( SIG_SETMASK, &blocked_set, NULL ); /* loop reading and precessing messages until told to die */ @@ -494,5 +489,5 @@ ring_slave_entry( void *ringctxp ) ring_slave_put( ringp, msgp ); } - exit( 0 ); + return 0; } diff --git a/common/ring.h b/common/ring.h index 56e3924..caa505c 100644 --- a/common/ring.h +++ b/common/ring.h @@ -134,7 +134,6 @@ struct ring { time32_t r_first_io_time; off64_t r_all_io_cnt; /* ALL BELOW PRIVATE!!! */ - pid_t r_slavepid; size_t r_len; ring_msg_t *r_msgp; size_t r_ready_in_ix; diff --git a/common/sproc.c b/common/sproc.c deleted file mode 100644 index 3b1425a..0000000 --- a/common/sproc.c +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright (c) 2000-2001 Silicon Graphics, Inc. - * All Rights Reserved. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it would be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - */ - -#include -#include -#include - -#define STACKSIZE 65536 - -int -sproc (int (*entry) (void *), int flags, void *arg) -{ - int retval = -1; -#ifdef HIDDEN - void *newstack; - - if ( (newstack = calloc (1, STACKSIZE)) != NULL ) { - void ** stackp = ((void **)newstack)+(STACKSIZE -1)/sizeof(void*); - - flags |= SIGCHLD; - - retval = clone (entry, stackp, flags, arg); - } -#endif - - return retval; -} diff --git a/common/sproc.h b/common/sproc.h deleted file mode 100644 index 46111ff..0000000 --- a/common/sproc.h +++ /dev/null @@ -1,23 +0,0 @@ -/* - * Copyright (c) 2000-2001 Silicon Graphics, Inc. - * All Rights Reserved. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it would be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, write the Free Software Foundation, - * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA - */ -#ifndef SPROC_H -#define SPROC_H - -int sproc (int (*) (void *), int, void *); - -#endif /* SPROC_H */ diff --git a/common/stream.c b/common/stream.c index adaf7c0..48e25ee 100644 --- a/common/stream.c +++ b/common/stream.c @@ -19,18 +19,19 @@ #include #include +#include + #include "types.h" #include "exit.h" #include "stream.h" #include "lock.h" #include "mlog.h" -#define PROCMAX ( STREAM_SIMMAX * 2 + 1 ) #define N(a) (sizeof((a)) / sizeof((a)[0])) struct spm { stream_state_t s_state; - pid_t s_pid; + pthread_t s_tid; intgen_t s_ix; int s_exit_code; rv_t s_exit_return; @@ -38,37 +39,28 @@ struct spm { }; typedef struct spm spm_t; -extern pid_t parentpid; static spm_t spm[ STREAM_SIMMAX * 3 ]; static bool_t initialized = BOOL_FALSE; void stream_init( void ) { -#ifdef HIDDEN - /* REFERENCED */ - intgen_t rval; - - rval = ( intgen_t )usconfig( CONF_INITUSERS, PROCMAX ); - ASSERT( rval >= 0 ); -#endif /* HIDDEN */ - ( void )memset( ( void * )spm, 0, sizeof( spm )); initialized = BOOL_TRUE; } /* * Note that the stream list structure (updated via the stream_* functions) - * is indexed by pid. Multiple processes can be registered against the same - * stream index, typically: the primary content process that does the work; - * and the drive slave process, which just processes stuff off the ring buffer. - * In general having multiple pids registered per stream is not an issue for - * termination status reporting, as the mlog_exit* logging functions only + * is indexed by pthread_t (tid). Multiple processes can be registered against + * the same stream index, typically: the primary content process that does the + * work; and the drive slave process, which just processes stuff off the ring + * buffer. In general having multiple tids registered per stream is not an issue + * for termination status reporting, as the mlog_exit* logging functions only * ever get called out of the primary content process. */ void -stream_register( pid_t pid, intgen_t streamix ) +stream_register( pthread_t tid, intgen_t streamix ) { spm_t *p = spm; spm_t *ep = spm + N(spm); @@ -87,7 +79,7 @@ stream_register( pid_t pid, intgen_t streamix ) if ( p >= ep ) return; - p->s_pid = pid; + p->s_tid = tid; p->s_ix = streamix; p->s_exit_code = -1; p->s_exit_return = RV_NONE; @@ -95,14 +87,14 @@ stream_register( pid_t pid, intgen_t streamix ) } void -stream_dead( pid_t pid ) +stream_dead( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); lock(); for ( ; p < ep ; p++ ) - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { p->s_state = S_ZOMBIE; break; } @@ -111,14 +103,14 @@ stream_dead( pid_t pid ) } void -stream_free( pid_t pid ) +stream_free( pthread_t tid ) { spm_t *p = spm; spm_t *ep = spm + N(spm); lock(); for ( ; p < ep ; p++ ) { - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { (void) memset( (void *) p, 0, sizeof(spm_t) ); p->s_state = S_FREE; break; @@ -130,22 +122,22 @@ stream_free( pid_t pid ) int stream_find_all( stream_state_t states[], int nstates, - pid_t pids[], int npids ) + pthread_t tids[], int ntids ) { int i, count = 0; spm_t *p = spm; spm_t *ep = spm + N(spm); - ASSERT(nstates > 0 && npids > 0); + ASSERT(nstates > 0 && ntids > 0); if (!initialized) return 0; /* lock - make sure we get a consistent snapshot of the stream status */ lock(); - for ( ; p < ep && count < npids; p++ ) + for ( ; p < ep && count < ntids; p++ ) for (i = 0; i < nstates; i++) if (p->s_state == states[i]) { - pids[count++] = p->s_pid; + tids[count++] = p->s_tid; break; } unlock(); @@ -153,7 +145,7 @@ stream_find_all( stream_state_t states[], int nstates, } static spm_t * -stream_find( pid_t pid, stream_state_t s[], int nstates ) +stream_find( pthread_t tid, stream_state_t s[], int nstates ) { int i; spm_t *p = spm; @@ -163,7 +155,7 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) /* note we don't lock the stream array in this function */ for ( ; p < ep ; p++ ) - if ( p->s_pid == pid ) { + if ( pthread_equal( p->s_tid, tid ) ) { /* check state */ for (i = 0; i < nstates; i++) if (p->s_state == s[i]) @@ -174,8 +166,8 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) { static const char *state_strings[] = { "S_FREE", "S_RUNNING", "S_ZOMBIE" }; mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE, - "stream_find(): no stream with pid: %d and state%s:", - pid, nstates == 1 ? "" : "s" ); + "stream_find(): no stream with tid: %lu and state%s:", + tid, nstates == 1 ? "" : "s" ); for (i = 0; i < nstates; i++) mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK | MLOG_BARE, " %s", state_strings[s[i]]); @@ -187,20 +179,18 @@ stream_find( pid_t pid, stream_state_t s[], int nstates ) } /* - * Note, the following function is called from two places: - * main.c:sighandler(), and mlog.c:mlog_va() in the first case we - * aren't allowed to take locks, and in the second locking may be - * disabled and we are already protected by another lock. So no - * locking is done in this function. + * Note, the following function is called from mlog.c:mlog_va(), + * where locking may be disabled and we are already protected by + * another lock. So no locking is done in this function. */ intgen_t -stream_getix( pid_t pid ) +stream_getix( pthread_t tid ) { stream_state_t states[] = { S_RUNNING }; spm_t *p; intgen_t ix; - p = stream_find( pid, states, N(states) ); + p = stream_find( tid, states, N(states) ); ix = p ? p->s_ix : -1; return ix; } @@ -213,43 +203,43 @@ stream_getix( pid_t pid ) * streams. */ -#define stream_set(field_name, pid, value) \ +#define stream_set(field_name, tid, value) \ stream_state_t states[] = { S_RUNNING }; \ spm_t *p; \ - pid_t mypid = getpid(); \ + pthread_t mytid = pthread_self(); \ \ - if (mypid != (pid)) { \ + if ( !pthread_equal(mytid, (tid))) { \ mlog( MLOG_DEBUG | MLOG_ERROR | MLOG_NOLOCK, \ "stream_set_" #field_name "(): " \ - "foreign stream (pid %d) " \ - "not permitted to update this stream (pid %d)\n", \ - mypid, (pid)); \ + "foreign stream (tid %lu) " \ + "not permitted to update this stream (tid %lu)\n",\ + mytid, (tid)); \ return; \ } \ \ lock(); \ - p = stream_find( (pid), states, N(states) ); \ + p = stream_find( (tid), states, N(states) ); \ if (p) p->s_exit_ ## field_name = (value); \ unlock(); -void stream_set_code( pid_t pid, int exit_code ) +void stream_set_code( pthread_t tid, int exit_code ) { - stream_set( code, pid, exit_code ); + stream_set( code, tid, exit_code ); } -void stream_set_return( pid_t pid, rv_t rv ) +void stream_set_return( pthread_t tid, rv_t rv ) { - stream_set( return, pid, rv ); + stream_set( return, tid, rv ); } -void stream_set_hint( pid_t pid, rv_t rv ) +void stream_set_hint( pthread_t tid, rv_t rv ) { - stream_set( hint, pid, rv ); + stream_set( hint, tid, rv ); } bool_t -stream_get_exit_status( pid_t pid, +stream_get_exit_status( pthread_t tid, stream_state_t states[], int nstates, stream_state_t *state, @@ -262,7 +252,7 @@ stream_get_exit_status( pid_t pid, spm_t *p; lock(); - p = stream_find( pid, states, nstates ); + p = stream_find( tid, states, nstates ); if (! p) goto unlock; if (state) *state = p->s_state; diff --git a/common/stream.h b/common/stream.h index 984634c..292792e 100644 --- a/common/stream.h +++ b/common/stream.h @@ -42,19 +42,19 @@ typedef enum { S_FREE, S_RUNNING, S_ZOMBIE } stream_state_t; extern void stream_init( void ); -extern void stream_register( pid_t pid, intgen_t streamix ); -extern void stream_dead( pid_t pid ); -extern void stream_free( pid_t pid ); +extern void stream_register( pthread_t tid, intgen_t streamix ); +extern void stream_dead( pthread_t tid ); +extern void stream_free( pthread_t tid ); extern int stream_find_all( stream_state_t states[], int nstates, - pid_t pids[], - int npids ); -extern intgen_t stream_getix( pid_t pid ); -extern void stream_set_code( pid_t pid, int code ); -extern void stream_set_return( pid_t pid, rv_t rv ); -extern void stream_set_hint( pid_t pid, rv_t rv ); -extern bool_t stream_exists( pid_t pid ); -extern bool_t stream_get_exit_status( pid_t pid, + pthread_t tids[], + int ntids ); +extern intgen_t stream_getix( pthread_t tid ); +extern void stream_set_code( pthread_t tid, int code ); +extern void stream_set_return( pthread_t tid, rv_t rv ); +extern void stream_set_hint( pthread_t tid, rv_t rv ); +extern bool_t stream_exists( pthread_t tid ); +extern bool_t stream_get_exit_status( pthread_t tid, stream_state_t states[], int nstates, stream_state_t *state, diff --git a/dump/Makefile b/dump/Makefile index d193f68..97879fa 100644 --- a/dump/Makefile +++ b/dump/Makefile @@ -30,7 +30,6 @@ COMMINCL = \ ts_mtio.h \ types.h \ util.h \ - sproc.h \ rec_hdr.h INVINCL = \ @@ -68,8 +67,7 @@ COMMON = \ ring.c \ stream.c \ timeutil.c \ - util.c \ - sproc.c + util.c LOCALS = \ content.c \ diff --git a/dump/content.c b/dump/content.c index 64bfe54..33f1166 100644 --- a/dump/content.c +++ b/dump/content.c @@ -1666,12 +1666,12 @@ baseuuidbypass: sigaddset( &tty_set, SIGINT ); sigaddset( &tty_set, SIGQUIT ); sigaddset( &tty_set, SIGHUP ); - sigprocmask( SIG_BLOCK, &tty_set, &orig_set ); + pthread_sigmask( SIG_BLOCK, &tty_set, &orig_set ); result = create_inv_session( gwhdrtemplatep, &fsid, mntpnt, fsdevice, subtreecnt, strmix ); - sigprocmask( SIG_SETMASK, &orig_set, NULL ); + pthread_sigmask( SIG_SETMASK, &orig_set, NULL ); if ( !result ) { return BOOL_FALSE; diff --git a/restore/Makefile b/restore/Makefile index ce3d6b4..c6f3f25 100644 --- a/restore/Makefile +++ b/restore/Makefile @@ -25,7 +25,6 @@ COMMINCL = \ qlock.h \ rec_hdr.h \ ring.h \ - sproc.h \ stream.h \ timeutil.h \ ts_mtio.h \ @@ -64,7 +63,6 @@ COMMON = \ path.c \ qlock.c \ ring.c \ - sproc.c \ stream.c \ timeutil.c \ util.c diff --git a/restore/content.c b/restore/content.c index 8dfa456..0108a40 100644 --- a/restore/content.c +++ b/restore/content.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "types.h" #include "timeutil.h" @@ -2227,7 +2228,7 @@ content_stream_restore( ix_t thrdix ) #if DEBUG_DUMPSTREAMS { static int count[STREAM_MAX] = {0}; - intgen_t streamix = stream_getix( getpid() ); + intgen_t streamix = stream_getix( pthread_self() ); if (++(count[streamix]) == 30) { mlog( MLOG_TRACE, "still waiting for dirs to be restored\n"); @@ -2390,7 +2391,7 @@ content_stream_restore( ix_t thrdix ) #if DEBUG_DUMPSTREAMS { static int count[STREAM_MAX] = {0}; - intgen_t streamix = stream_getix( getpid() ); + intgen_t streamix = stream_getix( pthread_self() ); if (++(count[streamix]) == 30) { mlog( MLOG_NORMAL, "still waiting for dirs post-processing\n"); -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 14:59:44 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7KxidJ076053 for ; Mon, 7 Nov 2011 14:59:44 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay1.corp.sgi.com (Postfix) with ESMTP id D903F8F8033; Mon, 7 Nov 2011 12:59:40 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 81A7570016E4; Mon, 7 Nov 2011 14:59:40 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 4B5F820E36FB; Mon, 7 Nov 2011 14:59:40 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH v2 4/8] xfsdump: simplify qlock ordinal bitmap Date: Mon, 7 Nov 2011 14:58:27 -0600 Message-Id: <1320699511-12281-5-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320699511-12281-1-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The qlock abstraction manages an array of ordinal bitmaps, one bitmap for each thread. The bitmap indicates which locks a thread is holding and is used to verify that the locks are obtained in the correct order. There is no need to use an array to store the bitmaps, and in fact adding entries to the array is broken because qlock_thrdcnt is not protected by a lock. A simpler approach is to create a per-thread bitmap using thread local storage. With this change, there is no need for each new thread to register with the qlock abstraction, so qlock_thrdinit() goes away. Remove qlock_init() since it can be statically initialized, and that means there's no need for a flag (qlock_inited) indicating that the module has been initialized. Also there's no longer a need to track or limit the number of threads that the qlock abstraction can handle. Signed-off-by: Bill Kendall Reviewed-by: Christoph Hellwig --- common/cldmgr.c | 4 - common/main.c | 7 -- common/qlock.c | 191 ++++--------------------------------------------------- common/qlock.h | 11 +--- 4 files changed, 14 insertions(+), 199 deletions(-) diff --git a/common/cldmgr.c b/common/cldmgr.c index 7784a15..d327bab 100644 --- a/common/cldmgr.c +++ b/common/cldmgr.c @@ -213,12 +213,8 @@ cldmgr_entry( void *arg1 ) { cld_t *cldp = ( cld_t * )arg1; pid_t pid = getpid( ); - /* REFERENCED */ - bool_t ok; cldp->c_pid = pid; - ok = qlock_thrdinit( ); - ASSERT( ok ); if ( ( intgen_t )( cldp->c_streamix ) >= 0 ) { stream_register( pid, ( intgen_t )cldp->c_streamix ); } diff --git a/common/main.c b/common/main.c index be8a921..25c0838 100644 --- a/common/main.c +++ b/common/main.c @@ -357,13 +357,6 @@ main( int argc, char *argv[] ) miniroot = BOOL_TRUE; } - /* initialize the spinlock allocator - */ - ok = qlock_init( ); - if ( ! ok ) { - return mlog_exit(EXIT_ERROR, RV_INIT); - } - /* initialize message logging (stage 2) - allocate the message lock */ ok = mlog_init2( ); diff --git a/common/qlock.c b/common/qlock.c index adaa7dd..ae8466d 100644 --- a/common/qlock.c +++ b/common/qlock.c @@ -37,14 +37,6 @@ typedef struct qlock qlock_t; /* internal qlock */ -#define QLOCK_THRDCNTMAX 256 - /* arbitrary limit on number of threads supported - */ - -static size_t qlock_thrdcnt; - /* how many threads have checked in - */ - typedef size_t ordmap_t; /* bitmap of ordinals. used to track what ordinals have * been allocated. @@ -58,12 +50,7 @@ static ordmap_t qlock_ordalloced; /* to enforce allocation of only one lock to each ordinal value */ -struct thrddesc { - pthread_t td_tid; - ordmap_t td_ordmap; -}; -typedef struct thrddesc thrddesc_t; -static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; +static __thread ordmap_t thread_ordmap; /* holds the ordmap for each thread */ @@ -83,69 +70,12 @@ static thrddesc_t qlock_thrddesc[ QLOCK_THRDCNTMAX ]; /* checks if any bits less than ord are set in the ordmap */ -/* REFERENCED */ -static bool_t qlock_inited = BOOL_FALSE; - /* to sanity check initialization - */ - -/* forward declarations - */ -static void qlock_ordmap_add( pthread_t tid ); -static ordmap_t *qlock_ordmapp_get( pthread_t tid ); -static ix_t qlock_thrdix_get( pthread_t tid ); - -bool_t -qlock_init( void ) -{ - /* sanity checks - */ - ASSERT( ! qlock_inited ); - - /* initially no threads checked in - */ - qlock_thrdcnt = 0; - - /* initially no ordinals allocated - */ - qlock_ordalloced = 0; - - /* now say we are initialized - */ - qlock_inited = BOOL_TRUE; - - /* add the parent thread to the thread list - */ - if ( ! qlock_thrdinit( )) { - qlock_inited = BOOL_FALSE; - return BOOL_FALSE; - } - - return BOOL_TRUE; -} - -bool_t -qlock_thrdinit( void ) -{ - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* add thread to ordmap list - */ - qlock_ordmap_add( pthread_self() ); - - return BOOL_TRUE; -} qlockh_t qlock_alloc( ix_t ord ) { qlock_t *qlockp; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* verify the ordinal is not already taken, and mark as taken */ ASSERT( ! QLOCK_ORDMAP_GET( qlock_ordalloced, ord )); @@ -172,48 +102,34 @@ qlock_lock( qlockh_t qlockh ) { qlock_t *qlockp = ( qlock_t * )qlockh; pthread_t tid; - ix_t thrdix; - ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* get the caller's tid and thread index + /* get the caller's tid */ tid = pthread_self(); - thrdix = qlock_thrdix_get( tid ); - - /* get the ordmap for this thread - */ - ordmapp = qlock_ordmapp_get( tid ); - /* assert that this lock not already held by this thread */ - if ( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )) { + if ( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock already held: thrd %d tid %lu ord %d map %x\n"), - thrdix, + _("lock already held: tid %lu ord %d map %x\n"), tid, qlockp->ql_ord, - *ordmapp ); + thread_ordmap ); } - ASSERT( ! QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); + ASSERT( ! QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )); /* assert that no locks with a lesser ordinal are held by this thread */ - if ( QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )) { + if ( QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord )) { mlog( MLOG_NORMAL | MLOG_WARNING | MLOG_NOLOCK, - _("lock ordinal violation: thrd %d tid %lu ord %d map %x\n"), - thrdix, + _("lock ordinal violation: tid %lu ord %d map %x\n"), tid, qlockp->ql_ord, - *ordmapp ); + thread_ordmap ); } - ASSERT( ! QLOCK_ORDMAP_CHK( *ordmapp, qlockp->ql_ord )); + ASSERT( ! QLOCK_ORDMAP_CHK( thread_ordmap, qlockp->ql_ord )); /* acquire the lock */ @@ -222,32 +138,23 @@ qlock_lock( qlockh_t qlockh ) /* add ordinal to this threads ordmap */ - QLOCK_ORDMAP_SET( *ordmapp, qlockp->ql_ord ); + QLOCK_ORDMAP_SET( thread_ordmap, qlockp->ql_ord ); } void qlock_unlock( qlockh_t qlockh ) { qlock_t *qlockp = ( qlock_t * )qlockh; - ordmap_t *ordmapp; /* REFERENCED */ intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - - /* get the ordmap for this thread - */ - ordmapp = qlock_ordmapp_get( pthread_self() ); - /* verify lock is held by this thread */ - ASSERT( QLOCK_ORDMAP_GET( *ordmapp, qlockp->ql_ord )); + ASSERT( QLOCK_ORDMAP_GET( thread_ordmap, qlockp->ql_ord )); /* clear lock's ord from thread's ord map */ - QLOCK_ORDMAP_CLR( *ordmapp, qlockp->ql_ord ); + QLOCK_ORDMAP_CLR( thread_ordmap, qlockp->ql_ord ); /* release the lock */ @@ -261,10 +168,6 @@ qsem_alloc( ix_t cnt ) sem_t *semp; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* allocate a semaphore */ semp = ( sem_t * )calloc( 1, sizeof( sem_t )); @@ -284,10 +187,6 @@ qsem_free( qsemh_t qsemh ) sem_t *semp = ( sem_t * )qsemh; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* destroy the mutex and condition */ rval = sem_destroy( semp ); @@ -304,10 +203,6 @@ qsemP( qsemh_t qsemh ) sem_t *semp = ( sem_t * )qsemh; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* "P" the semaphore */ rval = sem_wait( semp ); @@ -320,10 +215,6 @@ qsemV( qsemh_t qsemh ) sem_t *semp = ( sem_t * )qsemh; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - /* "V" the semaphore */ rval = sem_post( semp ); @@ -337,10 +228,6 @@ qsemPwouldblock( qsemh_t qsemh ) int count; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - rval = sem_getvalue( semp, &count ); ASSERT( !rval ); @@ -354,60 +241,8 @@ qsemPavail( qsemh_t qsemh ) int count; intgen_t rval; - /* sanity checks - */ - ASSERT( qlock_inited ); - rval = sem_getvalue( semp, &count ); ASSERT( !rval ); return count < 0 ? 0 : count; } - -/* internal ordinal map abstraction - */ -static void -qlock_ordmap_add( pthread_t tid ) -{ - ASSERT( qlock_thrdcnt < QLOCK_THRDCNTMAX ); - qlock_thrddesc[ qlock_thrdcnt ].td_tid = tid; - qlock_thrddesc[ qlock_thrdcnt ].td_ordmap = 0; - qlock_thrdcnt++; -} - -static thrddesc_t * -qlock_thrddesc_get( pthread_t tid ) -{ - thrddesc_t *p; - thrddesc_t *endp; - - for ( p = &qlock_thrddesc[ 0 ], - endp = &qlock_thrddesc[ qlock_thrdcnt ] - ; - p < endp - ; - p++ ) { - if ( pthread_equal( p->td_tid, tid ) ) { - return p; - } - } - - return 0; -} - -static ordmap_t * -qlock_ordmapp_get( pthread_t tid ) -{ - thrddesc_t *p; - p = qlock_thrddesc_get( tid ); - return &p->td_ordmap; -} - -static ix_t -qlock_thrdix_get( pthread_t tid ) -{ - thrddesc_t *p; - p = qlock_thrddesc_get( tid ); - ASSERT( p >= &qlock_thrddesc[ 0 ] ); - return ( ix_t )( p - &qlock_thrddesc[ 0 ] ); -} diff --git a/common/qlock.h b/common/qlock.h index ae411bb..6c2dd18 100644 --- a/common/qlock.h +++ b/common/qlock.h @@ -21,7 +21,7 @@ /* qlock - quick locks abstraction * * threads may allocate quick locks using qlock_alloc, and free them with - * qlock_free. the abstraction is initialized with qlock_init. + * qlock_free. * * deadlock detection is accomplished by giving an ordinal number to each * lock allocated, and record all locks held by each thread. locks may not @@ -48,15 +48,6 @@ typedef void *qlockh_t; /* opaque handle */ -extern bool_t qlock_init( void ); - /* called by main to initialize abstraction. returns FALSE if - * utility should abort. - */ - -extern bool_t qlock_thrdinit( void ); - /* called by each thread to prepare it for participation - */ - extern qlockh_t qlock_alloc( ix_t ord ); /* allocates a qlock with the specified ordinal. returns * NULL if lock can't be allocated. -- 1.7.0.4 From wkendall@sgi.com Mon Nov 7 15:04:00 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_64 autolearn=no version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7L40Ji076282 for ; Mon, 7 Nov 2011 15:04:00 -0600 Received: from estes.americas.sgi.com (estes.americas.sgi.com [128.162.236.10]) by relay3.corp.sgi.com (Postfix) with ESMTP id CDBF0AC007; Mon, 7 Nov 2011 13:03:56 -0800 (PST) Received: from augusta (augusta.americas.sgi.com [128.162.233.117]) by estes.americas.sgi.com (Postfix) with ESMTP id 94D3070016E1; Mon, 7 Nov 2011 15:03:56 -0600 (CST) Received: by augusta (Postfix, from userid 2022) id 5655320E36F6; Mon, 7 Nov 2011 15:03:56 -0600 (CST) From: Bill Kendall To: xfs@oss.sgi.com Cc: Bill Kendall Subject: [PATCH] xfstests: ignore fsstress file count for 022 Date: Mon, 7 Nov 2011 15:03:52 -0600 Message-Id: <1320699832-12693-1-git-send-email-wkendall@sgi.com> X-Mailer: git-send-email 1.7.0.4 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The recently added fsstress operations result in a different number of directories and files being created for xfsdump test 022, which causes the test to fail since the expected xfsrestore output records these numbers. Filter out the numbers to prevent this from breaking in the future. Also increase the number of fsstress operations used to populate the filesystem so that we get roughly the same number of directories and files as we used to for this test. Signed-off-by: Bill Kendall --- 022 | 3 ++- 022.out.irix | 2 +- 022.out.linux | 2 +- common.dump | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/022 b/022 index 76d18e6..b83b063 100755 --- a/022 +++ b/022 @@ -53,7 +53,8 @@ _require_tape $TAPE_DEV _create_dumpdir_stress _erase_hard _do_dump_sub -_do_restore +# filter out the file count, it changes as fsstress adds new operations +_do_restore | sed -e "/entries processed$/s/[0-9][0-9]*/NUM/g" _ls_compare_sub # success, all done diff --git a/022.out.irix b/022.out.irix index 725d6cd..f72876a 100644 --- a/022.out.irix +++ b/022.out.irix @@ -46,7 +46,7 @@ xfsrestore: searching media for directory dump xfsrestore: preparing drive xfsrestore: examining media file 0 xfsrestore: reading directories -xfsrestore: 30 directories and 89 entries processed +xfsrestore: NUM directories and NUM entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed diff --git a/022.out.linux b/022.out.linux index 9466747..a843b0c 100644 --- a/022.out.linux +++ b/022.out.linux @@ -46,7 +46,7 @@ xfsrestore: searching media for directory dump xfsrestore: preparing drive xfsrestore: examining media file 0 xfsrestore: reading directories -xfsrestore: 30 directories and 89 entries processed +xfsrestore: NUM directories and NUM entries processed xfsrestore: directory post-processing xfsrestore: restoring non-directory files xfsrestore: restore complete: SECS seconds elapsed diff --git a/common.dump b/common.dump index 31a75c5..493a5f7 100644 --- a/common.dump +++ b/common.dump @@ -301,7 +301,7 @@ _create_dumpdir_stress() _wipe_fs _param="-f link=10 -f creat=10 -f mkdir=10 -f truncate=5 -f symlink=10" - _count=200 + _count=240 rm -rf $dump_dir if ! mkdir $dump_dir; then echo " failed to mkdir $dump_dir" -- 1.7.0.4 From david@fromorbit.com Mon Nov 7 15:22:41 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7LMeIJ076792 for ; Mon, 7 Nov 2011 15:22:41 -0600 X-ASG-Debug-ID: 1320700957-56c700cc0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 300F41488917 for ; Mon, 7 Nov 2011 13:22:38 -0800 (PST) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id tzMeZyMNP21Myb76 for ; Mon, 07 Nov 2011 13:22:38 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAGhLuE55LIil/2dsb2JhbABDqXKBBoFyAQEFOhwjEAgDDgouFCUDIRPAPRKJGQSmIQ Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail05.adl6.internode.on.net with ESMTP; 08 Nov 2011 07:52:37 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNWeJ-00043Y-MZ; Tue, 08 Nov 2011 08:22:35 +1100 Date: Tue, 8 Nov 2011 08:22:35 +1100 From: Dave Chinner To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111107212235.GH5534@dastard> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1320700959 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79611 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 06:24:58PM +0400, Dmitry Monakhov wrote: > During stress testing we want to cover as much code paths as possible > fsstress is very good for this purpose. But it has expandable nature > (disk usage almost continually grow). So once it goes in no ENOSPC > condition it will be where till the end. But by running 'dd' writers > in parallel we can regularly trigger ENOSPC but only for a limited > periods of time because each time it opens the same file with O_TRUNC. ..... So you have a 512MB filesystem, and you do: > +# Disable all sync operations to get higher load > +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" > +_workout() > +{ > + echo "" > + echo "Run fsstress" > + echo "" > + num_iterations=10 > + enospc_time=2 > + out=$SCRATCH_MNT/fsstress.$$ > + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" > + echo "fsstress $args" >> $here/$seq.full > + $FSSTRESS_PROG $args > /dev/null 2>&1 & run a bunch of fsstress processes > + pid=$! > + echo "Run dd writers in parallel" > + for ((i=0; i < num_iterations; i++)) > + do > + # File will be opened with O_TRUNC each time > + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ > + > /dev/null 2>&1 > + sleep $enospc_time > + done Then write the same 1MB file 10 times, 2 seconds apart, giving a total space usage of the dd processes of 1MB over 20s. > + kill $pid > + wait $pid > +} Then kill the fsstress. AFAICT, fsstress won't always fill 511MB in 20s - on my test systems the fill rate is typically around 5s per 100MB, which would result in the filesystem not being filled with this test and hence not exercising ENOSPC. Perhaps this would be better done like test 083, which uses a fixed number of write-only operations per fsstress process that is known to end up at ENOSPC, rather than hoping it gets there in 20s. Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Nov 7 15:39:11 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA7LdBYd077662 for ; Mon, 7 Nov 2011 15:39:11 -0600 X-ASG-Debug-ID: 1320701131-1a7a02420000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ipmail05.adl6.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3954022903D for ; Mon, 7 Nov 2011 13:25:32 -0800 (PST) Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id isJRHpVsLnEyeiiH for ; Mon, 07 Nov 2011 13:25:32 -0800 (PST) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av0EAGhLuE55LIil/2dsb2JhbABDqXKBBoFyAQEFOhwjEAgDDgouFCUDIRPAPRKJGQSmIQ Received: from ppp121-44-136-165.lns20.syd7.internode.on.net (HELO dastard) ([121.44.136.165]) by ipmail05.adl6.internode.on.net with ESMTP; 08 Nov 2011 07:55:30 +1030 Received: from dave by dastard with local (Exim 4.76) (envelope-from ) id 1RNWh8-00044j-6K; Tue, 08 Nov 2011 08:25:30 +1100 Date: Tue, 8 Nov 2011 08:25:30 +1100 From: Dave Chinner To: Dmitry Monakhov Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111107212530.GI5534@dastard> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143] X-Barracuda-Start-Time: 1320701133 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79612 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Nov 03, 2011 at 06:24:58PM +0400, Dmitry Monakhov wrote: > +# real QA test starts here > +_supported_fs generic > +_supported_os Linux > +_require_scratch > +_need_to_be_root > + > +umount $SCRATCH_DEV 2>/dev/null > +_scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 > +_scratch_mount _require_scratch unmounts the SCRATCH_DEV. no need to do it explicitly. Same for the next new test in the series, too. Cheers, Dave. -- Dave Chinner david@fromorbit.com From aelder@sgi.com Mon Nov 7 20:02:04 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA8224oE085154 for ; Mon, 7 Nov 2011 20:02:04 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay2.corp.sgi.com (Postfix) with ESMTP id AB7C7304043; Mon, 7 Nov 2011 18:02:01 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:02:01 -0600 Message-ID: <1320717720.3186.87.camel@doink> Subject: Re: [PATCH v2 1/8] xfsdump: link with libpthread From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:02:00 -0600 In-Reply-To: <1320699511-12281-2-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-2-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > This patch changes xfsdump/xfsrestore to link with libpthread in > preparation for enabling multi-stream support. The libpthread checks > are the same as those used in xfsprogs. > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Looks good to me. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:02:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA822BY8085178 for ; Mon, 7 Nov 2011 20:02:11 -0600 Received: from xmail.sgi.com (pv-excas3-dc21.corp.sgi.com [137.38.102.206]) by relay3.corp.sgi.com (Postfix) with ESMTP id 32FB1AC004; Mon, 7 Nov 2011 18:02:08 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:02:07 -0600 Message-ID: <1320717726.3186.88.camel@doink> Subject: Re: [PATCH v2 2/8] xfsdump: remove multi-stream synchronous dir dump From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:02:06 -0600 In-Reply-To: <1320699511-12281-3-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-3-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > When doing multi-stream dumps, the directories are dumped by each > stream in lock-step fashion. A stream backs up a given directory, > then waits for all others to finish that directory, then they all > move on to the next one. During restore a single stream is chosen to > apply the directory dump. > > This patch changes xfsdump so that only one stream dumps the > directories. The other streams begin dumping files immediately. > This results in a faster backup, which is presumably why multiple > streams are being used in the first place. > > It also removes the barrier routines in qlock.c as they were only > used by the "SYNCDIR" code. > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Nice change. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:02:12 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA822CJG085187 for ; Mon, 7 Nov 2011 20:02:12 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay2.corp.sgi.com (Postfix) with ESMTP id 9F77A304043; Mon, 7 Nov 2011 18:02:12 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:02:12 -0600 Message-ID: <1320717731.3186.89.camel@doink> Subject: Re: [PATCH v2 3/8] xfsdump: implement lock abstraction with pthreads From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:02:11 -0600 In-Reply-To: <1320699511-12281-4-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-4-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > This patch implements xfsdump's locking abstraction using pthread > locking primitives. The obsolete IRIX locking code has been removed. > > The POSIX semaphore implementation on Linux does not provide a count > of threads waiting to decrement the semaphore, so qsemPblocked() > has been removed. It was called from a few asserts, but the current > count of the semaphore was also checked in adjacent asserts, so > it was deemed acceptable to remove the qsemPblocked() asserts. > > A few debug-level log calls had to be changed to use the "no lock" > flag to prevent locks from being acquired out of order. This is > used in various debug messages throughout xfsdump. > > Signed-off-by: Bill Kendall I have not done a very thorough review, but this looks OK to me. -Alex From aelder@sgi.com Mon Nov 7 20:02:18 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA822I8M085215 for ; Mon, 7 Nov 2011 20:02:18 -0600 Received: from xmail.sgi.com (pv-excas3-dc21.corp.sgi.com [137.38.102.206]) by relay3.corp.sgi.com (Postfix) with ESMTP id 1517BAC003; Mon, 7 Nov 2011 18:02:18 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:02:17 -0600 Message-ID: <1320717736.3186.90.camel@doink> Subject: Re: [PATCH v2 4/8] xfsdump: simplify qlock ordinal bitmap From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:02:16 -0600 In-Reply-To: <1320699511-12281-5-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-5-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > The qlock abstraction manages an array of ordinal bitmaps, one bitmap > for each thread. The bitmap indicates which locks a thread is holding > and is used to verify that the locks are obtained in the correct order. > > There is no need to use an array to store the bitmaps, and in fact > adding entries to the array is broken because qlock_thrdcnt is not > protected by a lock. A simpler approach is to create a per-thread bitmap > using thread local storage. > > With this change, there is no need for each new thread to register with > the qlock abstraction, so qlock_thrdinit() goes away. Remove > qlock_init() since it can be statically initialized, and that means > there's no need for a flag (qlock_inited) indicating that the module has > been initialized. Also there's no longer a need to track or limit the > number of threads that the qlock abstraction can handle. > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Wow, nice cleanup. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:02:25 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA822Ohm085235 for ; Mon, 7 Nov 2011 20:02:25 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay1.corp.sgi.com (Postfix) with ESMTP id D20DC8F804B; Mon, 7 Nov 2011 18:02:21 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:02:21 -0600 Message-ID: <1320717740.3186.91.camel@doink> Subject: Re: [PATCH v2 5/8] xfsdump: convert IRIX sproc threads to pthreads From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:02:20 -0600 In-Reply-To: <1320699511-12281-6-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-6-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > The existing (disabled) threading code in xfsdump is based on IRIX > sprocs. This patch converts the code to use pthreads. The threading code > remains disabled at this point. > > Changes: > - pid_t to pthread_t > - getpid() to pthread_self() > - "pid1 == pid2" to pthread_equal(tid1, tid2) > - sigprocmask() to pthread_sigmask() > - sproc() to pthread_create() > > Also the following are not referenced and have been removed: > - cldmgr_pid2streamix() > - PROCMAX > - r_slavepid > - sproc.c and sproc.h > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Looks good. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:03:23 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA823NYi085277 for ; Mon, 7 Nov 2011 20:03:23 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay1.corp.sgi.com (Postfix) with ESMTP id 2452E8F8050; Mon, 7 Nov 2011 18:03:23 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:03:22 -0600 Message-ID: <1320717802.3186.97.camel@doink> Subject: Re: [PATCH v2 6/8] xfsdump: process thread exit status From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:03:22 -0600 In-Reply-To: <1320699511-12281-7-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-7-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > When IRIX sprocs were in use, the main thread was notified of a thread > exit just as if a child process exited -- it received SIGCHLD. The main > thread would grab the pid and exit status, then call cldmgr_died() to > inform it that the child was gone so the slot in the child array could > be freed up for reuse. > > This patch implements a similar mechanism for pthreads. The "c_busy" > field in struct cld has been replaced with a "c_state" field that > indicates whether the array slot is free (C_AVAIL), in use (C_ALIVE), or > is waiting to be joined (C_EXITED). Additionally a "c_exit_code" field > has been added to store the thread's exit value. Normally this is set > when the thread entry function returns, but it is initialized to > EXIT_INTERRUPT in case the thread is cancelled or calls pthread_exit() > rather than returning (neither of which happens in the code today). > > When the child thread starts, it registers a pthread cleanup handler > which takes care of marking the child as C_EXITED and notifies the main > thread that a child is gone. Doing this in a cleanup handler ensures > that it's done regardless of how the thread exits. The main thread's > loop is based around sigsuspsend(), so the notification is done by > sending SIGUSR1. The main thread will then call cldmgr_join() to join > all exited threads and obtain their exit status. > > Additional changes: > * cldmgr_findbypid() has been removed, it's no longer referenced. > * stream_dead() no longer grabs the lock(), because it's called > only from cldmgr_join() which already holds the lock(). > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Looks OK to me. Again, I've looked over this pretty closely, but have not been as thorough as I often am. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:03:30 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA823ULA085293 for ; Mon, 7 Nov 2011 20:03:30 -0600 Received: from xmail.sgi.com (pv-excas3-dc21.corp.sgi.com [137.38.102.206]) by relay3.corp.sgi.com (Postfix) with ESMTP id BDCF5AC006; Mon, 7 Nov 2011 18:03:28 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:03:28 -0600 Message-ID: <1320717807.3186.98.camel@doink> Subject: Re: [PATCH v2 7/8] xfsdump: path lookup cache must be thread specific From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:03:27 -0600 In-Reply-To: <1320699511-12281-8-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-8-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > The xfsrestore path lookup routine (maps an ino + generation number to a > pathname) contains a cache for fast lookups of consecutive files in the > same parent directory. This cache either needs to be protected by a lock > or a cache is needed per thread. I took the latter approach since it > doesn't add lock contention and it increases the cache hit rate (it's > relatively unlikely that each stream is processing files in the same > directory). > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig So you're fixing a bug, and making it work better at the same time, by simply adding one keyword. Looks good. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:03:35 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA823ZSx085309 for ; Mon, 7 Nov 2011 20:03:35 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay1.corp.sgi.com (Postfix) with ESMTP id 87F588F8050; Mon, 7 Nov 2011 18:03:35 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:03:35 -0600 Message-ID: <1320717814.3186.99.camel@doink> Subject: Re: [PATCH v2 8/8] xfsdump: enable multiple streams From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:03:34 -0600 In-Reply-To: <1320699511-12281-9-git-send-email-wkendall@sgi.com> References: <1320699511-12281-1-git-send-email-wkendall@sgi.com> <1320699511-12281-9-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 14:58 -0600, Bill Kendall wrote: > IRIX contained an environment referred to as "miniroot" where > sproc threads were either not available, or at least not used > in xfsdump. Throughout xfsdump there's a "miniroot" variable > which indicates whether or not thread support is enabled. On > Linux this variable has always been false in order to disable > support for multiple streams. > > Now that the threading infracstructure has been converted over > to pthreads, this patch removes the "miniroot" variable and > enables the option of using multiple streams. > > Note that another feature in xfsdump, using a ring buffer for > I/O to tapes, also depends on thread support. I'm leaving that > disabled for now until more testing has been done. > > Signed-off-by: Bill Kendall > Reviewed-by: Christoph Hellwig Really, really nice work Bill. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:05:29 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA825TMJ085357 for ; Mon, 7 Nov 2011 20:05:29 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay1.corp.sgi.com (Postfix) with ESMTP id AFDF78F8039; Mon, 7 Nov 2011 18:05:29 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:05:29 -0600 Message-ID: <1320717928.3186.100.camel@doink> Subject: Re: [PATCH] xfstests: ignore fsstress file count for 022 From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:05:28 -0600 In-Reply-To: <1320699832-12693-1-git-send-email-wkendall@sgi.com> References: <1320699832-12693-1-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 15:03 -0600, Bill Kendall wrote: > The recently added fsstress operations result in a different > number of directories and files being created for xfsdump test > 022, which causes the test to fail since the expected xfsrestore > output records these numbers. Filter out the numbers to prevent > this from breaking in the future. > > Also increase the number of fsstress operations used to populate > the filesystem so that we get roughly the same number of > directories and files as we used to for this test. > > Signed-off-by: Bill Kendall Looks good. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:11:33 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA82BW1c085481 for ; Mon, 7 Nov 2011 20:11:32 -0600 Received: from xmail.sgi.com (pv-excas3-dc21.corp.sgi.com [137.38.102.206]) by relay3.corp.sgi.com (Postfix) with ESMTP id 272DAAC006; Mon, 7 Nov 2011 18:11:32 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:11:31 -0600 Message-ID: <1320718291.3186.103.camel@doink> Subject: Re: [PATCH v2] xfstests: add 059 and 060 for multi-stream xfsdump From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:11:31 -0600 In-Reply-To: <1320686536-1399-1-git-send-email-wkendall@sgi.com> References: <1320686536-1399-1-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 11:22 -0600, Bill Kendall wrote: > Implement the multi-stream placeholder tests 059 and 060. The comments > imply that these tests existed on IRIX, but I was unable to find them. > > Test 059 does a simple 4-way dump and restore. Test 060 does a 4-way > dump, then restores each dump file individually (in a cumulative > fashion). > > The tests are skipped if the installed xfsdump does not support > multi-stream output. > > Signed-off-by: Bill Kendall I haven't run these yet, but this looks good. Reviewed-by: Alex Elder From aelder@sgi.com Mon Nov 7 20:11:38 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA82BcFj085498 for ; Mon, 7 Nov 2011 20:11:38 -0600 Received: from xmail.sgi.com (pv-excas3-dc21-nlb.corp.sgi.com [137.38.102.207]) by relay1.corp.sgi.com (Postfix) with ESMTP id 769C38F8039; Mon, 7 Nov 2011 18:11:38 -0800 (PST) Received: from [127.0.0.1] (128.162.232.50) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Mon, 7 Nov 2011 20:11:38 -0600 Message-ID: <1320718297.3186.104.camel@doink> Subject: Re: [PATCH] xfstests: fix permissions on non-executable tests From: Alex Elder Reply-To: To: Bill Kendall CC: Date: Mon, 7 Nov 2011 20:11:37 -0600 In-Reply-To: <1320688253-2432-1-git-send-email-wkendall@sgi.com> References: <1320688253-2432-1-git-send-email-wkendall@sgi.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.0- Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Originating-IP: [128.162.232.50] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, 2011-11-07 at 11:50 -0600, Bill Kendall wrote: > Tests that are not executable are made executable when they are > run, causing git to report them as modified. Fix this by > changing non-executable tests to have permissions of 0755. > > Signed-off-by: Bill Kendall Looks good. I update these occasionally so they're all executable again. I guess I should do something so this is more automatic in the future... Thanks. Reviewed-by: Alex Elder From wu.bo@cn.fujitsu.com Mon Nov 7 21:39:14 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA83dCR7088823 for ; Mon, 7 Nov 2011 21:39:13 -0600 X-ASG-Debug-ID: 1320723550-2b6200530000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 042AB16235EA for ; Mon, 7 Nov 2011 19:39:10 -0800 (PST) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id 7T2NdLvLUv5hN4pJ for ; Mon, 07 Nov 2011 19:39:10 -0800 (PST) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id BC375170125; Tue, 8 Nov 2011 11:39:08 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA83d7Ue010512; Tue, 8 Nov 2011 11:39:07 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110811390767-21274 ; Tue, 8 Nov 2011 11:39:07 +0800 Message-ID: <4EB8A441.7090506@cn.fujitsu.com> Date: Tue, 08 Nov 2011 11:38:41 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com, Christoph Hellwig X-ASG-Orig-Subj: [PATCH v2 0/3] xfstests: Add a group of reservation space test Subject: [PATCH v2 0/3] xfstests: Add a group of reservation space test X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:39:07, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:39:09, Serialize complete at 2011-11-08 11:39:09 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320723551 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0041 1.0000 -1.9944 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.99 X-Barracuda-Spam-Status: No, SCORE=-1.99 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79637 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Changes v1->v2: - Make test-set support generic This patch set add a group of reservation space test. Especailly for btrfs. It includes three parts: copy workload, prealloc and write posix. For test 273, I hope it's usefull for Josef's reserve improve work. For test 274 and 275, the current btrfs is not pass yet. Wu Bo (3): xfstests 273: add a copy and reserve test xfstests 274: add a prealloc and reserve test xfstests 275: add a write and reserve test 273 | 145 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 273.out | 4 ++ 274 | 91 +++++++++++++++++++++++++++++++++++++++ 274.out | 5 ++ 275 | 89 ++++++++++++++++++++++++++++++++++++++ 275.out | 5 ++ group | 3 + 7 files changed, 342 insertions(+), 0 deletions(-) create mode 100755 273 create mode 100644 273.out create mode 100755 274 create mode 100644 274.out create mode 100755 275 create mode 100644 275.out -- 1.7.3.1 From wu.bo@cn.fujitsu.com Mon Nov 7 21:41:17 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA83fHOa088887 for ; Mon, 7 Nov 2011 21:41:17 -0600 X-ASG-Debug-ID: 1320723668-5ec900410000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4E1AF22A342 for ; Mon, 7 Nov 2011 19:41:13 -0800 (PST) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id PZnW87eXAmOOEHpW for ; Mon, 07 Nov 2011 19:41:13 -0800 (PST) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id 1D0A617011D; Tue, 8 Nov 2011 11:41:07 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA83f4LU010605; Tue, 8 Nov 2011 11:41:05 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110811410458-21287 ; Tue, 8 Nov 2011 11:41:04 +0800 Message-ID: <4EB8A4B6.3030604@cn.fujitsu.com> Date: Tue, 08 Nov 2011 11:40:38 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com, Christoph Hellwig X-ASG-Orig-Subj: [PATCH v2 1/3] xfstests 273: add a copy and reserve test Subject: [PATCH v2 1/3] xfstests 273: add a copy and reserve test References: <4EB8A441.7090506@cn.fujitsu.com> In-Reply-To: <4EB8A441.7090506@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:41:04, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:41:07, Serialize complete at 2011-11-08 11:41:07 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320723676 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79636 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is a stress test. It creates a set of threads for coping small files into disk. I use a 2G disk for test, the ENOSPC arises usually but the disk is not full under kenerl 3.0 with intel64. Signed-off-by: Wu Bo --- 273 | 145 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 273.out | 4 ++ group | 1 + 3 files changed, 150 insertions(+), 0 deletions(-) create mode 100755 273 create mode 100644 273.out diff --git a/273 b/273 new file mode 100755 index 0000000..2965132 --- /dev/null +++ b/273 @@ -0,0 +1,145 @@ +#! /bin/bash +# FS QA Test No. 273 +# +# reservation test with heavy cp workload +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -rf $SCRATCH_MNT/* $tmp.* + _scratch_unmount +} + +. ./common.rc +. ./common.filter + +threads=50 +count=1 + +_threads_set() +{ + _cpu_num=`cat /proc/cpuinfo | grep "processor" | wc -l` + threads=$(($_cpu_num * 50)) + if [ $threads -gt 200 ] + then + threads=200 + fi +} + +_file_create() +{ + _i=0 + + if ! mkdir $SCRATCH_MNT/origin + then + echo "mkdir origin err" + status=1 + exit + fi + + cd $SCRATCH_MNT/origin + + _disksize=`df --block-size=1 | grep $SCRATCH_DEV | awk '{print $2}'` + _disksize=$(($_disksize / 3)) + _num=$(($_disksize / $count / $threads / 4096)) + _count=$count + while [ $_i -lt $_num ] + do + dd if=/dev/zero of=file_$_i bs=4096 count=$_count >/dev/null 2>&1 + _i=$(($_i + 1)) + done + + cd $here +} + +_porter() +{ + _suffix=$1 + + if ! mkdir $SCRATCH_MNT/sub_$_suffix + then + echo "mkdir sub_xxx err" + status=1 + exit + fi + + cp -r $SCRATCH_MNT/origin $SCRATCH_MNT/sub_$_suffix >$seq.full 2>&1 + if [ $? -ne 0 ] + then + echo "_porter $_suffix not complete" + fi + + sync +} + +_do_workload() +{ + _pids="" + _pid=1 + + rm -rf $SCRATCH_MNT/* + + _threads_set + _file_create + + _threads=$threads + + while [ $_pid -lt $_threads ] + do + _porter $_pid & + _pids="$_pids $!" + _pid=$(($_pid + 1)) + done + + wait $_pids +} + +# real QA test starts here +_supported_fs generic +_supported_os IRIX Linux +_require_scratch + +echo "------------------------------" +echo "start the workload" +echo "------------------------------" + +rm -f $seq.full + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((2 * 1024 * 1024 * 1024)) >>$seq.full 2>&1 +_scratch_mount + +_do_workload + +_check_scratch_fs +status=$? +exit diff --git a/273.out b/273.out new file mode 100644 index 0000000..146af43 --- /dev/null +++ b/273.out @@ -0,0 +1,4 @@ +QA output created by 273 +------------------------------ +start the workload +------------------------------ diff --git a/group b/group index 616cbdd..b8245bf 100644 --- a/group +++ b/group @@ -386,3 +386,4 @@ deprecated 270 auto quota rw prealloc ioctl enospc 271 auto rw quick 272 auto enospc rw +273 auto rw -- 1.7.3.1 From wu.bo@cn.fujitsu.com Mon Nov 7 21:41:40 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_24, J_CHICKENPOX_47 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA83fdsE088914 for ; Mon, 7 Nov 2011 21:41:39 -0600 X-ASG-Debug-ID: 1320723697-27c3008b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D2CB61623603 for ; Mon, 7 Nov 2011 19:41:37 -0800 (PST) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id nTZwQZ3M0MdMojQl for ; Mon, 07 Nov 2011 19:41:37 -0800 (PST) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id 40C58170128; Tue, 8 Nov 2011 11:41:36 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA83fYdA010617; Tue, 8 Nov 2011 11:41:34 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110811413420-21293 ; Tue, 8 Nov 2011 11:41:34 +0800 Message-ID: <4EB8A4D4.4000000@cn.fujitsu.com> Date: Tue, 08 Nov 2011 11:41:08 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com, Christoph Hellwig X-ASG-Orig-Subj: [PATCH v2 2/3] xfstests 274: add a prealloc and reserve test Subject: [PATCH v2 2/3] xfstests 274: add a prealloc and reserve test References: <4EB8A441.7090506@cn.fujitsu.com> In-Reply-To: <4EB8A441.7090506@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:41:34, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:41:36, Serialize complete at 2011-11-08 11:41:36 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320723698 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79637 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is for preallocation test. If the disk is full, just with a prealloc file has some free space that prealloc early. We need to check whether the write to the free space is success or not. Signed-off-by: Wu Bo --- 274 | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 274.out | 5 +++ group | 1 + 3 files changed, 97 insertions(+), 0 deletions(-) create mode 100755 274 create mode 100644 274.out diff --git a/274 b/274 new file mode 100755 index 0000000..b658004 --- /dev/null +++ b/274 @@ -0,0 +1,91 @@ +#! /bin/bash +# FS QA Test No. 274 +# +# preallocation test +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $SCRATCH_MNT/* $tmp.* + _scratch_unmount +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs generic +_supported_os IRIX Linux +_require_scratch + +echo "------------------------------" +echo "preallocation test" +echo "------------------------------" + +rm -f $seq.full + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1 +_scratch_mount + +rm -rf $SCRATCH_MNT/* +cd $SCRATCH_MNT +dd if=/dev/zero of=test bs=4K count=1 >/dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +fallocate -n -o 4K -l 1M test >/dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fallocate file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp1 bs=1M >/dev/null 2>&1 +dd if=/dev/zero of=tmp2 bs=4K >/dev/null 2>&1 +sync + +dd if=/dev/zero of=test seek=1 bs=4K count=2 conv=notrunc >/dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "fill prealloc range err" + status=1 + exit +fi + +echo "done" +exit diff --git a/274.out b/274.out new file mode 100644 index 0000000..edbf3cc --- /dev/null +++ b/274.out @@ -0,0 +1,5 @@ +QA output created by 274 +------------------------------ +preallocation test +------------------------------ +done diff --git a/group b/group index b8245bf..44a9999 100644 --- a/group +++ b/group @@ -387,3 +387,4 @@ deprecated 271 auto rw quick 272 auto enospc rw 273 auto rw +274 auto rw -- 1.7.3.1 From wu.bo@cn.fujitsu.com Mon Nov 7 21:42:25 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA83gOoP088944 for ; Mon, 7 Nov 2011 21:42:25 -0600 X-ASG-Debug-ID: 1320723739-4b9a00ca0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from song.cn.fujitsu.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 40E5622A3BC for ; Mon, 7 Nov 2011 19:42:22 -0800 (PST) Received: from song.cn.fujitsu.com (cn.fujitsu.com [222.73.24.84]) by cuda.sgi.com with ESMTP id aSyzZVGUf6LVsOFK for ; Mon, 07 Nov 2011 19:42:22 -0800 (PST) Received: from tang.cn.fujitsu.com (tang.cn.fujitsu.com [10.167.250.3]) by song.cn.fujitsu.com (Postfix) with ESMTP id E1805170144; Tue, 8 Nov 2011 11:42:13 +0800 (CST) Received: from mailserver.fnst.cn.fujitsu.com (tang.cn.fujitsu.com [127.0.0.1]) by tang.cn.fujitsu.com (8.14.3/8.13.1) with ESMTP id pA83gCts010661; Tue, 8 Nov 2011 11:42:12 +0800 Received: from localhost.localdomain ([10.167.225.174]) by mailserver.fnst.cn.fujitsu.com (Lotus Domino Release 8.5.1FP4) with ESMTP id 2011110811421138-21310 ; Tue, 8 Nov 2011 11:42:11 +0800 Message-ID: <4EB8A4F9.8060006@cn.fujitsu.com> Date: Tue, 08 Nov 2011 11:41:45 +0800 From: WuBo User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100921 Fedora/3.1.4-1.fc14 Thunderbird/3.1.4 MIME-Version: 1.0 To: linux-fsdevel@vger.kernel.org, Linux Btrfs , xfs@oss.sgi.com, Christoph Hellwig X-ASG-Orig-Subj: [PATCH v2 3/3] xfstests 275: add a write and reserve test Subject: [PATCH v2 3/3] xfstests 275: add a write and reserve test References: <4EB8A441.7090506@cn.fujitsu.com> In-Reply-To: <4EB8A441.7090506@cn.fujitsu.com> X-MIMETrack: Itemize by SMTP Server on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:42:11, Serialize by Router on mailserver/fnst(Release 8.5.1FP4|July 25, 2010) at 2011-11-08 11:42:14, Serialize complete at 2011-11-08 11:42:14 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: cn.fujitsu.com[222.73.24.84] X-Barracuda-Start-Time: 1320723744 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79636 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This test is for write-posix test. If writing a file when the disk is almost full, the posix wants the call to write as much as possible but not none. quote the POSIX: If a write() requests that more bytes be written than there is room for (for example, [XSI] [Option Start] the process' file size limit or [Option End] the physical end of a medium), only as many bytes as there is room for shall be written. For example, suppose there is space for 20 bytes more in a file before reaching a limit. A write of 512 bytes will return 20. The next write of a non-zero number of bytes would give a failure return (except as noted below). Signed-off-by: Wu Bo --- 275 | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 275.out | 5 +++ group | 1 + 3 files changed, 95 insertions(+), 0 deletions(-) create mode 100755 275 create mode 100644 275.out diff --git a/275 b/275 new file mode 100755 index 0000000..214262e --- /dev/null +++ b/275 @@ -0,0 +1,89 @@ +#! /bin/bash +# FS QA Test No. 275 +# +# The posix write test. when write size is larger than disk free size, +# should write as more as possible +# +#----------------------------------------------------------------------- +# Copyright (c) 2011-2012 Fujitsu, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +#creator +owner=wu.bo@cn.fujitsu.com + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=0 # success is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $SCRATCH_MNT/* $tmp.* + _scratch_unmount +} + +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs generic +_supported_os IRIX Linux +_require_scratch + +echo "------------------------------" +echo "write lack test" +echo "------------------------------" + +rm -f $seq.full + +umount $SCRATCH_DEV 2>/dev/null +_scratch_mkfs_sized $((1 * 1024 * 1024 * 1024)) >>$seq.full 2>&1 +_scratch_mount + +rm -rf $SCRATCH_MNT/* +cd $SCRATCH_MNT + +dd if=/dev/zero of=tmp1 bs=4K count=1 >/dev/null 2>&1 +if [ $? -ne 0 ] +then + echo "create file err" + status=1 + exit +fi + +dd if=/dev/zero of=tmp2 bs=1M >/dev/null 2>&1 +dd if=/dev/zero of=tmp3 bs=4K >/dev/null 2>&1 +sync + +rm -f tmp1 +sync + +dd if=/dev/zero of=tmp1 bs=8K count=1 >/dev/null 2>&1 +_filesize=`du tmp1 | awk '{print $1}'` +if [ $_filesize -ne 4 ] +then + echo "write file err" + status=1 + exit +fi + +echo "done" +exit diff --git a/275.out b/275.out new file mode 100644 index 0000000..30af43c --- /dev/null +++ b/275.out @@ -0,0 +1,5 @@ +QA output created by 275 +------------------------------ +write lack test +------------------------------ +done diff --git a/group b/group index 44a9999..8cdddd6 100644 --- a/group +++ b/group @@ -388,3 +388,4 @@ deprecated 272 auto enospc rw 273 auto rw 274 auto rw +275 auto rw -- 1.7.3.1 From rjevskiy@gmail.com Tue Nov 8 02:36:55 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88athg101338 for ; Tue, 8 Nov 2011 02:36:55 -0600 X-ASG-Debug-ID: 1320741131-394902f00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6825922AAF0 for ; Tue, 8 Nov 2011 00:32:11 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id AGhOPFDqYHq5Sgl6 for ; Tue, 08 Nov 2011 00:32:11 -0800 (PST) Received: by bkaq10 with SMTP id q10so212773bka.26 for ; Tue, 08 Nov 2011 00:32:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=3XCQnW/ZKl3UBhNvO4pJcG85DcMuWpcswMoEV4KlepY=; b=TceRhVAY1rt+1ZEYR1nlBpGJckuH1PyJMyHa0t2HYtRIVJL96NNAj2ka7CDLnUR5EQ yDunDjD5mriOn3M+TA5h+UxEhc0/oAsli7X14hGlIDwI0b953RppbxEZeQ/wLsKVb4H7 UDH3mZcA7OrM8iJg0ZOTlc/lSow/VlYBG4rPw= Received: by 10.204.130.85 with SMTP id r21mr22042432bks.38.1320741131043; Tue, 08 Nov 2011 00:32:11 -0800 (PST) Received: from smtp.gmail.com (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id dq2sm103409bkb.11.2011.11.08.00.32.06 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Nov 2011 00:32:07 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: Dave Chinner Cc: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com, hch@lst.de, aelder@sgi.com, tytso@mit.edu X-ASG-Orig-Subj: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 09/12] xfstests: add a new test that runs fsstress under ENOSPC conditions In-Reply-To: <20111107212235.GH5534@dastard> References: <1320330301-2682-1-git-send-email-dmonakhov@openvz.org> <1320330301-2682-10-git-send-email-dmonakhov@openvz.org> <20111107212235.GH5534@dastard> User-Agent: Notmuch/0.5-69-g3e4a9d6 (http://notmuchmail.org) Emacs/23.1.1 (i486-pc-linux-gnu) Date: Tue, 08 Nov 2011 12:32:05 +0400 Message-ID: <87lirrkoai.fsf@dmbot.sw.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320741132 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79656 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, 8 Nov 2011 08:22:35 +1100, Dave Chinner wrote: > On Thu, Nov 03, 2011 at 06:24:58PM +0400, Dmitry Monakhov wrote: > > During stress testing we want to cover as much code paths as possible > > fsstress is very good for this purpose. But it has expandable nature > > (disk usage almost continually grow). So once it goes in no ENOSPC > > condition it will be where till the end. But by running 'dd' writers > > in parallel we can regularly trigger ENOSPC but only for a limited > > periods of time because each time it opens the same file with O_TRUNC. > > ..... > > So you have a 512MB filesystem, and you do: > > > +# Disable all sync operations to get higher load > > +FSSTRESS_AVOID="$FSSTRESS_AVOID -ffsync=0 -fsync=0 -ffdatasync=0" > > +_workout() > > +{ > > + echo "" > > + echo "Run fsstress" > > + echo "" > > + num_iterations=10 > > + enospc_time=2 > > + out=$SCRATCH_MNT/fsstress.$$ > > + args="-p128 -n999999999 -f setattr=1 $FSSTRESS_AVOID -d $out" > > + echo "fsstress $args" >> $here/$seq.full > > + $FSSTRESS_PROG $args > /dev/null 2>&1 & > > run a bunch of fsstress processes > > > + pid=$! > > + echo "Run dd writers in parallel" > > + for ((i=0; i < num_iterations; i++)) > > + do > > + # File will be opened with O_TRUNC each time > > + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ > > + > /dev/null 2>&1 > > + sleep $enospc_time > > + done > > Then write the same 1MB file 10 times, 2 seconds apart, giving a > total space usage of the dd processes of 1MB over 20s. Hmm.. definitely it should not have count parameter at all, it supposed to run until ENOSPC (that's why it's called spaceconsumer), similar to 270'th testcase. Patch will be in the list in a minute. > > > + kill $pid > > + wait $pid > > +} > > Then kill the fsstress. > > AFAICT, fsstress won't always fill 511MB in 20s - on my test systems > the fill rate is typically around 5s per 100MB, which would result > in the filesystem not being filled with this test and hence not > exercising ENOSPC. Perhaps this would be better done like test 083, > which uses a fixed number of write-only operations per fsstress > process that is known to end up at ENOSPC, rather than hoping it > gets there in 20s. > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html From dmonlist@gmail.com Tue Nov 8 02:42:02 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_43, T_DKIM_INVALID autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88g2iK101451 for ; Tue, 8 Nov 2011 02:42:02 -0600 X-ASG-Debug-ID: 1320741720-038502ee0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 972451DBA4B0 for ; Tue, 8 Nov 2011 00:42:01 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id WjCA7erIMvGb25Q6 for ; Tue, 08 Nov 2011 00:42:01 -0800 (PST) Received: by bkaq10 with SMTP id q10so218577bka.26 for ; Tue, 08 Nov 2011 00:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer; bh=qpoJbccGgXMJbKSfL+3envv+suRBVjan3uyf7vI5ZB4=; b=aaD4cnUFqqQq0pWH3D59Z080v5akkVBI2SDexB4zkrrPLoY2tu+OdtVlk4Z8HeagJp /Dti81FLD9luFLopBJYsf8WVT84KapkA+Iy4c9DoVtuas2bHnehlz7DKXofh55QbaDjV WnwxyQm4+dgeIh1kjx0xHwiZ7coTFYZIPii9c= Received: by 10.204.152.4 with SMTP id e4mr13187517bkw.56.1320741720494; Tue, 08 Nov 2011 00:42:00 -0800 (PST) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id fu17sm728823bkc.9.2011.11.08.00.41.58 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Nov 2011 00:41:59 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com Cc: hch@lst.de, david@fromorbit.com, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 1/3] xfstests: fix argument typo Subject: [PATCH 1/3] xfstests: fix argument typo Date: Tue, 8 Nov 2011 12:41:53 +0400 Message-Id: <1320741715-32692-1-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320741721 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0001 1.0000 -2.0207 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79656 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Signed-off-by: Dmitry Monakhov --- ltp/fsstress.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/ltp/fsstress.c b/ltp/fsstress.c index 09c9529..9b3af17 100644 --- a/ltp/fsstress.c +++ b/ltp/fsstress.c @@ -385,7 +385,7 @@ int main(int argc, char **argv) i = 0; if (optarg[0] == 'c') i = 1; - show_ops(1, NULL); + show_ops(i, NULL); printf("\n"); nousage=1; break; -- 1.7.1 From dmonlist@gmail.com Tue Nov 8 02:42:03 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88g3Kr101459 for ; Tue, 8 Nov 2011 02:42:03 -0600 X-ASG-Debug-ID: 1320741720-038502ee0001-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E8E791DBA4B2 for ; Tue, 8 Nov 2011 00:42:02 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id hi0T6s3XttaFDQ5R for ; Tue, 08 Nov 2011 00:42:02 -0800 (PST) Received: by mail-bw0-f53.google.com with SMTP id q10so218577bka.26 for ; Tue, 08 Nov 2011 00:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=IoM1ykMLVPD6GbFdGN7V0fBc11lG3scNXoxHC5o6kEA=; b=YT1t9Pb2RbvFxtNdhajCtbclRy8yECk1/9dtl5atHH4XFu3JXgFk12SoEkk+U+Em39 juOEiv4G87IVTLcbDcyh3UOLTBEalSy2j3+fmyuAsXyr1iSUmAOmc2vQJgyoPB14NQVP 8T/xA2yPzyO6R+isKBLH4GPLS7jrnKGU3p8rE= Received: by 10.205.114.65 with SMTP id ez1mr21546679bkc.99.1320741722206; Tue, 08 Nov 2011 00:42:02 -0800 (PST) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id fu17sm728823bkc.9.2011.11.08.00.42.00 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Nov 2011 00:42:01 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com Cc: hch@lst.de, david@fromorbit.com, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 2/3] xfstests: fix typo in 260'th test Subject: [PATCH 2/3] xfstests: fix typo in 260'th test Date: Tue, 8 Nov 2011 12:41:54 +0400 Message-Id: <1320741715-32692-2-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320741715-32692-1-git-send-email-dmonakhov@openvz.org> References: <1320741715-32692-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320741722 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79656 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean dd should not have count argument because it suposed to work untill ENOSPC. Also it is reasonable to log dd's output, do that for both 269'th and 270'th tests Signed-off-by: Dmitry Monakhov --- 269 | 4 ++-- 270 | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/269 b/269 index 61e83ad..206492b 100755 --- a/269 +++ b/269 @@ -53,8 +53,8 @@ _workout() for ((i=0; i < num_iterations; i++)) do # File will be opened with O_TRUNC each time - dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M count=1 \ - > /dev/null 2>&1 + dd if=/dev/zero of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M \ + >> $here/$seq.full 2>&1 sleep $enospc_time done kill $pid diff --git a/270 b/270 index ed1e31e..e45029f 100755 --- a/270 +++ b/270 @@ -63,7 +63,7 @@ _workout() # File will be opened with O_TRUNC each time su $qa_user -c "dd if=/dev/zero \ of=$SCRATCH_MNT/SPACE_CONSUMER bs=1M " \ - > /dev/null 2>&1 + >> $here/$seq.full 2>&1 sleep $enospc_time done -- 1.7.1 From dmonlist@gmail.com Tue Nov 8 02:42:05 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88g5hS101472 for ; Tue, 8 Nov 2011 02:42:05 -0600 X-ASG-Debug-ID: 1320741720-038502ee0002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8B9451DBA4BA for ; Tue, 8 Nov 2011 00:42:04 -0800 (PST) Received: from mail-bw0-f53.google.com (mail-bw0-f53.google.com [209.85.214.53]) by cuda.sgi.com with ESMTP id nB5aCEEZXowVZPIs for ; Tue, 08 Nov 2011 00:42:04 -0800 (PST) Received: by mail-bw0-f53.google.com with SMTP id q10so218577bka.26 for ; Tue, 08 Nov 2011 00:42:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references; bh=0p5OhOzrVq2zcR2ZFm0GoxKMLcUOHcWS/rcGlfzb8A4=; b=Ziuttg1IEL2vaZ7iS8fUJXQtUgbEMU/y/xq1aIjniw6tv9oj+AOC7u7O/X5Qyltcph IWT71Yc/PfCnTXwNnPu61FRG3F+Fi2cg0hRcuJKiF/KCj0ptWojoxBVBYGBHFf30Mc/w kpvcrABUjsj2C2WDWzOzrXYnRCsT9m7Rd/Dws= Received: by 10.205.112.6 with SMTP id eq6mr22064988bkc.16.1320741724007; Tue, 08 Nov 2011 00:42:04 -0800 (PST) Received: from localhost.localdomain (swsoft-msk-nat.sw.ru. [195.214.232.10]) by mx.google.com with ESMTPS id fu17sm728823bkc.9.2011.11.08.00.42.02 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 08 Nov 2011 00:42:03 -0800 (PST) Sender: Dmitry Monakhov From: Dmitry Monakhov To: linux-fsdevel@vger.kernel.org, xfs@oss.sgi.com Cc: hch@lst.de, david@fromorbit.com, Dmitry Monakhov X-ASG-Orig-Subj: [PATCH 3/3] xfstests: remove duplicated umount $scratch_dev Subject: [PATCH 3/3] xfstests: remove duplicated umount $scratch_dev Date: Tue, 8 Nov 2011 12:41:55 +0400 Message-Id: <1320741715-32692-3-git-send-email-dmonakhov@openvz.org> X-Mailer: git-send-email 1.7.0.4 In-Reply-To: <1320741715-32692-1-git-send-email-dmonakhov@openvz.org> References: <1320741715-32692-1-git-send-email-dmonakhov@openvz.org> X-Barracuda-Connect: mail-bw0-f53.google.com[209.85.214.53] X-Barracuda-Start-Time: 1320741724 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79656 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean _require_scratch unmounts the SCRATCH_DEV. So it is not necessery to do it explicitly at the first time. Signed-off-by: Dmitry Monakhov --- 232 | 1 - 233 | 1 - 269 | 3 +-- 270 | 3 +-- 271 | 3 +-- 272 | 3 +-- 6 files changed, 4 insertions(+), 10 deletions(-) diff --git a/232 b/232 index fb4a872..2795da7 100755 --- a/232 +++ b/232 @@ -72,7 +72,6 @@ _require_scratch _require_quota _need_to_be_root -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs >> $seq.full 2>&1 _scratch_mount "-o usrquota,grpquota" chmod 777 $SCRATCH_MNT diff --git a/233 b/233 index e6ba7ef..28e6ac7 100755 --- a/233 +++ b/233 @@ -80,7 +80,6 @@ _require_quota _require_user _need_to_be_root -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs >> $seq.full 2>&1 _scratch_mount "-o usrquota,grpquota" chmod 777 $SCRATCH_MNT diff --git a/269 b/269 index 206492b..7e13ed9 100755 --- a/269 +++ b/269 @@ -64,10 +64,9 @@ _workout() # real QA test starts here _supported_fs generic _supported_os Linux -_require_scratch _need_to_be_root +_require_scratch -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 _scratch_mount diff --git a/270 b/270 index e45029f..6a349b4 100755 --- a/270 +++ b/270 @@ -74,12 +74,11 @@ _workout() # real QA test starts here _supported_fs generic _supported_os Linux -_require_scratch _require_quota _require_user _need_to_be_root +_require_scratch -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs_sized $((512 * 1024 * 1024)) >> $seq.full 2>&1 _scratch_mount "-o usrquota,grpquota" chmod 777 $SCRATCH_MNT diff --git a/271 b/271 index b4e6043..58a7f91 100755 --- a/271 +++ b/271 @@ -39,10 +39,9 @@ trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 # real QA test starts here _supported_fs ext4 _supported_os Linux -_require_scratch _need_to_be_root +_require_scratch -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs_sized $((128 * 1024 * 1024)) >> $seq.full 2>&1 # -onoload and EXT4_SYNC_FL on file is important becase result in diff --git a/272 b/272 index a35ef83..26dfa3b 100755 --- a/272 +++ b/272 @@ -83,10 +83,9 @@ chattr_opt: $chattr_opt" # real QA test starts here _supported_fs ext3 ext4 _supported_os Linux -_require_scratch _need_to_be_root +_require_scratch -umount $SCRATCH_DEV 2>/dev/null _scratch_mkfs_sized $((64 * 1024 * 1024)) >> $seq.full 2>&1 _scratch_mount -- 1.7.1 From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:48 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wlGk101954 for ; Tue, 8 Nov 2011 02:58:48 -0600 X-ASG-Debug-ID: 1320742726-0fe1032b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0DB231DBA5B9 for ; Tue, 8 Nov 2011 00:58:46 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id uPiPaXbu07iDfots for ; Tue, 08 Nov 2011 00:58:46 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW1-0007Yy-TI for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:45 +0000 Message-Id: <20111108085614.478431403@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:14 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 0/5] log all file size updates Subject: [PATCH 0/5] log all file size updates X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742727 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0209 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79658 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean With more reports showing up that the VFS writeback code is not able to write back our size updates dirtied from the I/O completion handler in reasonable time I think it's time to move to logging all file size updates ASAP, that is for 3.2 and maybe after a reasonable testing period even -stable. This series has been sent out a few times, and I've been doing QA on it for weeks. Note that I haven't implemented the log space reservations from ->writepage - see the actual patch for the rationale. From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:48 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,J_CHICKENPOX_74, J_CHICKENPOX_84 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wm4I101959 for ; Tue, 8 Nov 2011 02:58:48 -0600 X-ASG-Debug-ID: 1320742726-571100290000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0E41822AB61 for ; Tue, 8 Nov 2011 00:58:46 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id IN2OEbCCSJggAoti for ; Tue, 08 Nov 2011 00:58:46 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW2-0007a4-An for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:46 +0000 Message-Id: <20111108085846.291838677@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:16 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 2/5] xfs: use per-filesystem I/O completion workqueues Subject: [PATCH 2/5] xfs: use per-filesystem I/O completion workqueues References: <20111108085614.478431403@bombadil.infradead.org> Content-Disposition: inline; filename=xfs-split-workqueues X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742727 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79658 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The new concurrency managed workqueues are cheap enough that we can create per-filesystem instead of global workqueues. This allows us to remove the trylock or defer scheme on the ilock, which has the potential of delaying size updates, and is not helpful once we start to log the inode size. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 39 ++++++++++----------------------------- fs/xfs/xfs_aops.h | 2 -- fs/xfs/xfs_buf.c | 17 ----------------- fs/xfs/xfs_mount.h | 3 +++ fs/xfs/xfs_super.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 5 files changed, 55 insertions(+), 49 deletions(-) Index: linux-2.6/fs/xfs/xfs_aops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.c 2011-11-08 08:08:11.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_aops.c 2011-11-08 08:09:29.458887066 +0100 @@ -131,21 +131,15 @@ static inline bool xfs_ioend_is_append(s * will be the intended file size until i_size is updated. If this write does * not extend all the way to the valid file size then restrict this update to * the end of the write. - * - * This function does not block as blocking on the inode lock in IO completion - * can lead to IO completion order dependency deadlocks.. If it can't get the - * inode ilock it will return EAGAIN. Callers must handle this. */ -STATIC int +STATIC void xfs_setfilesize( - xfs_ioend_t *ioend) + struct xfs_ioend *ioend) { - xfs_inode_t *ip = XFS_I(ioend->io_inode); + struct xfs_inode *ip = XFS_I(ioend->io_inode); xfs_fsize_t isize; - if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)) - return EAGAIN; - + xfs_ilock(ip, XFS_ILOCK_EXCL); isize = xfs_ioend_new_eof(ioend); if (isize) { trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size); @@ -154,7 +148,6 @@ xfs_setfilesize( } xfs_iunlock(ip, XFS_ILOCK_EXCL); - return 0; } /* @@ -168,10 +161,12 @@ xfs_finish_ioend( struct xfs_ioend *ioend) { if (atomic_dec_and_test(&ioend->io_remaining)) { + struct xfs_mount *mp = XFS_I(ioend->io_inode)->i_mount; + if (ioend->io_type == IO_UNWRITTEN) - queue_work(xfsconvertd_workqueue, &ioend->io_work); + queue_work(mp->m_unwritten_workqueue, &ioend->io_work); else if (xfs_ioend_is_append(ioend)) - queue_work(xfsdatad_workqueue, &ioend->io_work); + queue_work(mp->m_data_workqueue, &ioend->io_work); else xfs_destroy_ioend(ioend); } @@ -212,23 +207,9 @@ xfs_end_io( * We might have to update the on-disk file size after extending * writes. */ - error = xfs_setfilesize(ioend); - ASSERT(!error || error == EAGAIN); - + xfs_setfilesize(ioend); done: - /* - * If we didn't complete processing of the ioend, requeue it to the - * tail of the workqueue for another attempt later. Otherwise destroy - * it. - */ - if (error == EAGAIN) { - atomic_inc(&ioend->io_remaining); - xfs_finish_ioend(ioend); - /* ensure we don't spin on blocked ioends */ - delay(1); - } else { - xfs_destroy_ioend(ioend); - } + xfs_destroy_ioend(ioend); } /* Index: linux-2.6/fs/xfs/xfs_aops.h =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.h 2011-11-08 08:02:50.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_aops.h 2011-11-08 08:08:55.915886285 +0100 @@ -18,8 +18,6 @@ #ifndef __XFS_AOPS_H__ #define __XFS_AOPS_H__ -extern struct workqueue_struct *xfsdatad_workqueue; -extern struct workqueue_struct *xfsconvertd_workqueue; extern mempool_t *xfs_ioend_pool; /* Index: linux-2.6/fs/xfs/xfs_buf.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_buf.c 2011-11-08 08:02:50.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_buf.c 2011-11-08 08:08:55.919886682 +0100 @@ -45,8 +45,6 @@ static kmem_zone_t *xfs_buf_zone; STATIC int xfsbufd(void *); static struct workqueue_struct *xfslogd_workqueue; -struct workqueue_struct *xfsdatad_workqueue; -struct workqueue_struct *xfsconvertd_workqueue; #ifdef XFS_BUF_LOCK_TRACKING # define XB_SET_OWNER(bp) ((bp)->b_last_holder = current->pid) @@ -1797,21 +1795,8 @@ xfs_buf_init(void) if (!xfslogd_workqueue) goto out_free_buf_zone; - xfsdatad_workqueue = alloc_workqueue("xfsdatad", WQ_MEM_RECLAIM, 1); - if (!xfsdatad_workqueue) - goto out_destroy_xfslogd_workqueue; - - xfsconvertd_workqueue = alloc_workqueue("xfsconvertd", - WQ_MEM_RECLAIM, 1); - if (!xfsconvertd_workqueue) - goto out_destroy_xfsdatad_workqueue; - return 0; - out_destroy_xfsdatad_workqueue: - destroy_workqueue(xfsdatad_workqueue); - out_destroy_xfslogd_workqueue: - destroy_workqueue(xfslogd_workqueue); out_free_buf_zone: kmem_zone_destroy(xfs_buf_zone); out: @@ -1821,8 +1806,6 @@ xfs_buf_init(void) void xfs_buf_terminate(void) { - destroy_workqueue(xfsconvertd_workqueue); - destroy_workqueue(xfsdatad_workqueue); destroy_workqueue(xfslogd_workqueue); kmem_zone_destroy(xfs_buf_zone); } Index: linux-2.6/fs/xfs/xfs_super.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_super.c 2011-11-08 08:02:50.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_super.c 2011-11-08 08:08:55.927886477 +0100 @@ -769,6 +769,40 @@ xfs_setup_devices( return 0; } +STATIC int +xfs_init_mount_workqueues( + struct xfs_mount *mp) +{ +#define XFS_WQ_NAME_LEN 512 + char name[XFS_WQ_NAME_LEN]; + + snprintf(name, XFS_WQ_NAME_LEN, "xfs-data/%s", mp->m_fsname); + mp->m_data_workqueue = alloc_workqueue(name, WQ_MEM_RECLAIM, 1); + if (!mp->m_data_workqueue) + goto out; + + snprintf(name, XFS_WQ_NAME_LEN, "xfs-conv/%s", mp->m_fsname); + mp->m_unwritten_workqueue = alloc_workqueue(name, WQ_MEM_RECLAIM, 1); + if (!mp->m_unwritten_workqueue) + goto out_destroy_data_iodone_queue; + + return 0; + +out_destroy_data_iodone_queue: + destroy_workqueue(mp->m_data_workqueue); +out: + return -ENOMEM; +#undef XFS_WQ_NAME_LEN +} + +STATIC void +xfs_destroy_mount_workqueues( + struct xfs_mount *mp) +{ + destroy_workqueue(mp->m_data_workqueue); + destroy_workqueue(mp->m_unwritten_workqueue); +} + /* Catch misguided souls that try to use this interface on XFS */ STATIC struct inode * xfs_fs_alloc_inode( @@ -1020,6 +1054,7 @@ xfs_fs_put_super( xfs_unmountfs(mp); xfs_freesb(mp); xfs_icsb_destroy_counters(mp); + xfs_destroy_mount_workqueues(mp); xfs_close_devices(mp); xfs_free_fsname(mp); kfree(mp); @@ -1353,10 +1388,14 @@ xfs_fs_fill_super( if (error) goto out_free_fsname; - error = xfs_icsb_init_counters(mp); + error = xfs_init_mount_workqueues(mp); if (error) goto out_close_devices; + error = xfs_icsb_init_counters(mp); + if (error) + goto out_destroy_workqueues; + error = xfs_readsb(mp, flags); if (error) goto out_destroy_counters; @@ -1419,6 +1458,8 @@ xfs_fs_fill_super( xfs_freesb(mp); out_destroy_counters: xfs_icsb_destroy_counters(mp); +out_destroy_workqueues: + xfs_destroy_mount_workqueues(mp); out_close_devices: xfs_close_devices(mp); out_free_fsname: Index: linux-2.6/fs/xfs/xfs_mount.h =================================================================== --- linux-2.6.orig/fs/xfs/xfs_mount.h 2011-11-08 08:02:50.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_mount.h 2011-11-08 08:08:55.931904609 +0100 @@ -211,6 +211,9 @@ typedef struct xfs_mount { struct shrinker m_inode_shrink; /* inode reclaim shrinker */ int64_t m_low_space[XFS_LOWSP_MAX]; /* low free space thresholds */ + + struct workqueue_struct *m_data_workqueue; + struct workqueue_struct *m_unwritten_workqueue; } xfs_mount_t; /* From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:48 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wmsS101956 for ; Tue, 8 Nov 2011 02:58:48 -0600 X-ASG-Debug-ID: 1320742726-44e402540000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4C68C1DBA5BE for ; Tue, 8 Nov 2011 00:58:47 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id ec32fMQcq21h4Xef for ; Tue, 08 Nov 2011 00:58:47 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW2-0007b6-Me for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:46 +0000 Message-Id: <20111108085846.654476311@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:18 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 4/5] xfs: log file size updates as part of unwritten extent conversion Subject: [PATCH 4/5] xfs: log file size updates as part of unwritten extent conversion References: <20111108085614.478431403@bombadil.infradead.org> Content-Disposition: inline; filename=xfs-piggy-back-unwritten-size-updates X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742727 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79658 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean If we convert and unwritten extent past the current i_size log the size update as part of the extent manipulation transactions instead of doing an unlogged metadata update later. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 11 ++++++----- fs/xfs/xfs_iomap.c | 19 ++++++++++++++++++- 2 files changed, 24 insertions(+), 6 deletions(-) Index: linux-2.6/fs/xfs/xfs_iomap.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_iomap.c 2011-11-08 08:02:50.234386118 +0100 +++ linux-2.6/fs/xfs/xfs_iomap.c 2011-11-08 08:14:04.319888994 +0100 @@ -31,6 +31,7 @@ #include "xfs_ialloc_btree.h" #include "xfs_dinode.h" #include "xfs_inode.h" +#include "xfs_inode_item.h" #include "xfs_btree.h" #include "xfs_bmap.h" #include "xfs_rtalloc.h" @@ -645,6 +646,7 @@ xfs_iomap_write_unwritten( xfs_trans_t *tp; xfs_bmbt_irec_t imap; xfs_bmap_free_t free_list; + xfs_fsize_t i_size; uint resblks; int committed; int error; @@ -705,7 +707,22 @@ xfs_iomap_write_unwritten( if (error) goto error_on_bmapi_transaction; - error = xfs_bmap_finish(&(tp), &(free_list), &committed); + /* + * Log the updated inode size as we go. We have to be careful + * to only log it up to the actual write offset if it is + * halfway into a block. + */ + i_size = XFS_FSB_TO_B(mp, offset_fsb + count_fsb); + if (i_size > offset + count) + i_size = offset + count; + + i_size = xfs_new_eof(ip, i_size); + if (i_size) { + ip->i_d.di_size = i_size; + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + } + + error = xfs_bmap_finish(&tp, &free_list, &committed); if (error) goto error_on_bmapi_transaction; Index: linux-2.6/fs/xfs/xfs_aops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.c 2011-11-08 08:12:31.586400976 +0100 +++ linux-2.6/fs/xfs/xfs_aops.c 2011-11-08 08:14:04.319888994 +0100 @@ -183,13 +183,14 @@ xfs_end_io( ioend->io_error = -error; goto done; } + } else { + /* + * We might have to update the on-disk file size after + * extending writes. + */ + xfs_setfilesize(ioend); } - /* - * We might have to update the on-disk file size after extending - * writes. - */ - xfs_setfilesize(ioend); done: xfs_destroy_ioend(ioend); } From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:48 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,J_CHICKENPOX_35 autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wmNq101957 for ; Tue, 8 Nov 2011 02:58:48 -0600 X-ASG-Debug-ID: 1320742726-0bcc03370000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4DDF916243B2 for ; Tue, 8 Nov 2011 00:58:47 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id 4dmDGOh3cWXUjkCh for ; Tue, 08 Nov 2011 00:58:47 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW2-0007aa-GR for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:46 +0000 Message-Id: <20111108085846.466994570@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:17 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 3/5] xfs: do not require an ioend for new EOF calculation Subject: [PATCH 3/5] xfs: do not require an ioend for new EOF calculation References: <20111108085614.478431403@bombadil.infradead.org> Content-Disposition: inline; filename=xfs-simplify-eof-calculation X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742727 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79657 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Replace xfs_ioend_new_eof with a new inline xfs_new_eof helper that doesn't require and ioend, and is available also outside of xfs_aops.c. Also make the code a bit more clear by using a normal if statement instead of a slightly misleading MIN(). Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 26 +++++--------------------- fs/xfs/xfs_inode.h | 14 ++++++++++++++ 2 files changed, 19 insertions(+), 21 deletions(-) Index: linux-2.6/fs/xfs/xfs_aops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.c 2011-11-08 08:11:44.891887054 +0100 +++ linux-2.6/fs/xfs/xfs_aops.c 2011-11-08 08:12:31.586400976 +0100 @@ -99,24 +99,6 @@ xfs_destroy_ioend( } /* - * If the end of the current ioend is beyond the current EOF, - * return the new EOF value, otherwise zero. - */ -STATIC xfs_fsize_t -xfs_ioend_new_eof( - xfs_ioend_t *ioend) -{ - xfs_inode_t *ip = XFS_I(ioend->io_inode); - xfs_fsize_t isize; - xfs_fsize_t bsize; - - bsize = ioend->io_offset + ioend->io_size; - isize = MAX(ip->i_size, ip->i_new_size); - isize = MIN(isize, bsize); - return isize > ip->i_d.di_size ? isize : 0; -} - -/* * Fast and loose check if this write could update the on-disk inode size. */ static inline bool xfs_ioend_is_append(struct xfs_ioend *ioend) @@ -140,7 +122,7 @@ xfs_setfilesize( xfs_fsize_t isize; xfs_ilock(ip, XFS_ILOCK_EXCL); - isize = xfs_ioend_new_eof(ioend); + isize = xfs_new_eof(ip, ioend->io_offset + ioend->io_size); if (isize) { trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size); ip->i_d.di_size = isize; @@ -362,6 +344,8 @@ xfs_submit_ioend_bio( xfs_ioend_t *ioend, struct bio *bio) { + struct xfs_inode *ip = XFS_I(ioend->io_inode); + atomic_inc(&ioend->io_remaining); bio->bi_private = ioend; bio->bi_end_io = xfs_end_bio; @@ -370,8 +354,8 @@ xfs_submit_ioend_bio( * If the I/O is beyond EOF we mark the inode dirty immediately * but don't update the inode size until I/O completion. */ - if (xfs_ioend_new_eof(ioend)) - xfs_mark_inode_dirty(XFS_I(ioend->io_inode)); + if (xfs_new_eof(ip, ioend->io_offset + ioend->io_size)) + xfs_mark_inode_dirty(ip); submit_bio(wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE, bio); } Index: linux-2.6/fs/xfs/xfs_inode.h =================================================================== --- linux-2.6.orig/fs/xfs/xfs_inode.h 2011-11-08 08:02:50.000000000 +0100 +++ linux-2.6/fs/xfs/xfs_inode.h 2011-11-08 08:13:01.290386996 +0100 @@ -278,6 +278,20 @@ static inline struct inode *VFS_I(struct } /* + * If this I/O goes past the on-disk inode size update it unless it would + * be past the current in-core or write in-progress inode size. + */ +static inline xfs_fsize_t +xfs_new_eof(struct xfs_inode *ip, xfs_fsize_t new_size) +{ + xfs_fsize_t i_size = max(ip->i_size, ip->i_new_size); + + if (new_size > i_size) + new_size = i_size; + return new_size > ip->i_d.di_size ? new_size : 0; +} + +/* * i_flags helper functions */ static inline void From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:49 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wm4h101980 for ; Tue, 8 Nov 2011 02:58:49 -0600 X-ASG-Debug-ID: 1320742727-0bc403560000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CF46716243B3 for ; Tue, 8 Nov 2011 00:58:47 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id FI7SynxiKTLDSCeh for ; Tue, 08 Nov 2011 00:58:47 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW2-0007bc-T2 for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:46 +0000 Message-Id: <20111108085846.850882616@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:19 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 5/5] xfs: log file size updates at I/O completion time Subject: [PATCH 5/5] xfs: log file size updates at I/O completion time References: <20111108085614.478431403@bombadil.infradead.org> Content-Disposition: inline; filename=xfs-log-file-size-updates X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742727 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79657 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Do not use unlogged metadata updates and the VFS dirty bit for updating the file size after writeback. In addition to causing various problems with updates getting delayed for far too log this also drags in the unscalable VFS dirty tracking, and is one of the few remaining unlogged metadata updates. Note that we allocate a new transaction from the I/O completion handler. While this sounds fairly dangerous it isn't an issue in practice given that any appending write alreay had to start a transaction in writepages to allocate blocks, and we'll start throtteling there if we run low on log space or memory. We could still occasionally stall in the completion handler, but given that we have per-filesystems workqueues for the I/O completions, and completions that do not have to either convert unwritten extents or update the file size are processed from interrupt context we do not have to worry about this stalling a system to death. In addition to that implementing log reservations from ->writepage that are only released by a different thread requires a lot of work, and even with that wasn't quite doable in a deadlock-free manner. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 49 ++++++++++++++++++++++++++++--------------------- fs/xfs/xfs_file.c | 36 +++++++++++++++++++++++++++++++++++- 2 files changed, 63 insertions(+), 22 deletions(-) Index: linux-2.6/fs/xfs/xfs_aops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.c 2011-11-07 12:19:32.935315100 +0100 +++ linux-2.6/fs/xfs/xfs_aops.c 2011-11-07 12:19:35.625815654 +0100 @@ -26,6 +26,7 @@ #include "xfs_bmap_btree.h" #include "xfs_dinode.h" #include "xfs_inode.h" +#include "xfs_inode_item.h" #include "xfs_alloc.h" #include "xfs_error.h" #include "xfs_rw.h" @@ -114,22 +115,39 @@ static inline bool xfs_ioend_is_append(s * not extend all the way to the valid file size then restrict this update to * the end of the write. */ -STATIC void +STATIC int xfs_setfilesize( struct xfs_ioend *ioend) { struct xfs_inode *ip = XFS_I(ioend->io_inode); + struct xfs_mount *mp = ip->i_mount; + struct xfs_trans *tp; xfs_fsize_t isize; + int error = 0; xfs_ilock(ip, XFS_ILOCK_EXCL); isize = xfs_new_eof(ip, ioend->io_offset + ioend->io_size); - if (isize) { - trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size); - ip->i_d.di_size = isize; - xfs_mark_inode_dirty(ip); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + + if (!isize) + return 0; + + trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size); + + tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS); + error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0); + if (error) { + xfs_trans_cancel(tp, 0); + return error; } - xfs_iunlock(ip, XFS_ILOCK_EXCL); + xfs_ilock(ip, XFS_ILOCK_EXCL); + + ip->i_d.di_size = isize; + xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + + return xfs_trans_commit(tp, 0); } /* @@ -183,12 +201,10 @@ xfs_end_io( ioend->io_error = -error; goto done; } - } else { - /* - * We might have to update the on-disk file size after - * extending writes. - */ - xfs_setfilesize(ioend); + } else if (xfs_ioend_is_append(ioend)) { + error = xfs_setfilesize(ioend); + if (error) + ioend->io_error = error; } done: @@ -345,19 +361,10 @@ xfs_submit_ioend_bio( xfs_ioend_t *ioend, struct bio *bio) { - struct xfs_inode *ip = XFS_I(ioend->io_inode); - atomic_inc(&ioend->io_remaining); bio->bi_private = ioend; bio->bi_end_io = xfs_end_bio; - /* - * If the I/O is beyond EOF we mark the inode dirty immediately - * but don't update the inode size until I/O completion. - */ - if (xfs_new_eof(ip, ioend->io_offset + ioend->io_size)) - xfs_mark_inode_dirty(ip); - submit_bio(wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE, bio); } Index: linux-2.6/fs/xfs/xfs_file.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_file.c 2011-11-07 12:19:20.490314774 +0100 +++ linux-2.6/fs/xfs/xfs_file.c 2011-11-07 12:58:28.834315624 +0100 @@ -436,6 +436,36 @@ xfs_aio_write_isize_update( } } +STATIC int +xfs_aio_write_isize_reset( + struct xfs_inode *ip) +{ + struct xfs_mount *mp = ip->i_mount; + struct xfs_trans *tp; + int error = 0; + + tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS); + error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0); + if (error) { + xfs_trans_cancel(tp, 0); + return error; + } + + xfs_ilock(ip, XFS_ILOCK_EXCL); + + if (ip->i_d.di_size <= ip->i_size) { + xfs_iunlock(ip, XFS_ILOCK_EXCL); + xfs_trans_cancel(tp, 0); + return 0; + } + + ip->i_d.di_size = ip->i_size; + xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + + return xfs_trans_commit(tp, 0); +} + /* * If this was a direct or synchronous I/O that failed (such as ENOSPC) then * part of the I/O may have been written to disk before the error occurred. In @@ -447,14 +477,18 @@ xfs_aio_write_newsize_update( struct xfs_inode *ip, xfs_fsize_t new_size) { + bool reset = false; if (new_size == ip->i_new_size) { xfs_rw_ilock(ip, XFS_ILOCK_EXCL); if (new_size == ip->i_new_size) ip->i_new_size = 0; if (ip->i_d.di_size > ip->i_size) - ip->i_d.di_size = ip->i_size; + reset = true; xfs_rw_iunlock(ip, XFS_ILOCK_EXCL); } + + if (reset) + xfs_aio_write_isize_reset(ip); } /* From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 02:58:50 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA88wnuW101994 for ; Tue, 8 Nov 2011 02:58:50 -0600 X-ASG-Debug-ID: 1320742726-0bc9035e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id F1AE116243AF for ; Tue, 8 Nov 2011 00:58:46 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id jEUeUFcIqkOfzW9J for ; Tue, 08 Nov 2011 00:58:46 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNhW2-0007ZY-4P for xfs@oss.sgi.com; Tue, 08 Nov 2011 08:58:46 +0000 Message-Id: <20111108085846.083679026@bombadil.infradead.org> User-Agent: quilt/0.48-1 Date: Tue, 08 Nov 2011 03:56:15 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 1/5] fix: force shutdown handling in xfs_end_io Subject: [PATCH 1/5] fix: force shutdown handling in xfs_end_io References: <20111108085614.478431403@bombadil.infradead.org> Content-Disposition: inline; filename=xfs-fix-end-io-error-handling X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320742726 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79657 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Only ioend->io_error gets propagated back to e.g. AIO completions. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: linux-2.6/fs/xfs/xfs_aops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_aops.c 2011-11-02 09:00:45.569119028 +0100 +++ linux-2.6/fs/xfs/xfs_aops.c 2011-11-07 12:19:22.282316090 +0100 @@ -189,7 +189,7 @@ xfs_end_io( int error = 0; if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { - error = -EIO; + ioend->io_error = -EIO; goto done; } if (ioend->io_error) From BATV+24cb1aa6f689616f0478+2998+infradead.org+hch@bombadil.srs.infradead.org Tue Nov 8 08:38:55 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA8EctK8125373 for ; Tue, 8 Nov 2011 08:38:55 -0600 X-ASG-Debug-ID: 1320763134-256b03420000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CE6B71F31541; Tue, 8 Nov 2011 06:38:54 -0800 (PST) Received: from bombadil.infradead.org (173-166-109-252-newengland.hfc.comcastbusiness.net [173.166.109.252]) by cuda.sgi.com with ESMTP id X6pKMJcPxO7d8UzW; Tue, 08 Nov 2011 06:38:54 -0800 (PST) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1RNmpA-0000u2-Iy; Tue, 08 Nov 2011 14:38:52 +0000 Date: Tue, 8 Nov 2011 09:38:52 -0500 From: Christoph Hellwig To: Alex Elder Cc: XFS Mailing List , Ben Hutchings , Carlos Maiolino X-ASG-Orig-Subj: Re: [PATCH, updated] xfs: Fix possible memory corruption in xfs_readlink Subject: Re: [PATCH, updated] xfs: Fix possible memory corruption in xfs_readlink Message-ID: <20111108143852.GA2486@infradead.org> References: <1320156842.30281.28.camel@deadeye> <1320682224.3186.5.camel@doink> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1320682224.3186.5.camel@doink> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: 173-166-109-252-newengland.hfc.comcastbusiness.net[173.166.109.252] X-Barracuda-Start-Time: 1320763134 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.32 X-Barracuda-Spam-Status: No, SCORE=-1.32 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=COMMA_SUBJECT, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79680 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 COMMA_SUBJECT Subject is like 'Re: FDSDS, this is a subject' 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good, Reviewed-by: Christoph Hellwig From sales@homegames.co.uk Tue Nov 8 09:49:15 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: ** X-Spam-Status: No, score=2.5 required=5.0 tests=BAYES_50,URIBL_BLACK autolearn=no version=3.4.0-r929098 Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA8FnF4X129747 for ; Tue, 8 Nov 2011 09:49:15 -0600 X-ASG-Debug-ID: 1320767346-2fe400aa0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp-out-59.livemail.co.uk (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 878951F3176D; Tue, 8 Nov 2011 07:49:06 -0800 (PST) Received: from smtp-out-59.livemail.co.uk (mail213-171-216-216.livemail.co.uk [213.171.216.216]) by cuda.sgi.com with ESMTP id ytUdhC0wJsy5JRtN; Tue, 08 Nov 2011 07:49:06 -0800 (PST) Received: by smtp-out-59.livemail.co.uk (Postfix, from userid 1000) id 4B0F716C1A; Tue, 8 Nov 2011 15:49:05 +0000 (GMT) Received: from IIS690.xtreme.support.co.uk (server213-171-218-90.livedns.org.uk [213.171.218.90]) by smtp-out-59.livemail.co.uk (Postfix) with ESMTP id 261DE16C20; Tue, 8 Nov 2011 15:49:03 +0000 (GMT) Received: from mail pickup service by IIS690.xtreme.support.co.uk with Microsoft SMTPSVC; Tue, 8 Nov 2011 15:49:02 +0000 Reply-To: , From: To: <271975erasmusadele@gmail.com>, , , , , , , , , , , , , , <17667barrett.mckool@gmail.com>, , , , <5seowebgek@gmail.com>, , , , , , , , <102031rgoginen@gmail.com>, , , , , , , , , , , , , , , , , , , <1302524879-4737-1-git-send-email-namhyung@gmail.com>, , <1302534596-31250-1-git-send-email-wanlong.gao@gmail.com>, , , , , , , , , , , , , , , , , , , , , , <4da289e1.3090709@gmail.com>, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , <240145@gmail.com>, , , , , , , , , , , , , <142259rrexecjobs@gmail.com>, , , , , , , , , , , , , , , <4vastation@gmail.com>, , , , , , , , , , , <1111neerjt@gmail.com>, , , <212259kittie.edwards@gmail.com>, , , , , , <8263airmicronesia@gmail.com>, , , , , , , , , , , , , , , , , , , , , , <2chainzcasting@gmail.com>, , , , , , , , , , , , , , <136280mfymadari@gmail.com>, , <20218prakashjob123@gmail.com>, <201104111641.16852.lisi.reisz@gmail.com>, <155343bio.taiga@gmail.com>, , , , , , , , , , <19pmcwhjcg@mail.gmail.com>, , , , , , , , , , , , , , , , , , , , , , , , , , <48836rrexecjobs@gmail.com>, , , <132daveblinux@gmail.com>, , , , , , , <9434187imrajib11@gmail.com>, <422892doddytop@gmail.com>, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , <3annefi7@gmail.com>, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , X-ASG-Orig-Subj: Buy Custom Made University Diplomas, Fake College Degrees and Transcripts of World Famous Universities. recommends this site Subject: ***** SUSPECTED SPAM ***** Buy Custom Made University Diplomas, Fake College Degrees and Transcripts of World Famous Universities. recommends this site Date: Tue, 8 Nov 2011 15:49:01 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4862 Message-ID: X-OriginalArrivalTime: 08 Nov 2011 15:49:02.0942 (UTC) FILETIME=[F05067E0:01CC9E2D] X-Barracuda-Connect: mail213-171-216-216.livemail.co.uk[213.171.216.216] X-Barracuda-Start-Time: 1320767348 X-Barracuda-Bayes: INNOCENT GLOBAL 0.4463 1.0000 0.0000 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 2.00 X-Barracuda-Spam-Status: Yes, SCORE=2.00 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC5_MJ1963, HEAD_LONG, NO_REAL_NAME, RDNS_DYNAMIC, SUBJ_BUY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79684 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 NO_REAL_NAME From: does not include a real name 0.90 SUBJ_BUY Subject line starts with Buy or Buying 0.50 HEAD_LONG Message headers are very long 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 X-Priority: 5 (Lowest) X-MSMail-Priority: Low Importance: Low X-Barracuda-Spam-Flag: YES X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Your friend Buy Custom Made University Diplomas, Fake College Degrees = and Transcripts of World Famous Universities. (janebrow1@​aol.com) = has recommended this site to you, and sends you the following message: *************************************************************************= *********************************************** Buy fake/novelty university diplomas and degrees online. We design our Degree or Diploma Certificates and Transcripts to look = 99.99% identical to world famous originals including identical security = grade transcript paper, identical water markings, raised-ink crests, = embossed seals, correct card stock weight (60-100lb depending upon = institution specifications). Most post-secondary institutions available. ---> http://rurls.ru/31s <--- *************************************************************************= *********************************************** http://www.darts-online.co.uk/proddetail.asp?prod=3D1045-25 From bpm@sgi.com Tue Nov 8 10:39:18 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA8GdITU132787 for ; Tue, 8 Nov 2011 10:39:18 -0600 Received: from whiskey.americas.sgi.com (whiskey.americas.sgi.com [128.162.233.19]) by relay1.corp.sgi.com (Postfix) with ESMTP id 682118F8033 for ; Tue, 8 Nov 2011 08:39:15 -0800 (PST) Received: by whiskey.americas.sgi.com (Postfix, from userid 4600) id 3FCBB431F74; Tue, 8 Nov 2011 10:39:32 -0600 (CST) Date: Tue, 8 Nov 2011 10:39:32 -0600 From: Ben Myers To: xfs@oss.sgi.com Subject: [BUG] assert in xfs_imap_to_bp, line 185 Message-ID: <20111108163932.GA22095@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi XFS Folks, I am able to hit this assert consistently when running test 111 on recent (commit 9e4c109a) oss xfs bits: [ 1930.808669] XFS (sdb4): Corruption detected. Unmount and run xfs_repair [ 1930.808676] XFS (sdb4): bad inode magic/vsn daddr 64 #8 (magic=5858) [ 1930.815037] XFS: Assertion failed: 0, file: /root/xfs/fs/xfs/xfs_inode.c, line: 185 [ 1930.822727] ------------[ cut here ]------------ [ 1930.826688] kernel BUG at /root/xfs/fs/xfs/xfs_message.c:101! [ 1930.826688] invalid opcode: 0000 [#1] SMP [ 1930.826688] CPU 2 [ 1930.826688] Modules linked in: xfs exportfs af_packet microcode fuse loop dm_mod e1000 shpchp iTCO_wdt tpm_tis tpm sg sr_mod pci_hotplug cdrom serio_raw iTCO_vendor_support floppy intel_rng tpm_bios e752x_e dac edac_core pcspkr i2c_i801 i2c_core container button uhci_hcd ehci_hcd usbcore sd_mod crc_t10dif edd ext3 mbcache jbd fan processor ide_pci_generic piix ide_core ata_generic ata_piix libata aic79xx scsi_tra nsport_spi scsi_mod thermal thermal_sys hwmon [ 1930.826688] [ 1930.826688] Pid: 1894, comm: mount Not tainted 3.1.0-rc9-0.7-default+ #2 Supermicro X6DHR-8G/X6DHR-8GS/X6DHR-8G/X6DHR-8GS [ 1930.826688] RIP: 0010:[] [] assfail+0x1d/0x30 [xfs] [ 1930.826688] RSP: 0018:ffff8801fd157b38 EFLAGS: 00010292 [ 1930.826688] RAX: 000000000000005d RBX: 0000000000000008 RCX: 00000000000016a1 [ 1930.826688] RDX: 000000000000b4a9 RSI: 0000000000000046 RDI: 0000000000000246 [ 1930.826688] RBP: ffff8801fd157b38 R08: 0000000000000005 R09: 0000000000000000 [ 1930.826688] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88021ed0a800 [ 1930.826688] R13: 0000000000000020 R14: ffff88021f4ca800 R15: 0000000000000000 [ 1930.826688] FS: 00007fe83ed507c0(0000) GS:ffff88022fd00000(0000) knlGS:0000000000000000 [ 1930.826688] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 1930.826688] CR2: 00007f4bc6f92bd0 CR3: 00000001fd7ff000 CR4: 00000000000006e0 [ 1930.826688] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 1930.826688] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 1930.826688] Process mount (pid: 1894, threadinfo ffff8801fd156000, task ffff88020e10e200) [ 1930.826688] Stack: [ 1930.826688] ffff8801fd157bb8 ffffffffa032d0a4 ffffffffa032fb7c ffff88020ec4ce00 [ 1930.826688] 0000000000000000 ffff8801fd157bd8 ffff8802027cb020 0000000000000000 [ 1930.826688] ffff88021f4caa20 00000000000002d0 ffff88020ec4ce00 0000000000000000 [ 1930.826688] Call Trace: [ 1930.826688] [] xfs_imap_to_bp+0x154/0x260 [xfs] [ 1930.826688] [] ? xfs_iread+0x7c/0x220 [xfs] [ 1930.826688] [] xfs_iread+0x7c/0x220 [xfs] [ 1930.826688] [] xfs_iget+0x1d0/0x5a0 [xfs] [ 1930.826688] [] xfs_mountfs+0x37a/0x670 [xfs] [ 1930.826688] [] xfs_fs_fill_super+0x1bf/0x270 [xfs] [ 1930.826688] [] mount_bdev+0x17e/0x200 [ 1930.826688] [] ? xfs_parseargs+0xbd0/0xbd0 [xfs] [ 1930.826688] [] xfs_fs_mount+0x10/0x20 [xfs] [ 1930.826688] [] mount_fs+0x48/0x190 [ 1930.826688] [] vfs_kern_mount+0x65/0xc0 [ 1931.070470] [] do_kern_mount+0x4e/0x100 [ 1931.070470] [] do_mount+0x3c3/0x810 [ 1931.070470] [] ? strndup_user+0x5b/0x70 [ 1931.070470] [] sys_mount+0xb3/0xe0 [ 1931.070470] [] system_call_fastpath+0x16/0x1b [ 1931.070470] Code: 00 00 00 48 89 45 c8 e8 72 fc ff ff c9 c3 55 41 89 d0 48 89 f1 48 89 fa 48 c7 c6 68 c6 35 a0 31 ff 48 89 e5 31 c0 e8 93 ff ff ff <0f> 0b eb fe 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 4c [ 1931.070470] RIP [] assfail+0x1d/0x30 [xfs] [ 1931.070470] RSP Does this ring a bell for anyone? I should mention that this is a machine recently set up for this purpose, so there isn't a history of it working correctly before... I will go back a release or two and try to reproduce this. THanks, Ben From dave@twin.jikos.cz Tue Nov 8 11:02:08 2011 X-Spam-Checker-Version: SpamAssassin 3.4.0-r929098 (2010-03-30) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.4.0-r929098 Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id pA8H27bR134270 for ; Tue, 8 Nov 2011 11:02:08 -0600 X-ASG-Debug-ID: 1320771724-67bc00cb0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from twin.jikos.cz (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 83AB41036608; Tue, 8 Nov 2011 09:02:05 -0800 (PST) Received: from twin.jikos.cz (twin.jikos.cz [89.185.236.188]) by cuda.sgi.com with ESMTP id xlIE9hqFId3m7xxP; Tue, 08 Nov 2011 09:02:05 -0800 (PST) Received: from twin.jikos.cz (dave@localhost [127.0.0.1]) by twin.jikos.cz (8.13.6/8.13.6) with ESMTP id pA8H1rjD024591 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 8 Nov 2011 18:01:53 +0100 Received: (from dave@localhost) by twin.jikos.cz (8.13.6/8.13.6/Submit) id pA8H1m0v024589; Tue, 8 Nov 2011 18:01:48 +0100 Date: Tue, 8 Nov 2011 18:01:48 +0100 From: David Sterba To: Dmitry Monakhov Cc: dave@jikos.cz, linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Subject: Re: [PATCH 6/9] xfstests: add a new test that runs fsstress under ENOSPC conditions Message-ID: <20111108170146.GR12759@twin.jikos.cz> Reply-To: dave@jikos.cz Mail-Followup-To: Dmitry Monakhov , dave@jikos.cz, linux-fsdevel@vger.kernel.org, aelder@sgi.com, tytso@mit.edu, hch@lst.de, xfs@oss.sgi.com References: <1320342953-7973-1-git-send-email-dmonakhov@openvz.org> <1320342953-7973-6-git-send-email-dmonakhov@openvz.org> <20111107132533.GP12759@twin.jikos.cz> <87sjm0kqjk.fsf@dmbot.sw.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87sjm0kqjk.fsf@dmbot.sw.ru> User-Agent: Mutt/1.4.2.2i X-Barracuda-Connect: twin.jikos.cz[89.185.236.188] X-Barracuda-Start-Time: 1320771726 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.79689 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milt