From owner-xfs@oss.sgi.com Fri Jun 1 10:06:27 2007 Received: with ECARTIS (v1.0.0; list xfs); Fri, 01 Jun 2007 10:06:32 -0700 (PDT) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.227]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l51H6QWt007883 for ; Fri, 1 Jun 2007 10:06:27 -0700 Received: by wr-out-0506.google.com with SMTP id i22so545183wra for ; Fri, 01 Jun 2007 10:06:26 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:subject:from:to:cc:content-type:date:message-id:mime-version:x-mailer; b=pz8R51Z6i4fOJfGtBhElNE7doQo34Mxu8YBbcj/g6nnTqdxmT2b6Ndl4o9hQ4xs4yCfUqOp0MVBiDsaJv6BZoDeH7Hb7a03ss/hI++nEQdiUbMd8u0bby88XUFTQcbzMsD4N6Dnne8xCpeoMkR2ptMYu4n4ZeleqX+zPb0OvHIE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:subject:from:to:cc:content-type:date:message-id:mime-version:x-mailer; b=RS+qe2Re2iokfmxCG49fuFKhW4o2bj9huEjL2J+HJ47vaj2ba1sFNKlxK5LhpliE7KlNvTJK/JkF0RsNc4HA7LCWG2pQNm28TWS4nIGGqteHerkn3Xoi/wId323QAmk+FceW06rDxk2/qw9hzACnW4dF0KQCHlwF022nOvayuUw= Received: by 10.78.193.19 with SMTP id q19mr1313567huf.1180715979445; Fri, 01 Jun 2007 09:39:39 -0700 (PDT) Received: from ?192.168.1.55? ( [84.59.122.193]) by mx.google.com with ESMTP id f7sm484088nfh.2007.06.01.09.39.36; Fri, 01 Jun 2007 09:39:36 -0700 (PDT) Subject: XFS shrink functionality From: Ruben Porras To: xfs@oss.sgi.com Cc: iusty@k1024.org, cw@f00f.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-cNVvu74QiAC041q1y3BN" Date: Fri, 01 Jun 2007 18:39:34 +0200 Message-Id: <1180715974.10796.46.camel@localhost> Mime-Version: 1.0 X-Mailer: Evolution 2.10.1 X-archive-position: 11572 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: nahoo82@gmail.com Precedence: bulk X-list: xfs --=-cNVvu74QiAC041q1y3BN Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hello,=20 I'm investigating the possibility to write myself the necessary code to shrink an xfs filesystem (I'd be able to dedicate a day/week). Trying to know if something is already done I came across the mails of a previous intent [0], [1] (I'm cc'ing the people involved).=20 At a first glance the patch is a little outdated and will no more apply (as of linux 2.16.18, which is the last customised kernel that I was able to run under a XEN environment), because at least the function xfs_fs_geometry is changed.=20 I'm really curious about what happened to this patches and why they were discontinued. The second part never was made public, and there was also no answer. Was there any flaw in any of the posted code or anything in XFS that makes it especially hard to shrink [3] that discouraged the development? After that, the first questions that arouse are, would there be some assistance/groove in from the developers?=20 How doable is it? What are the programmers requirements from your point of view? Thank you. [1] http://oss.sgi.com/archives/xfs/2005-08/msg00142.html [2] http://oss.sgi.com/archives/xfs/2005-09/msg00038.html [3] the only limitation that I might think of is not being able to shrink past the internal journal. --=-cNVvu74QiAC041q1y3BN Content-Type: application/pgp-signature; name=signature.asc Content-Description: Dies ist ein digital signierter Nachrichtenteil -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQBGYEvGYubrKblAx+oRAhjZAJ4terI47D96a4JX8wsyge5iA/f6nQCgkTeN 3qlB2vKeB7015poCrDexuWQ= =7rZq -----END PGP SIGNATURE----- --=-cNVvu74QiAC041q1y3BN-- From owner-xfs@oss.sgi.com Fri Jun 1 18:14:08 2007 Received: with ECARTIS (v1.0.0; list xfs); Fri, 01 Jun 2007 18:14:11 -0700 (PDT) Received: from silver.tritoncore.com (silver.tritoncore.com [209.59.142.74]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l521E6Wt027310 for ; Fri, 1 Jun 2007 18:14:08 -0700 Received: from powersle by silver.tritoncore.com with local (Exim 4.63) (envelope-from ) id 1HtotX-00045K-VP for xfs@oss.sgi.com; Thu, 31 May 2007 13:57:08 -0400 To: xfs@oss.sgi.com Subject: Vacancy! Vacancy!! Vacancy!!! From: "Kenneth Fabrics Ltd." Reply-To: kennethfabricsltdd@mail2recruiter.com MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit Message-Id: Date: Thu, 31 May 2007 13:57:07 -0400 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - silver.tritoncore.com X-AntiAbuse: Original Domain - oss.sgi.com X-AntiAbuse: Originator/Caller UID/GID - [32761 32002] / [47 12] X-AntiAbuse: Sender Address Domain - silver.tritoncore.com X-Source: X-Source-Args: X-Source-Dir: X-archive-position: 11573 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: employment.dept@kennethfabricsltd.com Precedence: bulk X-list: xfs From The Desk Of The: Recruitment Manager Mr Kenneth Holley Kenneth Fabrics Limited. Dear , KENNETH FABRICS LTD.Is committed to global citizenship by operating in a responsible and sustainable manner around the globe. As part of our Multi Level Marketing scheme, we need capable hands to act as representative/book keeper in the United Kingdom and Canada on the company’s behalf. Kenneth Fabrics Ltd.. Is a new Store under KENNETH FARICS in India? We are into supplies of Raw Materials. We are ranked No.1 among India private enterprises with annual production capacity exceeding 1 million units sold everywhere in India and exported to all over the world including UK, Mexico, Southeast Asia countries and European countries. We have won a good reputation for high-quality products, prompt delivery and close cooperation among our customers. We needs a representative in the United states, United Kingdom, Canada, Mexico, Southeast Asia countries and European countries, to act as our Online Staff through which our customers can pay outstanding bills owed by them to us in your Region via Bank Wire Transfer. JOB DESCRIPTION: 1. Receive payment from Clients by wire transfer and Cheques 2. Deduct 10% which will be your commission on each payment processed. 3. Forward the balance after deducting of 10% commission to offices which shall be provided by you as soon as the fund becomes available. HOW MUCH WILL YOU EARN: 10% from each operation! For instance: you receive Ł 5000 or $5000 via wire transfer Or Cheques on our behalf. You will cash the money and keep Ł 500 or $500 (10% from Ł 5000-$5000) for yourself! At the beginning your commission will equal 10%. After creditable performance, your commission may be reviewed for increment. We are looking only for the Honest and Open – Hearted Individual who satisfies our requirements and glad to offer this job position to you. If our proposals interest you, Do get back to us with your under listed detailed information; Names:.................. Address:................ City:................... Zip Code:............... State:.................. Country;................ Home Phone:............. Cell Phone:............. Gender:................. Age..................... Thanks for Reading Our Job Offer Kenneth Fabrics Limited 301-A, World Trade Tower Barakhamba Lane New Delhi -110001 India From owner-xfs@oss.sgi.com Sat Jun 2 14:25:00 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 14:25:03 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52LOxWt024035 for ; Sat, 2 Jun 2007 14:25:00 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 57052B000083; Sat, 2 Jun 2007 17:24:58 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 51C8350001A7; Sat, 2 Jun 2007 17:24:58 -0400 (EDT) Date: Sat, 2 Jun 2007 17:24:58 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: linux-kernel@vger.kernel.org cc: xfs@oss.sgi.com Subject: Kernel 2.6.22-rc3 safe to migrate to? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11574 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs Wondering as their were a lot of XFS related issues early on in development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch by ruik which I want be running as long as 2.6.22-rc3 does not have any severe XFS issues? Justin. From owner-xfs@oss.sgi.com Sat Jun 2 14:35:55 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 14:35:57 -0700 (PDT) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.181]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52LZsWt026693 for ; Sat, 2 Jun 2007 14:35:54 -0700 Received: by wa-out-1112.google.com with SMTP id l24so1189895waf for ; Sat, 02 Jun 2007 14:35:54 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition; b=GN6AfKnezeJpV3TkJoPQujAokhgjWw2yXxH39wXfwWmEBMPzjSn5b18uVJz0I44qkZ7JlLX2AjJGOFsCIZ1VL/P3VBlSPD0uHydGGIFfXcrm+nuSlnQGBnj3EUHonhkvz67PqPV03jsfIDTUqzwRMml8yTWRnkwUYfdSXk0+/Ps= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition; b=gmPrf1JYRMcZujT3O3OgGZOA+Fet2Agyr5OC5Mv8kUQhx7Whpj/65MZTNhL9JHW3Naw4ZgAhSDyQ5EzvV2c1JT51N+v2KWBIbM/9sl41kS3m0JZ/ljOzE9LANaV3kDL96+kLPIUNYURQZRFa59qIIV0Lz9s4eMysTY8e97RBZ9E= Received: by 10.115.92.2 with SMTP id u2mr3130430wal.1180818448644; Sat, 02 Jun 2007 14:07:28 -0700 (PDT) Received: by 10.114.14.8 with HTTP; Sat, 2 Jun 2007 14:07:28 -0700 (PDT) Message-ID: <5d96567b0706021407q4455b60asd9d23ef82cb90b55@mail.gmail.com> Date: Sun, 3 Jun 2007 00:07:28 +0300 From: "Raz Ben-Jehuda(caro)" To: linux-xfs@oss.sgi.com Subject: corruption bug in 2.6.17 Cc: alkirkco@sgi.com MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-archive-position: 11575 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: raziebe@gmail.com Precedence: bulk X-list: xfs Mandy Hello I have got into a serious trouble with xfs . I had some directories returning "no such file or directory". This xfs file system is over a raid5 of 4 disks, with 1MB stripe unit size. files are hierarchly being "XFS_IOC_FSSETXATTR" extended to 1MB. when I tried to create a new file I got the bellow oops. could it be that your fix for 2.6.17.7 solves this problem ? thank you. raz 4499008.609000] Filesystem "md1": XFS internal error xfs_dir2_block_addname at line 105 of file fs/xfs/xfs_dir2_block.c. Caller 0xc10e7e1c [4499008.634000] xfs_dir2_block_addname+0x88b/0x8a9 xfs_dir2_createname+0x11a/0x179 [4499008.654000] xfs_dir2_createname+0x11a/0x179 xfs_bmap_last_offset+0xce/0x128 [4499008.673000] xfs_dir2_isblock+0x32/0x82 xfs_dir2_createname+0x11a/0x179 [4499008.690000] kmem_zone_alloc+0x57/0xc5 xfs_trans_ijoin+0x35/0x7f [4499008.707000] xfs_create+0x45b/0x7aa xfs_vn_mknod+0x2f9/0x37e [4499008.725000] xfs_vn_lookup+0x79/0x96 lookup_mnt+0x32/0x5c [4499008.741000] do_lookup+0x50/0xa8 dput+0x23/0x183 [4499008.755000] __link_path_walk+0x50/0xe17 lookup_mnt+0x32/0x5c [4499008.772000] mntput_no_expire+0x22/0xb1 mntput_no_expire+0x22/0xb1 [4499008.789000] link_path_walk+0x71/0xc9 permission+0x7e/0x9b [4499008.805000] vfs_create+0x7d/0x122 open_namei+0x69e/0x6ff [4499008.820000] page_add_file_rmap+0x29/0x2b do_filp_open+0x43/0x61 [4499008.837000] do_sys_open+0x57/0xe1 sys_open+0x27/0x2b [4499008.852000] syscall_call+0x7/0xb [4499008.863000] Filesystem "md1": XFS internal error xfs_trans_cancel at line 1150 of file fs/xfs/xfs_trans.c. Caller 0xc112100e [4499008.886000] xfs_trans_cancel+0x103/0x136 xfs_create+0x2ec/0x7aa [4499008.903000] xfs_create+0x2ec/0x7aa xfs_vn_mknod+0x2f9/0x37e [4499008.918000] xfs_vn_lookup+0x79/0x96 lookup_mnt+0x32/0x5c [4499008.935000] do_lookup+0x50/0xa8 dput+0x23/0x183 [4499008.948000] __link_path_walk+0x50/0xe17 lookup_mnt+0x32/0x5c [4499008.966000] mntput_no_expire+0x22/0xb1 mntput_no_expire+0x22/0xb1 [4499008.983000] link_path_walk+0x71/0xc9 permission+0x7e/0x9b [4499008.999000] vfs_create+0x7d/0x122 open_namei+0x69e/0x6ff [4499009.015000] page_add_file_rmap+0x29/0x2b do_filp_open+0x43/0x61 [4499009.031000] do_sys_open+0x57/0xe1 sys_open+0x27/0x2b [4499009.046000] syscall_call+0x7/0xb [4499009.057000] xfs_force_shutdown(md1,0x8) called from line 1151 of file fs/xfs/xfs_trans.c. Return address = 0xc1131b54 [4499009.078000] Filesystem "md1": Corruption of in-memory data detected. Shutting down filesystem: md1 [4499009.097000] Please umount the filesystem, and rectify the problem(s) -- Raz From owner-xfs@oss.sgi.com Sat Jun 2 14:56:36 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 14:56:37 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52LuYWt003639 for ; Sat, 2 Jun 2007 14:56:35 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 8B9C4B000083; Sat, 2 Jun 2007 17:39:13 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 856EA50001A7; Sat, 2 Jun 2007 17:39:13 -0400 (EDT) Date: Sat, 2 Jun 2007 17:39:13 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: "Raz Ben-Jehuda(caro)" cc: linux-xfs@oss.sgi.com, alkirkco@sgi.com Subject: Re: corruption bug in 2.6.17 In-Reply-To: <5d96567b0706021407q4455b60asd9d23ef82cb90b55@mail.gmail.com> Message-ID: References: <5d96567b0706021407q4455b60asd9d23ef82cb90b55@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11576 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs Raz, 2.6.17-2.6.17.6 had the nasty corruption bug you're seeing, best to restore from backup and/or check the FAQ and get off that kernel range asap. On Sun, 3 Jun 2007, Raz Ben-Jehuda(caro) wrote: > Mandy Hello > > I have got into a serious trouble with xfs . I had some directories > returning "no such file or directory". This xfs file system is over a > raid5 of 4 disks, with 1MB stripe unit size. files are hierarchly > being "XFS_IOC_FSSETXATTR" extended to 1MB. > > when I tried to create a new file I got the bellow oops. > could it be that your fix for 2.6.17.7 solves this problem ? > thank you. > raz > > 4499008.609000] Filesystem "md1": XFS internal error > xfs_dir2_block_addname at line 105 of file fs/xfs/xfs_dir2_block.c. > Caller 0xc10e7e1c > [4499008.634000] xfs_dir2_block_addname+0x88b/0x8a9 > xfs_dir2_createname+0x11a/0x179 > [4499008.654000] xfs_dir2_createname+0x11a/0x179 > xfs_bmap_last_offset+0xce/0x128 > [4499008.673000] xfs_dir2_isblock+0x32/0x82 > xfs_dir2_createname+0x11a/0x179 > [4499008.690000] kmem_zone_alloc+0x57/0xc5 > xfs_trans_ijoin+0x35/0x7f > [4499008.707000] xfs_create+0x45b/0x7aa > xfs_vn_mknod+0x2f9/0x37e > [4499008.725000] xfs_vn_lookup+0x79/0x96 > lookup_mnt+0x32/0x5c > [4499008.741000] do_lookup+0x50/0xa8 dput+0x23/0x183 > [4499008.755000] __link_path_walk+0x50/0xe17 > lookup_mnt+0x32/0x5c > [4499008.772000] mntput_no_expire+0x22/0xb1 > mntput_no_expire+0x22/0xb1 > [4499008.789000] link_path_walk+0x71/0xc9 > permission+0x7e/0x9b > [4499008.805000] vfs_create+0x7d/0x122 > open_namei+0x69e/0x6ff > [4499008.820000] page_add_file_rmap+0x29/0x2b > do_filp_open+0x43/0x61 > [4499008.837000] do_sys_open+0x57/0xe1 > sys_open+0x27/0x2b > [4499008.852000] syscall_call+0x7/0xb > [4499008.863000] Filesystem "md1": XFS internal error xfs_trans_cancel > at line 1150 of file fs/xfs/xfs_trans.c. Caller 0xc112100e > [4499008.886000] xfs_trans_cancel+0x103/0x136 > xfs_create+0x2ec/0x7aa > [4499008.903000] xfs_create+0x2ec/0x7aa > xfs_vn_mknod+0x2f9/0x37e > [4499008.918000] xfs_vn_lookup+0x79/0x96 > lookup_mnt+0x32/0x5c > [4499008.935000] do_lookup+0x50/0xa8 dput+0x23/0x183 > [4499008.948000] __link_path_walk+0x50/0xe17 > lookup_mnt+0x32/0x5c > [4499008.966000] mntput_no_expire+0x22/0xb1 > mntput_no_expire+0x22/0xb1 > [4499008.983000] link_path_walk+0x71/0xc9 > permission+0x7e/0x9b > [4499008.999000] vfs_create+0x7d/0x122 > open_namei+0x69e/0x6ff > [4499009.015000] page_add_file_rmap+0x29/0x2b > do_filp_open+0x43/0x61 > [4499009.031000] do_sys_open+0x57/0xe1 > sys_open+0x27/0x2b > [4499009.046000] syscall_call+0x7/0xb > [4499009.057000] xfs_force_shutdown(md1,0x8) called from line 1151 of > file fs/xfs/xfs_trans.c. Return address = 0xc1131b54 > [4499009.078000] Filesystem "md1": Corruption of in-memory data > detected. Shutting down filesystem: md1 > [4499009.097000] Please umount the filesystem, and rectify the problem(s) > > > -- > Raz > > From owner-xfs@oss.sgi.com Sat Jun 2 14:59:10 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 14:59:14 -0700 (PDT) Received: from mail.g-house.de (ns2.g-housing.de [81.169.133.75]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52Lx9Wt005112 for ; Sat, 2 Jun 2007 14:59:10 -0700 Received: from [82.41.246.210] (helo=[10.0.0.30]) by mail.g-house.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1Hubck-0003Ai-Dd; Sat, 02 Jun 2007 23:59:02 +0200 Date: Sat, 2 Jun 2007 22:59:01 +0100 (BST) From: Christian Kujau X-X-Sender: evil@sheep.housecafe.de To: Justin Piszcz cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=us-ascii X-archive-position: 11577 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: lists@nerdbynature.de Precedence: bulk X-list: xfs On Sat, 2 Jun 2007, Justin Piszcz wrote: > Wondering as their were a lot of XFS related issues early on in > development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch by > ruik which I want be running as long as 2.6.22-rc3 does not have any severe > XFS issues? Tracking -rc and running 2.6.22-rc3 for a couple of days now, no problems with xfs since this infamous 2.6.17 thingy[0]. But that's pretty useless information I guess, since your setup will be different from mine. Did you have anything "special" in mind when asking this? C. [0] http://oss.sgi.com/projects/xfs/faq.html#dir2 -- BOFH excuse #339: manager in the cable duct From owner-xfs@oss.sgi.com Sat Jun 2 15:09:32 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 15:09:35 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52M9UWt009389 for ; Sat, 2 Jun 2007 15:09:32 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 1F629B000083; Sat, 2 Jun 2007 18:09:30 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 1C358500014A; Sat, 2 Jun 2007 18:09:30 -0400 (EDT) Date: Sat, 2 Jun 2007 18:09:30 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Christian Kujau cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11578 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs On Sat, 2 Jun 2007, Christian Kujau wrote: > On Sat, 2 Jun 2007, Justin Piszcz wrote: >> Wondering as their were a lot of XFS related issues early on in >> development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch by >> ruik which I want be running as long as 2.6.22-rc3 does not have any severe >> XFS issues? > > Tracking -rc and running 2.6.22-rc3 for a couple of days now, no problems > with xfs since this infamous 2.6.17 thingy[0]. But that's pretty useless > information I guess, since your setup will be different from mine. Did you > have anything "special" in mind when asking this? > > C. > > [0] http://oss.sgi.com/projects/xfs/faq.html#dir2 > -- > BOFH excuse #339: > > manager in the cable duct > Thanks, I'll give it a go then--! From owner-xfs@oss.sgi.com Sat Jun 2 15:27:06 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 15:27:09 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52MR5Wt016783 for ; Sat, 2 Jun 2007 15:27:06 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 8F780B000083; Sat, 2 Jun 2007 18:27:05 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 8BAB15000149; Sat, 2 Jun 2007 18:27:05 -0400 (EDT) Date: Sat, 2 Jun 2007 18:27:05 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Christian Kujau cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11579 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs Jun 2 18:23:23 p34 upsd[2225]: Can't connect to UPS [belkin] (newhidups-auto): No such file or directory Jun 2 18:24:23 p34 upsmon[2228]: Poll UPS [belkin@localhost] failed - Driver not connected Hmm, something changes with USB in 2.6.22-rc3 form 2.6.21.. On Sat, 2 Jun 2007, Christian Kujau wrote: > On Sat, 2 Jun 2007, Justin Piszcz wrote: >> Wondering as their were a lot of XFS related issues early on in >> development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch by >> ruik which I want be running as long as 2.6.22-rc3 does not have any severe >> XFS issues? > > Tracking -rc and running 2.6.22-rc3 for a couple of days now, no problems > with xfs since this infamous 2.6.17 thingy[0]. But that's pretty useless > information I guess, since your setup will be different from mine. Did you > have anything "special" in mind when asking this? > > C. > > [0] http://oss.sgi.com/projects/xfs/faq.html#dir2 > -- > BOFH excuse #339: > > manager in the cable duct > > From owner-xfs@oss.sgi.com Sat Jun 2 15:52:59 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 15:53:02 -0700 (PDT) Received: from mail.g-house.de (ns2.g-housing.de [81.169.133.75]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52MqwWt032611 for ; Sat, 2 Jun 2007 15:52:59 -0700 Received: from [82.41.246.210] (helo=[10.0.0.30]) by mail.g-house.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1Hubhc-0003SC-Op; Sun, 03 Jun 2007 00:04:04 +0200 Date: Sat, 2 Jun 2007 23:04:03 +0100 (BST) From: Christian Kujau X-X-Sender: evil@sheep.housecafe.de To: "Raz Ben-Jehuda(caro)" cc: linux-xfs@oss.sgi.com, alkirkco@sgi.com Subject: Re: corruption bug in 2.6.17 In-Reply-To: <5d96567b0706021407q4455b60asd9d23ef82cb90b55@mail.gmail.com> Message-ID: References: <5d96567b0706021407q4455b60asd9d23ef82cb90b55@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=us-ascii; format=flowed X-archive-position: 11580 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: lists@nerdbynature.de Precedence: bulk X-list: xfs On Sun, 3 Jun 2007, Raz Ben-Jehuda(caro) wrote: > when I tried to create a new file I got the bellow oops. > could it be that your fix for 2.6.17.7 solves this problem ? If you're running a 2.6.17 kernel (< 2.6.17.7) when you're getting these errors, I guess it's most likely this particular 2.6.17-bug. Please read http://oss.sgi.com/projects/xfs/faq.html#dir2 very carefully and upgrade xfsprogs and your kernel. C. -- BOFH excuse #55: Plumber mistook routing panel for decorative wall fixture From owner-xfs@oss.sgi.com Sat Jun 2 15:54:15 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 15:54:18 -0700 (PDT) Received: from mail.goop.org (gw.goop.org [64.81.55.164]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52MsEWt000899 for ; Sat, 2 Jun 2007 15:54:14 -0700 Received: by lurch.goop.org (Postfix, from userid 525) id 4E4B72C8058; Sat, 2 Jun 2007 15:53:17 -0700 (PDT) Received: from lurch.goop.org (localhost [127.0.0.1]) by lurch.goop.org (Postfix) with ESMTP id 2EB9B2C8053; Sat, 2 Jun 2007 15:53:17 -0700 (PDT) Received: from [192.168.28.126] (outer-dhcp-126.goop.org [192.168.28.126]) by lurch.goop.org (Postfix) with ESMTP; Sat, 2 Jun 2007 15:53:17 -0700 (PDT) Message-ID: <4661F511.4070207@goop.org> Date: Sat, 02 Jun 2007 15:54:09 -0700 From: Jeremy Fitzhardinge User-Agent: Thunderbird 1.5.0.10 (X11/20070302) MIME-Version: 1.0 To: Justin Piszcz CC: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? References: In-Reply-To: Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 11581 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jeremy@goop.org Precedence: bulk X-list: xfs Justin Piszcz wrote: > Wondering as their were a lot of XFS related issues early on in > development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch > by ruik which I want be running as long as 2.6.22-rc3 does not have > any severe XFS issues? XFS currently has a data-corrupting bug, where files which were appended by small amounts may lose their updates on umount - I see this corrupting hg repos. There's a patch which works for me, and is in 2.6.22-rc3-mm1, but it hasn't been merged upstream yet. J From owner-xfs@oss.sgi.com Sat Jun 2 15:55:45 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 15:55:48 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52MtiWt001689 for ; Sat, 2 Jun 2007 15:55:45 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 33BA8B000083; Sat, 2 Jun 2007 18:55:45 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 317BD5000091; Sat, 2 Jun 2007 18:55:45 -0400 (EDT) Date: Sat, 2 Jun 2007 18:55:45 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Jeremy Fitzhardinge cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: <4661F511.4070207@goop.org> Message-ID: References: <4661F511.4070207@goop.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11582 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs On Sat, 2 Jun 2007, Jeremy Fitzhardinge wrote: > Justin Piszcz wrote: >> Wondering as their were a lot of XFS related issues early on in >> development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch >> by ruik which I want be running as long as 2.6.22-rc3 does not have >> any severe XFS issues? > > XFS currently has a data-corrupting bug, where files which were appended > by small amounts may lose their updates on umount - I see this > corrupting hg repos. There's a patch which works for me, and is in > 2.6.22-rc3-mm1, but it hasn't been merged upstream yet. > > J > > Ah that's it- and USB appears to be broken as well, I'll stick with 2.6.21.3 for now. Thanks! Justin. From owner-xfs@oss.sgi.com Sat Jun 2 16:00:46 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 16:00:49 -0700 (PDT) Received: from smtp1.linux-foundation.org (smtp1.linux-foundation.org [207.189.120.13]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52N0jWt003744; Sat, 2 Jun 2007 16:00:46 -0700 Received: from localhost (phoenix.linux-foundation.org [207.189.120.27]) by smtp1.linux-foundation.org (8.13.5.20060308/8.13.5/Debian-3ubuntu1.1) with ESMTP id l52Mkqbd009363; Sat, 2 Jun 2007 15:46:53 -0700 Date: Sat, 2 Jun 2007 15:46:47 -0700 (PDT) From: Linus Torvalds To: David Greaves cc: "Rafael J. Wysocki" , xfs@oss.sgi.com, "'linux-kernel@vger.kernel.org'" , netdev@oss.sgi.com, linux-pm Subject: Re: 2.6.22-rc3 hibernate(?) fails totally - regression In-Reply-To: <4661EFBB.5010406@dgreaves.com> Message-ID: References: <46608E3F.4060201@dgreaves.com> <200706012342.45657.rjw@sisk.pl> <46609FAD.7010203@dgreaves.com> <200706020122.49989.rjw@sisk.pl> <4661EFBB.5010406@dgreaves.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=us-ascii Received-SPF: neutral (207.189.120.27 is neither permitted nor denied by domain of torvalds@linux-foundation.org) X-MIMEDefang-Filter: osdl$Revision: 1.179 $ X-Scanned-By: MIMEDefang 2.53 on 207.189.120.13 X-archive-position: 11583 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: torvalds@linux-foundation.org Precedence: bulk X-list: xfs On Sat, 2 Jun 2007, David Greaves wrote: > > Then 2.6.22-rc3 again but CONFIG_DISABLE_CONSOLE_SUSPEND=y > It suspended again. > Froze on restore. > Screen photo here: > http://www.dgreaves.com/pub/2.6.21-rc3-resume-failure.jpg Ok, it wasn't a hidden oops. The DISABLE_CONSOLE_SUSPEND=y thing sometimes shows oopses that are otherwise hidden, but at other times it just causes more problems (hard hangs when trying to display something on a device that is suspended, or behind a bridge that got suspended). In your case, the screen output just shows normal resume output, and it apparently just hung for some unknown reason. It *may* be worth trying to do a SysRQ + 't' thing to see what tasks are running (or rather, not running), but since you won't be able to capture it, it's probably not going to be useful. > Then 2.6.22-rc3 again but CONFIG_DISABLE_CONSOLE_SUSPEND=y > This time, before suspending I unmounted my xfs/lvm/raid6 filesystem. > Just a umount, I left the devices/array up. > It suspended again. > This time it resumed without fault. It would be interesting to see what triggered it, since it apparently worked before. So yes, a bisection would be great. Linus From owner-xfs@oss.sgi.com Sat Jun 2 16:02:11 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 16:02:16 -0700 (PDT) Received: from mail.ukfsn.org (s2.ukfsn.org [217.158.120.143]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l52N29Wt004683; Sat, 2 Jun 2007 16:02:11 -0700 Received: from localhost (mailman.ukfsn.org [80.168.53.75]) by mail.ukfsn.org (Postfix) with ESMTP id B51C7E6CAD; Sat, 2 Jun 2007 23:31:21 +0100 (BST) Received: from mail.ukfsn.org ([80.168.53.20]) by localhost (smtp-filter.ukfsn.org [80.168.53.75]) (amavisd-new, port 10024) with ESMTP id BfTBdpdPbsQE; Sat, 2 Jun 2007 23:29:21 +0100 (BST) Received: from elm.dgreaves.com (i-83-67-36-194.freedom2surf.net [83.67.36.194]) by mail.ukfsn.org (Postfix) with ESMTP id 2406EE6C62; Sat, 2 Jun 2007 23:31:21 +0100 (BST) Received: from ash.dgreaves.com ([10.0.0.90]) by elm.dgreaves.com with esmtp (Exim 4.62) (envelope-from ) id 1Huc83-0003U5-Kv; Sat, 02 Jun 2007 23:31:23 +0100 Message-ID: <4661EFBB.5010406@dgreaves.com> Date: Sat, 02 Jun 2007 23:31:23 +0100 From: David Greaves User-Agent: Icedove 1.5.0.10 (X11/20070329) MIME-Version: 1.0 To: "Rafael J. Wysocki" , Linus Torvalds , xfs@oss.sgi.com Cc: "'linux-kernel@vger.kernel.org'" , netdev@oss.sgi.com, linux-pm Subject: Re: 2.6.22-rc3 hibernate(?) fails totally - regression References: <46608E3F.4060201@dgreaves.com> <200706012342.45657.rjw@sisk.pl> <46609FAD.7010203@dgreaves.com> <200706020122.49989.rjw@sisk.pl> In-Reply-To: <200706020122.49989.rjw@sisk.pl> X-Enigmail-Version: 0.94.2.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-archive-position: 11584 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: david@dgreaves.com Precedence: bulk X-list: xfs This started as a non-regression bug-report about wakeonlan During tests I found a real regression and this email is only about 2.6.22-rc3 without Rafael's patches - which I'll happily come back to later :) Rafael J. Wysocki wrote: > On Saturday, 2 June 2007 00:37, David Greaves wrote: >> Rafael J. Wysocki wrote: >>> On Friday, 1 June 2007 23:23, David Greaves wrote: >> The real situation is worse :( > > Ouch. > >> 2.6.22-rc3 (no patches) just hangs on suspend at: >> Suspending consoles >> >> console switching works but needs a hard reset to reboot. >> >> 2.6.22-rc3-skge (with Rafael's patches) > Can you set CONFIG_DISABLE_CONSOLE_SUSPEND in .config and see where exactly it > fails? Given that I expected it to fail I unmounted my 1Tb array data before suspending. It succeeded... so I started digging... So I tried again. 2.6.22-rc3 (vanilla) suspended OK this time but on resume hung at: Stopping tasks done Shrinking memory done (0 pages freed) Freed 0 kb in 0.0 secs (0.0MB/s) Suspending console(s) Then 2.6.22-rc3 again but CONFIG_DISABLE_CONSOLE_SUSPEND=y It suspended again. Froze on restore. Screen photo here: http://www.dgreaves.com/pub/2.6.21-rc3-resume-failure.jpg Then 2.6.22-rc3 again but CONFIG_DISABLE_CONSOLE_SUSPEND=y This time, before suspending I unmounted my xfs/lvm/raid6 filesystem. Just a umount, I left the devices/array up. It suspended again. This time it resumed without fault. The machine has another xfs filesystem : /dev/hdb2 on /scratch type xfs (rw) I've started bisecting... any other info needed? David Added Linus as it's a regression Added xfs as unmounting an xfs filesystem 'fixes' it. From owner-xfs@oss.sgi.com Sat Jun 2 17:27:24 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 17:27:28 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l530RMWt003142 for ; Sat, 2 Jun 2007 17:27:24 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id C6A05B000083; Sat, 2 Jun 2007 20:27:22 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id C019150025A7; Sat, 2 Jun 2007 20:27:22 -0400 (EDT) Date: Sat, 2 Jun 2007 20:27:22 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Christian Kujau cc: Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: Message-ID: References: <4661F511.4070207@goop.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11586 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs On Sun, 3 Jun 2007, Christian Kujau wrote: > On Sat, 2 Jun 2007, Justin Piszcz wrote: >> On Sat, 2 Jun 2007, Jeremy Fitzhardinge wrote: >>> XFS currently has a data-corrupting bug, where files which were appended >>> by small amounts may lose their updates on umount - I see this >>> corrupting hg repos. There's a patch which works for me, and is in >>> 2.6.22-rc3-mm1, but it hasn't been merged upstream yet. > > Just for the record, you mean this one: > http://lkml.org/lkml/2007/5/12/93 ..right? (haven't been bitten by this > one...yet) > >> Ah that's it- and USB appears to be broken as well, I'll stick with >> 2.6.21.3 for now. > > Got any pointers? I'm using USB right now, one is i386 with 2.6.22-rc3, > another one is powerpc, tracking -git and USB seems to work. > > C. > -- > BOFH excuse #15: > > temporary routing anomaly > Sent this in another e-mail to LKML, it breaks support for my UPS: From jpiszcz@lucidpixels.com Sat Jun 2 18:43:52 2007 Date: Sat, 2 Jun 2007 18:43:52 -0400 (EDT) From: Justin Piszcz To: linux-kernel@vger.kernel.org Subject: Kernel 2.6.22-rc3 breaks USB: Unable to get HID descriptor (error sending control message: Operation not permitted) I use nut-2.0.4-4 with a UPS attached via USB and from 2.6.21.3 -> 2.6.22-rc3 it stops working, see below. My .config is attached. 2.6.21.3: p34:~# /lib/nut/newhidups -u nut -DDDDDD auto Checking device (050D/0912) (005/002) - VendorID: 050d - ProductID: 0912 - Manufacturer: - Product: UPS - Serial Number: unknown - Bus: 005 Trying to match device Device matches HID descriptor retrieved (Reportlen = 820) Size read for the report descriptor: 820 Report descriptor retrieved (Reportlen = 820) Found HID device Network UPS Tools: New USB/HID UPS driver 0.28 (2.0.4) Report Descriptor size = 820 Report Descriptor: (200 bytes) => 05 84 09 04 A1 01 05 86 09 26 A1 02 85 01 75 08 Detected a UPS: /UPS Using subdriver: Belkin HID 0.1 Looking up 00840004 Looking up 00860026 Looking up 00860040 entering string_to_path() parsing UPS Looking up UPS hid_lookup_usage: found 840004 parsing BELKINConfig Looking up BELKINConfig hid_lookup_usage: found 860026 parsing BELKINConfigVoltage Looking up BELKINConfigVoltage hid_lookup_usage: found 860040 Path depth = 3 0: UPage(84), Usage(4) 1: UPage(86), Usage(26) 2: UPage(86), Usage(40) Entering libusb_get_report =>> Before exponent: 120, 0/0) =>> After conversion: 120.000000 (120), 0/0) Report : (8 bytes) => 01 78 80 00 00 00 00 00 Path: UPS.BELKINConfig.BELKINConfigVoltage, Type: Feature, Value: 120.000000 Looking up 00840004 Looking up 00860026 Looking up 00860042 (works fine) 2.6.22-rc3: p34:~# /lib/nut/newhidups -u nut -DDDDDD auto Checking device (050D/0912) (005/002) - VendorID: 050d - ProductID: 0912 - Manufacturer: unknown - Product: unknown - Serial Number: unknown - Bus: 005 Trying to match device Device matches failed to claim USB device, trying 2 more time(s)... detaching kernel driver from USB device... failed to detach kernel driver from USB device... trying again to claim USB device... failed to claim USB device, trying 1 more time(s)... detaching kernel driver from USB device... failed to detach kernel driver from USB device... trying again to claim USB device... failed to claim USB device, trying 0 more time(s)... detaching kernel driver from USB device... failed to detach kernel driver from USB device... trying again to claim USB device... Unable to get HID descriptor (error sending control message: Operation not permitted) [ Part 2, "" Application/OCTET-STREAM (Name: "config-2.6.22-rc3.bz2") ] [ 10KB. ] [ Unable to print this part. ] From owner-xfs@oss.sgi.com Sat Jun 2 17:26:01 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 17:26:04 -0700 (PDT) Received: from mail.g-house.de (ns2.g-housing.de [81.169.133.75]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l530PxWt002207 for ; Sat, 2 Jun 2007 17:26:00 -0700 Received: from [82.41.246.210] (helo=[10.0.0.30]) by mail.g-house.de with esmtpsa (TLS-1.0:DHE_RSA_AES_256_CBC_SHA:32) (Exim 4.50) id 1Hudub-0008LF-CH; Sun, 03 Jun 2007 02:25:37 +0200 Date: Sun, 3 Jun 2007 01:25:35 +0100 (BST) From: Christian Kujau X-X-Sender: evil@sheep.housecafe.de To: Justin Piszcz cc: Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: Message-ID: References: <4661F511.4070207@goop.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=us-ascii X-archive-position: 11585 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: lists@nerdbynature.de Precedence: bulk X-list: xfs On Sat, 2 Jun 2007, Justin Piszcz wrote: > On Sat, 2 Jun 2007, Jeremy Fitzhardinge wrote: >> XFS currently has a data-corrupting bug, where files which were appended >> by small amounts may lose their updates on umount - I see this >> corrupting hg repos. There's a patch which works for me, and is in >> 2.6.22-rc3-mm1, but it hasn't been merged upstream yet. Just for the record, you mean this one: http://lkml.org/lkml/2007/5/12/93 ..right? (haven't been bitten by this one...yet) > Ah that's it- and USB appears to be broken as well, I'll stick with 2.6.21.3 > for now. Got any pointers? I'm using USB right now, one is i386 with 2.6.22-rc3, another one is powerpc, tracking -git and USB seems to work. C. -- BOFH excuse #15: temporary routing anomaly From owner-xfs@oss.sgi.com Sat Jun 2 23:30:56 2007 Received: with ECARTIS (v1.0.0; list xfs); Sat, 02 Jun 2007 23:31:01 -0700 (PDT) Received: from mail.goop.org (gw.goop.org [64.81.55.164]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l536UtWt020599 for ; Sat, 2 Jun 2007 23:30:56 -0700 Received: by lurch.goop.org (Postfix, from userid 525) id D7F6C2C805B; Sat, 2 Jun 2007 23:29:57 -0700 (PDT) Received: from lurch.goop.org (localhost [127.0.0.1]) by lurch.goop.org (Postfix) with ESMTP id A9B452C8056; Sat, 2 Jun 2007 23:29:57 -0700 (PDT) Received: from [192.168.28.126] (outer-dhcp-126.goop.org [192.168.28.126]) by lurch.goop.org (Postfix) with ESMTP; Sat, 2 Jun 2007 23:29:57 -0700 (PDT) Message-ID: <46626019.3070807@goop.org> Date: Sat, 02 Jun 2007 23:30:49 -0700 From: Jeremy Fitzhardinge User-Agent: Thunderbird 1.5.0.10 (X11/20070302) MIME-Version: 1.0 To: Christian Kujau CC: Justin Piszcz , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? References: <4661F511.4070207@goop.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-archive-position: 11587 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jeremy@goop.org Precedence: bulk X-list: xfs Christian Kujau wrote: > Just for the record, you mean this one: > http://lkml.org/lkml/2007/5/12/93 ..right? (haven't been bitten by > this one...yet) That's the one. It's fairly subtle; there's no metadata/filesystem corruption, and the file just reverts back to a previous length, so if it were append-only, it would just return to a previous state. J From owner-xfs@oss.sgi.com Sun Jun 3 00:23:44 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 00:23:51 -0700 (PDT) Received: from pd3mo2so.prod.shaw.ca (shawidc-mo1.cg.shawcable.net [24.71.223.10]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l537NhWt020808 for ; Sun, 3 Jun 2007 00:23:44 -0700 Received: from pd2mr4so.prod.shaw.ca (pd2mr4so-qfe3.prod.shaw.ca [10.0.141.107]) by l-daemon (Sun ONE Messaging Server 6.0 HotFix 1.01 (built Mar 15 2004)) with ESMTP id <0JJ10086OR3JP670@l-daemon> for xfs@oss.sgi.com; Sun, 03 Jun 2007 00:23:43 -0600 (MDT) Received: from pn2ml3so.prod.shaw.ca ([10.0.121.147]) by pd2mr4so.prod.shaw.ca (Sun Java System Messaging Server 6.2-7.05 (built Sep 5 2006)) with ESMTP id <0JJ100A65R3J1M20@pd2mr4so.prod.shaw.ca> for xfs@oss.sgi.com; Sun, 03 Jun 2007 00:23:43 -0600 (MDT) Received: from [192.168.1.113] ([70.64.1.86]) by l-daemon (Sun ONE Messaging Server 6.0 HotFix 1.01 (built Mar 15 2004)) with ESMTP id <0JJ100DXZR3IE3N3@l-daemon> for xfs@oss.sgi.com; Sun, 03 Jun 2007 00:23:42 -0600 (MDT) Date: Sun, 03 Jun 2007 00:23:48 -0600 From: Robert Hancock Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-reply-to: To: Justin Piszcz Cc: Christian Kujau , Jeremy Fitzhardinge , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Message-id: <46625E74.8070800@shaw.ca> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1; format=flowed Content-transfer-encoding: 7bit References: User-Agent: Thunderbird 2.0.0.0 (Windows/20070326) X-archive-position: 11588 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: hancockr@shaw.ca Precedence: bulk X-list: xfs Justin Piszcz wrote: > Sent this in another e-mail to LKML, it breaks support for my UPS: > > From jpiszcz@lucidpixels.com Sat Jun 2 18:43:52 2007 > Date: Sat, 2 Jun 2007 18:43:52 -0400 (EDT) > From: Justin Piszcz > To: linux-kernel@vger.kernel.org > Subject: Kernel 2.6.22-rc3 breaks USB: Unable to get HID descriptor > (error sending control message: Operation not permitted) > > I use nut-2.0.4-4 with a UPS attached via USB and from 2.6.21.3 -> > 2.6.22-rc3 it > stops working, see below. My .config is attached. > I also have seen this on -rc2-mm1, my APC UPS status was not showing up. I haven't investigated in any more detail. -- Robert Hancock Saskatoon, SK, Canada To email, remove "nospam" from hancockr@nospamshaw.ca Home Page: http://www.roberthancock.com/ From owner-xfs@oss.sgi.com Sun Jun 3 01:02:14 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 01:02:18 -0700 (PDT) Received: from mail.goop.org (gw.goop.org [64.81.55.164]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l53829Wt008118 for ; Sun, 3 Jun 2007 01:02:12 -0700 Received: by lurch.goop.org (Postfix, from userid 525) id C96452C805B; Sun, 3 Jun 2007 01:01:09 -0700 (PDT) Received: from lurch.goop.org (localhost [127.0.0.1]) by lurch.goop.org (Postfix) with ESMTP id A8CAF2C8056; Sun, 3 Jun 2007 01:01:09 -0700 (PDT) Received: from [192.168.28.126] (outer-dhcp-126.goop.org [192.168.28.126]) by lurch.goop.org (Postfix) with ESMTP; Sun, 3 Jun 2007 01:01:09 -0700 (PDT) Message-ID: <46627579.3040507@goop.org> Date: Sun, 03 Jun 2007 01:02:01 -0700 From: Jeremy Fitzhardinge User-Agent: Thunderbird 1.5.0.10 (X11/20070302) MIME-Version: 1.0 To: Michal Piotrowski CC: Justin Piszcz , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? References: <4661F511.4070207@goop.org> <6bffcb0e0706030059o3e53e647uf3ae1aa3f16609d8@mail.gmail.com> In-Reply-To: <6bffcb0e0706030059o3e53e647uf3ae1aa3f16609d8@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-archive-position: 11589 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jeremy@goop.org Precedence: bulk X-list: xfs Michal Piotrowski wrote: > It's already merged into mainline > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=df3c7244264f1d12562413aa32d56be802486516 > Oh, good. I hadn't read through the last couple of days of commits. J From owner-xfs@oss.sgi.com Sun Jun 3 01:25:29 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 01:25:34 -0700 (PDT) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.182]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l538PSWt019301 for ; Sun, 3 Jun 2007 01:25:29 -0700 Received: by wa-out-1112.google.com with SMTP id l24so1348090waf for ; Sun, 03 Jun 2007 01:25:28 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=qnDdIp58N/s+cogb5y3YWwYZrtZxI1esxvfHwQizEwtpia3oz6PydT8YZi95ijOBjo/5uMwzsBKfquAIjMTeqAbygVRGt6oMy6V8E3lYVGnr/Wyq1mNIkx/9rdat6Qa0epYb9PKzbuq3yIoacsHw/w9WxXEvDgfl1luj2nzcKGE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=aws8cpktjN3Zi3FuT+Fpb2VlzJ4nB826yZaHSW3MqQ50u1MlScXPtrKZdaGBTE6oGq4zP2oUh7TrDdZayX2Nupm+/tZlymSi77iUAYxx1oElfNq1ezg1oKbMUnWY0ALHYkuJS5ss0xZTX0pysXK/1xwxWWScd+Ep4omP2YW1wPo= Received: by 10.114.190.6 with SMTP id n6mr3618372waf.1180857555600; Sun, 03 Jun 2007 00:59:15 -0700 (PDT) Received: by 10.114.182.4 with HTTP; Sun, 3 Jun 2007 00:59:15 -0700 (PDT) Message-ID: <6bffcb0e0706030059o3e53e647uf3ae1aa3f16609d8@mail.gmail.com> Date: Sun, 3 Jun 2007 09:59:15 +0200 From: "Michal Piotrowski" To: "Jeremy Fitzhardinge" Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? Cc: "Justin Piszcz" , linux-kernel@vger.kernel.org, xfs@oss.sgi.com In-Reply-To: <4661F511.4070207@goop.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Disposition: inline References: <4661F511.4070207@goop.org> Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by oss.sgi.com id l538PTWt019318 X-archive-position: 11590 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: michal.k.k.piotrowski@gmail.com Precedence: bulk X-list: xfs H, On 03/06/07, Jeremy Fitzhardinge wrote:> Justin Piszcz wrote:> > Wondering as their were a lot of XFS related issues early on in> > development..? The 2.6.22-rc3 kernel has the core 2 duo coretemp patch> > by ruik which I want be running as long as 2.6.22-rc3 does not have> > any severe XFS issues?>> XFS currently has a data-corrupting bug, where files which were appended> by small amounts may lose their updates on umount - I see this> corrupting hg repos. There's a patch which works for me, and is in> 2.6.22-rc3-mm1, but it hasn't been merged upstream yet.> It's already merged into mainline http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=df3c7244264f1d12562413aa32d56be802486516 Regards,Michal -- "Najbardziej brakowało mi twojego milczenia."-- Andrzej Sapkowski "Co¶ więcej" From owner-xfs@oss.sgi.com Sun Jun 3 01:41:40 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 01:41:42 -0700 (PDT) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l538fcWt026312 for ; Sun, 3 Jun 2007 01:41:39 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id B9B3FB000083; Sun, 3 Jun 2007 04:41:38 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id B5DDA500009A; Sun, 3 Jun 2007 04:41:38 -0400 (EDT) Date: Sun, 3 Jun 2007 04:41:38 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Jeremy Fitzhardinge cc: Michal Piotrowski , linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: Kernel 2.6.22-rc3 safe to migrate to? In-Reply-To: <46627579.3040507@goop.org> Message-ID: References: <4661F511.4070207@goop.org> <6bffcb0e0706030059o3e53e647uf3ae1aa3f16609d8@mail.gmail.com> <46627579.3040507@goop.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-archive-position: 11591 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs On Sun, 3 Jun 2007, Jeremy Fitzhardinge wrote: > Michal Piotrowski wrote: >> It's already merged into mainline >> >> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=df3c7244264f1d12562413aa32d56be802486516 >> > > Oh, good. I hadn't read through the last couple of days of commits. > > J > > Nice, any word on the USB bug? From owner-xfs@oss.sgi.com Sun Jun 3 08:03:52 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 08:03:56 -0700 (PDT) Received: from mail.ukfsn.org (s2.ukfsn.org [217.158.120.143]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l53F3oWt017809; Sun, 3 Jun 2007 08:03:51 -0700 Received: from localhost (mailman.ukfsn.org [80.168.53.75]) by mail.ukfsn.org (Postfix) with ESMTP id 44CA5E6E87; Sun, 3 Jun 2007 16:03:42 +0100 (BST) Received: from mail.ukfsn.org ([80.168.53.20]) by localhost (smtp-filter.ukfsn.org [80.168.53.75]) (amavisd-new, port 10024) with ESMTP id E4XNnJ5bECXv; Sun, 3 Jun 2007 16:01:41 +0100 (BST) Received: from elm.dgreaves.com (i-83-67-36-194.freedom2surf.net [83.67.36.194]) by mail.ukfsn.org (Postfix) with ESMTP id 782C3E6D03; Sun, 3 Jun 2007 16:03:40 +0100 (BST) Received: from ash.dgreaves.com ([10.0.0.90]) by elm.dgreaves.com with esmtp (Exim 4.62) (envelope-from ) id 1HurcR-0004gS-3f; Sun, 03 Jun 2007 16:03:47 +0100 Message-ID: <4662D852.4000005@dgreaves.com> Date: Sun, 03 Jun 2007 16:03:46 +0100 From: David Greaves User-Agent: Icedove 1.5.0.10 (X11/20070329) MIME-Version: 1.0 To: Linus Torvalds , Tejun Heo Cc: "Rafael J. Wysocki" , xfs@oss.sgi.com, "'linux-kernel@vger.kernel.org'" , netdev@oss.sgi.com, linux-pm , Neil Brown Subject: Re: 2.6.22-rc3 hibernate(?) fails totally - regression References: <46608E3F.4060201@dgreaves.com> <200706012342.45657.rjw@sisk.pl> <46609FAD.7010203@dgreaves.com> <200706020122.49989.rjw@sisk.pl> <4661EFBB.5010406@dgreaves.com> In-Reply-To: X-Enigmail-Version: 0.94.2.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-archive-position: 11592 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: david@dgreaves.com Precedence: bulk X-list: xfs Linus Torvalds wrote: > It would be interesting to see what triggered it, since it apparently > worked before. So yes, a bisection would be great. Tejun, all the problematic patches are yours - so adding you. Neil, since the problem only occurs whilst an xfs filesystem is mounted on a raid6 array, I've cc'ed you too... OK Got as far as I could... I've run 9 or 10 kernels/bisects and got to a point with 8 of Tejun's changesets where it wouldn't compile: CC drivers/ata/sata_via.o drivers/ata/sata_via.c:120: error: `ata_scsi_device_suspend' undeclared here (not in a function) drivers/ata/sata_via.c:120: error: initializer element is not constant drivers/ata/sata_via.c:120: error: (near initialization for `svia_sht.suspend') drivers/ata/sata_via.c:121: error: `ata_scsi_device_resume' undeclared here (not in a function) drivers/ata/sata_via.c:121: error: initializer element is not constant drivers/ata/sata_via.c:121: error: (near initialization for `svia_sht.resume') make[2]: *** [drivers/ata/sata_via.o] Error 1 git bisect visualise gave: bad: 48aaae7a2fa46e1ed0d0b7677fde79ccfcb8c963 bisect: 54936f8b099325992f0f212a5e366fd5257c6c9c good: 0a3fd051c7036ef71b58863f8e5da7c3dabd9d3f I used: git reset --hard 8575b814097af648dad284bd3087875a11b13d18 git reset --hard e92351bb53c0849fabfa80be53cbf3b0aa166e54 git reset --hard 3a32a8e96694a243ec7e7feb6d76dfc4b1fe90c1 git reset --hard 9666f4009c22f6520ac3fb8a19c9e32ab973e828 to step through - non compiled git reset --hard 1d30c33d8d07868199560b24f10ed6280e78a89c compiled and hung on resume. given the first patch identified is 9666f4009c22f6520ac3fb8a19c9e32ab973e828: "libata: reimplement suspend/resume support using sdev->manage_start_stop" That seems a good candidate... Incidentally, when I compile 1d30c33d8d07868199560b24f10ed6280e78a89c (far side of the implicated changesets) if I umount my xfs over raid6 filesystem (no lvm as I said in the OP) the resume succeeds. David PS I hope I've interpreted bisect correctly - first use and all that... From owner-xfs@oss.sgi.com Sun Jun 3 14:37:59 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 14:38:03 -0700 (PDT) Received: from node21.rhrz.uni-bonn.de (node21-gb.rhrz.uni-bonn.de [131.220.15.211]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l53LbuWt026814 for ; Sun, 3 Jun 2007 14:37:58 -0700 Received: (from wwwserv@localhost) by node21.rhrz.uni-bonn.de (8.9.3/8.9.3) id XAA83218; Sun, 3 Jun 2007 23:37:56 +0200 Date: Sun, 3 Jun 2007 23:37:56 +0200 Message-Id: <200706032137.XAA83218@node21.rhrz.uni-bonn.de> To: xfs@oss.sgi.com Subject: Properties. From: Peter Kok Reply-To: as564ag@yahoo.com.hk MIME-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 8bit X-archive-position: 11593 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: as564ag@yahoo.com.hk Precedence: bulk X-list: xfs Good Day, I wish to introduce myself to you. I am Peter Kok a top Sudanese Goverment official who opposed the war in Dalfour in my country Sudan.Due to my oppostion to the war, the goverment of my country has been persecuting me and my family.Consequently my wife,children and I managed to enter a red cross airplane that was evacuating foreigners and we are presently in Cape Town, South Africa. We wish to invest in properties in your country with your assistance and cooperation.If you are in a good position to help my family, please send an e-mail to the e-mail address below indicating your desire to help my family invest this funds in your country. best regards Hope to meet you soon. God bless, Peter Kok E-mail:as564ag@yahoo.com.hk From owner-xfs@oss.sgi.com Sun Jun 3 17:16:55 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 17:16:59 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l540GpWt026604 for ; Sun, 3 Jun 2007 17:16:53 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id KAA27993; Mon, 4 Jun 2007 10:16:41 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l540GcAf113319342; Mon, 4 Jun 2007 10:16:39 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l540GWCC113287721; Mon, 4 Jun 2007 10:16:32 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 10:16:32 +1000 From: David Chinner To: Ruben Porras Cc: xfs@oss.sgi.com, iusty@k1024.org, cw@f00f.org Subject: Re: XFS shrink functionality Message-ID: <20070604001632.GA86004887@sgi.com> References: <1180715974.10796.46.camel@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1180715974.10796.46.camel@localhost> User-Agent: Mutt/1.4.2.1i X-archive-position: 11594 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs On Fri, Jun 01, 2007 at 06:39:34PM +0200, Ruben Porras wrote: > Hello, > > I'm investigating the possibility to write myself the necessary code to > shrink an xfs filesystem (I'd be able to dedicate a day/week). Trying to > know if something is already done I came across the mails of a previous > intent [0], [1] (I'm cc'ing the people involved). Oh, thanks for pointing those out - they're before my time ;) > At a first glance the patch is a little outdated and will no more apply > (as of linux 2.16.18, which is the last customised kernel that I was > able to run under a XEN environment), because at least the function > xfs_fs_geometry is changed. Any work for this would need to be done against current mainline of the xfs-dev tree. Yes, that patch is out of date, and it also did things that were not necessary i.e. walk btrees to work out if AGs are empty or not. > I'm really curious about what happened to this patches and why they were > discontinued. The second part never was made public, and there was also > no answer. Was there any flaw in any of the posted code or anything in > XFS that makes it especially hard to shrink [3] that discouraged the > development? The posted code is only a *tiny* part of the shrink problem. > After that, the first questions that arouse are, > would there be some assistance/groove in from the developers? Certainly there's help available. ;) > How doable is it? It is doable. > What are the programmers requirements from your point of view? Here's the "simple" bits that will allow you to shrink the filesystem down to the end of the internal log: 0. Check space is available for shrink 1. Mark allocation groups as "don't use - going away soon" - so we don't put new stuff in them while we are moving all the bits out of them - requires hooks in the allocators to prevent the AG from being selected for allllocations - must still allow allocations for the free lists so that extent freeing can succeed - *new transaction required*. - also needs an "undo" (e.g. on partial failure) so we need to be able to mark allocation groups online again. 2. Move inodes out of offline AGs - On Irix, we have a program called 'xfs_reno' which converts 64 bit inode filesystems to 32 bit inode filesystems. This needs to be: - released under the GPL (should not be a problem). - ported to linux - modified to understand inodes sit in certain AGs and to move them out of those AGs as needed. - requires filesystem traversal to find all the inodes to be moved. % wc -l xfs_reno.c 1991 xfs_reno.c - even with "-o ikeep", this needs to trigger inode cluster deletion in offline AGs (needs hooks in xfs_ifree()). 3. Move data out of offline AGs. - this is difficult to do efficiently as we do not have a block-to-owner reverse mapping in the filesystem. Hence requires a walk of the *entire* filesystem to find the owners of data blocks in the AGs being offlined. - xfs_db wrapper might be the best way to do this... 4. Execute shrink - new transaction - XFS_TRANS_SHRINKFS - check AGs are empty - icount == 0 - freeblks == mp->m_sb.sb_agblocks (will be a little more than this) - check shrink won't go past end of internal log - free AGs, updating superblock fields - update perag structure - not a simple realloc() as there may be other threads using the structure at the same time.... Initially, I'd say just support shrinking to whole AGs - you've got to empty the whole "partial-last-ag" to ensure we can shrink it anyway, so doing a subsequent grow operation to increase the size afterwards should be trivial. Once this all works, we can then tackle the "move the log" problem which will allow you to shrink to much smaller sizes. As you can see, doing a shrink properly is not trivial, which is probably why it has't gone anywhere fast.... Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group From owner-xfs@oss.sgi.com Sun Jun 3 20:09:56 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 20:10:06 -0700 (PDT) Received: from MFWJ042.mfw.is.co.za (mfwj042.mfw.is.co.za [196.35.77.58]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l5439rWt028417 for ; Sun, 3 Jun 2007 20:09:54 -0700 Received: from MailMarshal.Engine ([127.0.0.1]) by MFWJ042.mfw.is.co.za with MailMarshal (v5.5.7.1596) id ; Mon, 04 Jun 2007 05:14:35 +0200 Message-ID: From: mailfwadmin@bec.co.za To: linux-xfs@oss.sgi.com CC: archspec@plascon.co.za Date: Mon, 04 Jun 2007 05:14:35 +0200 Subject: Message Stopped ---- Virus Detected ---- MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="--=614e9445-c29c-4b50-a572-36ff706140c7" X-archive-position: 11595 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: mailfwadmin@bec.co.za Precedence: bulk X-list: xfs ----=614e9445-c29c-4b50-a572-36ff706140c7 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Barloworld Plascon MailFirewall has stopped the following message: Message: BA02f6702b.00000001.mml From: linux-xfs@oss.sgi.com To: archspec@plascon.co.za Subject: Because it believes the message or an attachment to the message contains a virus. The virus scanning software used was: McAfee for Marshal W32/Mydoom.o@MM!zip Barloworld Plascon MailFirewall Rule: Plascon Inbound : Block Virus Email Content Security provided by NetIQ MailMarshal. ----=614e9445-c29c-4b50-a572-36ff706140c7-- From owner-xfs@oss.sgi.com Sun Jun 3 21:31:59 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 21:32:12 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l544VoWt011482 for ; Sun, 3 Jun 2007 21:31:58 -0700 Received: from chook.melbourne.sgi.com (chook.melbourne.sgi.com [134.14.54.237]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA03794; Mon, 4 Jun 2007 14:31:46 +1000 Received: by chook.melbourne.sgi.com (Postfix, from userid 16346) id 225DC58C38C1; Mon, 4 Jun 2007 14:31:46 +1000 (EST) To: xfs@oss.sgi.com, sgi.bugs.xfs@engr.sgi.com Subject: PARTIAL TAKE 964092 - synchronous direct I/O write calls are incomplete when returning to user space Message-Id: <20070604043146.225DC58C38C1@chook.melbourne.sgi.com> Date: Mon, 4 Jun 2007 14:31:46 +1000 (EST) From: dgc@sgi.com (David Chinner) X-archive-position: 11596 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs QA test to exercise unwritten extent conversion for sync direct I/O Date: Mon Jun 4 14:31:01 AEST 2007 Workarea: chook.melbourne.sgi.com:/build/dgc/isms/xfs-cmds Inspected by: donaldd The following file(s) were checked into: longdrop.melbourne.sgi.com:/isms/xfs-cmds/master-melb Modid: master-melb:xfs-cmds:28769a xfstests/167 - 1.1 - new http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/167 xfstests/167.out - 1.1 - new http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/167.out xfstests/src/unwritten_sync.c - 1.1 - new http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/src/unwritten_sync.c xfstests/group - 1.105 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/group.diff?r1=text&tr1=1.105&r2=text&tr2=1.104&f=h xfstests/src/Makefile - 1.40 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/src/Makefile.diff?r1=text&tr1=1.40&r2=text&tr2=1.39&f=h - QA test to exercise unwritten extent conversion for sync direct I/O. From owner-xfs@oss.sgi.com Sun Jun 3 21:41:02 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 21:41:06 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l544exWt015527 for ; Sun, 3 Jun 2007 21:41:01 -0700 Received: from chook.melbourne.sgi.com (chook.melbourne.sgi.com [134.14.54.237]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA03959; Mon, 4 Jun 2007 14:40:54 +1000 Received: by chook.melbourne.sgi.com (Postfix, from userid 16346) id C4CA958C38C1; Mon, 4 Jun 2007 14:40:54 +1000 (EST) To: xfs@oss.sgi.com, sgi.bugs.xfs@engr.sgi.com Subject: TAKE 965626 - xfsqa - test 030 doesn't test what it's supposed to Message-Id: <20070604044054.C4CA958C38C1@chook.melbourne.sgi.com> Date: Mon, 4 Jun 2007 14:40:54 +1000 (EST) From: dgc@sgi.com (David Chinner) X-archive-position: 11597 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs Make sure the repair tests dirty the filesystem before corrupting it. Date: Mon Jun 4 14:40:06 AEST 2007 Workarea: chook.melbourne.sgi.com:/build/dgc/isms/xfs-cmds Inspected by: ddiss The following file(s) were checked into: longdrop.melbourne.sgi.com:/isms/xfs-cmds/master-melb Modid: master-melb:xfs-cmds:28770a xfstests/common.repair - 1.14 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/common.repair.diff?r1=text&tr1=1.14&r2=text&tr2=1.13&f=h xfstests/030.out.irix - 1.2 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/030.out.irix.diff?r1=text&tr1=1.2&r2=text&tr2=1.1&f=h xfstests/030.out.linux - 1.2 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/030.out.linux.diff?r1=text&tr1=1.2&r2=text&tr2=1.1&f=h xfstests/148.out - 1.2 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-cmds/xfstests/148.out.diff?r1=text&tr1=1.2&r2=text&tr2=1.1&f=h - Make sure the repair tests dirty the filesystem before corrupting it. From owner-xfs@oss.sgi.com Sun Jun 3 21:52:30 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 21:52:33 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l544qPWt021124 for ; Sun, 3 Jun 2007 21:52:28 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA04203; Mon, 4 Jun 2007 14:52:21 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l544qKAf112221586; Mon, 4 Jun 2007 14:52:21 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l544qJKp110131735; Mon, 4 Jun 2007 14:52:19 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 14:52:19 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss Subject: Review: Be smarter about handling ENOSPC during writeback Message-ID: <20070604045219.GG86004887@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11598 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs During delayed allocation extent conversion or unwritten extent conversion, we need to reserve some blocks for transactions reservations. We need to reserve these blocks in case a btree split occurs and we need to allocate some blocks. Unfortunately, we've only ever reserved the number of data blocks we are allocating, so in both the unwritten and delalloc case we can get ENOSPC to the transaction reservation. This is bad because in both cases we cannot report the failure to the writing application. The fix is two-fold: 1 - leverage the reserved block infrastructure XFS already has to reserve a small pool of blocks by default to allow specially marked transactions to dip into when we are at ENOSPC. Default setting is min(5%, 1024 blocks). 2 - convert critical transaction reservations to be allowed to dip into this pool. Spots changed are delalloc conversion, unwritten extent conversion and growing a filesystem at ENOSPC. Comments? Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- fs/xfs/xfs_fsops.c | 10 +++++++--- fs/xfs/xfs_iomap.c | 22 ++++++++-------------- fs/xfs/xfs_mount.c | 37 +++++++++++++++++++++++++++++++++++-- 3 files changed, 50 insertions(+), 19 deletions(-) Index: 2.6.x-xfs-new/fs/xfs/xfs_fsops.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_fsops.c 2007-05-11 10:35:29.288847149 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_fsops.c 2007-05-11 11:13:34.195363437 +1000 @@ -179,6 +179,7 @@ xfs_growfs_data_private( up_write(&mp->m_peraglock); } tp = xfs_trans_alloc(mp, XFS_TRANS_GROWFS); + tp->t_flags |= XFS_TRANS_RESERVE; if ((error = xfs_trans_reserve(tp, XFS_GROWFS_SPACE_RES(mp), XFS_GROWDATA_LOG_RES(mp), 0, 0, 0))) { xfs_trans_cancel(tp, 0); @@ -500,8 +501,9 @@ xfs_reserve_blocks( unsigned long s; /* If inval is null, report current values and return */ - if (inval == (__uint64_t *)NULL) { + if (!outval) + return EINVAL; outval->resblks = mp->m_resblks; outval->resblks_avail = mp->m_resblks_avail; return 0; @@ -564,8 +566,10 @@ retry: } } out: - outval->resblks = mp->m_resblks; - outval->resblks_avail = mp->m_resblks_avail; + if (outval) { + outval->resblks = mp->m_resblks; + outval->resblks_avail = mp->m_resblks_avail; + } XFS_SB_UNLOCK(mp, s); if (fdblks_delta) { Index: 2.6.x-xfs-new/fs/xfs/xfs_mount.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_mount.c 2007-05-11 10:35:29.292846630 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_mount.c 2007-05-11 11:13:47.229662318 +1000 @@ -718,7 +718,7 @@ xfs_mountfs( bhv_vnode_t *rvp = NULL; int readio_log, writeio_log; xfs_daddr_t d; - __uint64_t ret64; + __uint64_t resblks; __int64_t update_flags; uint quotamount, quotaflags; int agno; @@ -835,6 +835,7 @@ xfs_mountfs( */ if ((mfsi_flags & XFS_MFSI_SECOND) == 0 && (mp->m_flags & XFS_MOUNT_NOUUID) == 0) { + __uint64_t ret64; if (xfs_uuid_mount(mp)) { error = XFS_ERROR(EINVAL); goto error1; @@ -1127,13 +1128,27 @@ xfs_mountfs( goto error4; } - /* * Complete the quota initialisation, post-log-replay component. */ if ((error = XFS_QM_MOUNT(mp, quotamount, quotaflags, mfsi_flags))) goto error4; + /* + * Now we are mounted, reserve a small amount of unused space for + * privileged transactions. This is needed so that transaction + * space required for critical operations can dip into this pool + * when at ENOSPC. This is needed for operations like create with + * attr, unwritten extent conversion at ENOSPC, etc. Data allocations + * are not allowed to use this reserved space. + * + * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. + * This may drive us straight to ENOSPC on mount, but that implies + * we were already there on the last unmount. + */ + resblks = min_t(__uint64_t, mp->m_sb.sb_dblocks / 20, 1024); + xfs_reserve_blocks(mp, &resblks, NULL); + return 0; error4: @@ -1172,6 +1187,7 @@ xfs_unmountfs(xfs_mount_t *mp, struct cr #if defined(DEBUG) || defined(INDUCE_IO_ERROR) int64_t fsid; #endif + __uint64_t resblks; /* * We can potentially deadlock here if we have an inode cluster @@ -1200,6 +1216,23 @@ xfs_unmountfs(xfs_mount_t *mp, struct cr xfs_binval(mp->m_rtdev_targp); } + /* + * Unreserve any blocks we have so that when we unmount we don't account + * the reserved free space as used. This is really only necessary for + * lazy superblock counting because it trusts the incore superblock + * counters to be aboslutely correct on clean unmount. + * + * We don't bother correcting this elsewhere for lazy superblock + * counting because on mount of an unclean filesystem we reconstruct the + * correct counter value and this is irrelevant. + * + * For non-lazy counter filesystems, this doesn't matter at all because + * we only every apply deltas to the superblock and hence the incore + * value does not matter.... + */ + resblks = 0; + xfs_reserve_blocks(mp, &resblks, NULL); + xfs_log_sbcount(mp, 1); xfs_unmountfs_writesb(mp); xfs_unmountfs_wait(mp); /* wait for async bufs */ Index: 2.6.x-xfs-new/fs/xfs/xfs_iomap.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_iomap.c 2007-05-11 11:13:13.862017149 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_iomap.c 2007-05-11 11:13:34.199362915 +1000 @@ -489,13 +489,13 @@ xfs_iomap_write_direct( if (unlikely(rt)) { resrtextents = qblocks = resaligned; resrtextents /= mp->m_sb.sb_rextsize; - resblks = XFS_DIOSTRAT_SPACE_RES(mp, 0); - quota_flag = XFS_QMOPT_RES_RTBLKS; - } else { - resrtextents = 0; + resblks = XFS_DIOSTRAT_SPACE_RES(mp, 0); + quota_flag = XFS_QMOPT_RES_RTBLKS; + } else { + resrtextents = 0; resblks = qblocks = XFS_DIOSTRAT_SPACE_RES(mp, resaligned); - quota_flag = XFS_QMOPT_RES_REGBLKS; - } + quota_flag = XFS_QMOPT_RES_REGBLKS; + } /* * Allocate and setup the transaction @@ -788,18 +788,12 @@ xfs_iomap_write_allocate( nimaps = 0; while (nimaps == 0) { tp = xfs_trans_alloc(mp, XFS_TRANS_STRAT_WRITE); + tp->t_flags |= XFS_TRANS_RESERVE; nres = XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK); error = xfs_trans_reserve(tp, nres, XFS_WRITE_LOG_RES(mp), 0, XFS_TRANS_PERM_LOG_RES, XFS_WRITE_LOG_COUNT); - if (error == ENOSPC) { - error = xfs_trans_reserve(tp, 0, - XFS_WRITE_LOG_RES(mp), - 0, - XFS_TRANS_PERM_LOG_RES, - XFS_WRITE_LOG_COUNT); - } if (error) { xfs_trans_cancel(tp, 0); return XFS_ERROR(error); @@ -917,8 +911,8 @@ xfs_iomap_write_unwritten( * from unwritten to real. Do allocations in a loop until * we have covered the range passed in. */ - tp = xfs_trans_alloc(mp, XFS_TRANS_STRAT_WRITE); + tp->t_flags |= XFS_TRANS_RESERVE; error = xfs_trans_reserve(tp, resblks, XFS_WRITE_LOG_RES(mp), 0, XFS_TRANS_PERM_LOG_RES, From owner-xfs@oss.sgi.com Sun Jun 3 21:53:27 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 21:53:31 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l544rNWt021743 for ; Sun, 3 Jun 2007 21:53:26 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id OAA04217; Mon, 4 Jun 2007 14:53:19 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l544rIAf110554529; Mon, 4 Jun 2007 14:53:18 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l544rHLO111675106; Mon, 4 Jun 2007 14:53:17 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 14:53:17 +1000 From: David Chinner To: David Chinner Cc: xfs-dev , xfs-oss Subject: Re: Review - writing to multiple non-contiguous unwritten extents within a page is broken. Message-ID: <20070604045317.GH86004887@sgi.com> References: <20070523092103.GT85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070523092103.GT85884050@sgi.com> User-Agent: Mutt/1.4.2.1i X-archive-position: 11599 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs Ping? On Wed, May 23, 2007 at 07:21:03PM +1000, David Chinner wrote: > [Nathan - probably another one for you] > > This test run on ia64 (16k page size) on a 4k block size filesystem: > > #!/bin/bash > > file=$1 > rm -f $file > > xfs_io -f \ > -c "truncate 1048576" \ > -c "resvsp 1032192 16384" \ > -c "pwrite 1033216 2560" \ > -c "pwrite 1040384 8192" \ > -c "bmap -vvp" \ > -c "fsync" \ > -c "bmap -vvp" \ > -c "close" \ > $file > > Which writes 3 unwritten blocks in a page (first block and last 2) > results in a corrupted write. > > The problem is that the second block on teh page is uninitialised > and so is skipped by xfs_page_state_convert. The problem is that the > xfs_ioend structures are not getting created correctly. > > When we skip the uninitialised block, we add the second unwritten block > we are writing to into the original ioend. While this results in > the correct I/O being sent to disk, it results in a ioend with a > start offset of 0 and a length of 3 blocks. When we do unwritten > extent conversion based on this range, we convert the wrong blocks. > > What we need to be doing is creating two xfs_ioend structures, one > for the first block and one for the second set of blocks in the page. > That way we get two separate I/O completion events and convert the > ranges separately and correctly. > > I've checked xfs_convert_page(), and I don't think it needs any > fix here - it already appears to force multiple ioends to be used in this > case... > > Thoughts? > > Cheers, > > Dave. > -- > Dave Chinner > Principal Engineer > SGI Australian Software Group > > --- > fs/xfs/linux-2.6/xfs_aops.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > Index: 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_aops.c > =================================================================== > --- 2.6.x-xfs-new.orig/fs/xfs/linux-2.6/xfs_aops.c 2007-05-23 16:33:04.000000000 +1000 > +++ 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_aops.c 2007-05-23 17:52:15.540456674 +1000 > @@ -1008,6 +1008,8 @@ xfs_page_state_convert( > if (buffer_unwritten(bh) || buffer_delay(bh) || > ((buffer_uptodate(bh) || PageUptodate(page)) && > !buffer_mapped(bh) && (unmapped || startio))) { > + int new_ioend = 0; > + > /* > * Make sure we don't use a read-only iomap > */ > @@ -1026,6 +1028,15 @@ xfs_page_state_convert( > } > > if (!iomap_valid) { > + /* > + * if we didn't have a valid mapping then we > + * need to ensure that we put the new mapping > + * in a new ioend structure. This needs to be > + * done to ensure that the ioends correctly > + * reflect the block mappings at io completion > + * for unwritten extent conversion. > + */ > + new_ioend = 1; > if (type == IOMAP_NEW) { > size = xfs_probe_cluster(inode, > page, bh, head, 0); > @@ -1045,7 +1056,7 @@ xfs_page_state_convert( > if (startio) { > xfs_add_to_ioend(inode, bh, offset, > type, &ioend, > - !iomap_valid); > + new_ioend); > } else { > set_buffer_dirty(bh); > unlock_buffer(bh); -- Dave Chinner Principal Engineer SGI Australian Software Group From owner-xfs@oss.sgi.com Sun Jun 3 22:14:44 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 22:14:47 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l545EdWt002262 for ; Sun, 3 Jun 2007 22:14:42 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA04599; Mon, 4 Jun 2007 15:14:35 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l545EYAf113392920; Mon, 4 Jun 2007 15:14:35 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l545EXRb113181890; Mon, 4 Jun 2007 15:14:33 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 15:14:33 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss Subject: Review: remount read-only path is as broken as freezing was.... Message-ID: <20070604051433.GP85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11600 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs I recently had a remount,ro test fail in a way I had previously only seen freezing fail. That is, it failed because we still had active transactions after calling xfs_quiesce_fs(). Further investigation shows that this path is broken in the same ways that the xfs freeze path was broken (and recently fixed). Make the remount,ro path properly flush the filesystem down to a clean state before returning. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- fs/xfs/linux-2.6/xfs_super.c | 2 - fs/xfs/linux-2.6/xfs_vfs.h | 10 +++++++ fs/xfs/xfs_vfsops.c | 54 ++++++++++++++++++++++++------------------- 3 files changed, 42 insertions(+), 24 deletions(-) Index: 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_super.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/linux-2.6/xfs_super.c 2007-05-10 16:56:09.594774832 +1000 +++ 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_super.c 2007-05-10 17:39:02.374544197 +1000 @@ -726,7 +726,7 @@ xfs_fs_sync_super( * occur here so don't bother flushing the buftarg (i.e * SYNC_QUIESCE) because it'll just get dirty again. */ - flags = SYNC_FSDATA | SYNC_DELWRI | SYNC_WAIT | SYNC_IOWAIT; + flags = SYNC_DATA_QUIESCE; } else flags = SYNC_FSDATA | (wait ? SYNC_WAIT : 0); Index: 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_vfs.h =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/linux-2.6/xfs_vfs.h 2007-05-10 16:56:09.590775350 +1000 +++ 2.6.x-xfs-new/fs/xfs/linux-2.6/xfs_vfs.h 2007-05-10 17:39:02.386542627 +1000 @@ -94,6 +94,16 @@ typedef enum { #define SYNC_IOWAIT 0x0100 /* wait for all I/O to complete */ #define SYNC_SUPER 0x0200 /* flush superblock to disk */ +/* + * When remounting a filesystem read-only or freezing the filesystem, + * we have two phases to execute. This first phase is syncing the data + * before we quiesce the filesystem, and the second is flushing all the + * inodes out after we've waited for all the transactions created by + * the first phase to complete. + */ +#define SYNC_DATA_QUIESCE (SYNC_DELWRI|SYNC_FSDATA|SYNC_WAIT|SYNC_IOWAIT) +#define SYNC_INODE_QUIESCE (SYNC_REMOUNT|SYNC_ATTR|SYNC_WAIT) + #define SHUTDOWN_META_IO_ERROR 0x0001 /* write attempt to metadata failed */ #define SHUTDOWN_LOG_IO_ERROR 0x0002 /* write attempt to the log failed */ #define SHUTDOWN_FORCE_UMOUNT 0x0004 /* shutdown from a forced unmount */ Index: 2.6.x-xfs-new/fs/xfs/xfs_vfsops.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_vfsops.c 2007-05-10 17:38:58.351070788 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_vfsops.c 2007-05-10 17:58:09.958425162 +1000 @@ -665,7 +665,7 @@ xfs_quiesce_fs( * we can write the unmount record. */ do { - xfs_syncsub(mp, SYNC_REMOUNT|SYNC_ATTR|SYNC_WAIT, NULL); + xfs_syncsub(mp, SYNC_INODE_QUIESCE, NULL); pincount = xfs_flush_buftarg(mp->m_ddev_targp, 1); if (!pincount) { count++; @@ -682,6 +682,30 @@ xfs_quiesce_fs( return 0; } +/* + * Second stage of a quiesce. The data is already synced, now we have to take + * care of the metadata. New transactions are already blocked, so we need to + * wait for any remaining transactions to drain out before proceding. + */ +STATIC void +xfs_attr_quiesce( + xfs_mount_t *mp) +{ + /* wait for all modifications to complete */ + while (atomic_read(&mp->m_active_trans) > 0) + delay(100); + + /* flush inodes and push all remaining buffers out to disk */ + xfs_quiesce_fs(mp); + + ASSERT_ALWAYS(atomic_read(&mp->m_active_trans) == 0); + + /* Push the superblock and write an unmount record */ + xfs_log_sbcount(mp, 1); + xfs_log_unmount_write(mp); + xfs_unmountfs_writesb(mp); +} + STATIC int xfs_mntupdate( bhv_desc_t *bdp, @@ -701,11 +725,7 @@ xfs_mntupdate( mp->m_flags &= ~XFS_MOUNT_BARRIER; } } else if (!(vfsp->vfs_flag & VFS_RDONLY)) { /* rw -> ro */ - bhv_vfs_sync(vfsp, SYNC_FSDATA|SYNC_BDFLUSH|SYNC_ATTR, NULL); - xfs_quiesce_fs(mp); - xfs_log_sbcount(mp, 1); - xfs_log_unmount_write(mp); - xfs_unmountfs_writesb(mp); + bhv_vfs_sync(vfsp, SYNC_DATA_QUIESCE, NULL); + xfs_attr_quiesce(mp); vfsp->vfs_flag |= VFS_RDONLY; } return 0; @@ -1998,9 +2018,9 @@ xfs_showargs( } /* - * Second stage of a freeze. The data is already frozen, now we have to take - * care of the metadata. New transactions are already blocked, so we need to - * wait for any remaining transactions to drain out before proceding. + * Second stage of a freeze. The data is already frozen so we only + * need to take care of the metadata. Once that's done write a dummy + * record to dirty the log in case of a crash while frozen. */ STATIC void xfs_freeze( @@ -2008,19 +2028,7 @@ xfs_freeze( { xfs_mount_t *mp = XFS_BHVTOM(bdp); - /* wait for all modifications to complete */ - while (atomic_read(&mp->m_active_trans) > 0) - delay(100); - - /* flush inodes and push all remaining buffers out to disk */ - xfs_quiesce_fs(mp); - - ASSERT_ALWAYS(atomic_read(&mp->m_active_trans) == 0); - - /* Push the superblock and write an unmount record */ - xfs_log_sbcount(mp, 1); - xfs_log_unmount_write(mp); - xfs_unmountfs_writesb(mp); + xfs_attr_quiesce(mp); xfs_fs_log_dummy(mp); } From owner-xfs@oss.sgi.com Sun Jun 3 22:19:55 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 22:19:57 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l545JoWt004337 for ; Sun, 3 Jun 2007 22:19:54 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA04811; Mon, 4 Jun 2007 15:19:46 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l545JjAf113117607; Mon, 4 Jun 2007 15:19:46 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l545JiJ1113297283; Mon, 4 Jun 2007 15:19:44 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 15:19:44 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss Subject: Review: xfs_bmapi does not update previous extent pointer correctly Message-ID: <20070604051944.GQ85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11601 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs When looping across multiple extents, xfs_bmapi will fail to update the previous extent pointer which is used in subsequent loops. As a result, we can end up with the second loop in xfs_bmapi trying to use an incorrect previous extent pointer and assert failures or corrupted in-memory extent lists will result. Correctly update the previous extent at the end of each loop so that we DTRT when processing multiple map requests. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- fs/xfs/xfs_bmap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) Index: 2.6.x-xfs-new/fs/xfs/xfs_bmap.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_bmap.c 2007-05-23 16:33:00.000000000 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_bmap.c 2007-05-25 11:53:31.949847746 +1000 @@ -5575,10 +5575,10 @@ xfs_bmapi( * Else go on to the next record. */ ep = xfs_iext_get_ext(ifp, ++lastx); - if (lastx >= nextents) { + prev = got; + if (lastx >= nextents) eof = 1; - prev = got; - } else + else xfs_bmbt_get_all(ep, &got); } ifp->if_lastex = lastx; From owner-xfs@oss.sgi.com Sun Jun 3 22:23:45 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 22:23:48 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l545NdWt006180 for ; Sun, 3 Jun 2007 22:23:43 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA04962; Mon, 4 Jun 2007 15:23:35 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l545NYAf110674749; Mon, 4 Jun 2007 15:23:35 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l545NXlg113315390; Mon, 4 Jun 2007 15:23:33 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 15:23:33 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss Subject: Review: factor extracting extent size hints from the inode Message-ID: <20070604052333.GR85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11602 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs Replace frequently repeated, open coded extraction of the extent size hint from the xfs_inode with a single helper function. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- fs/xfs/xfs_bmap.c | 33 +++++++++++---------------------- fs/xfs/xfs_iomap.c | 19 ++++--------------- fs/xfs/xfs_rw.h | 22 ++++++++++++++++++++++ fs/xfs/xfs_vnodeops.c | 17 +++++------------ 4 files changed, 42 insertions(+), 49 deletions(-) Index: 2.6.x-xfs-new/fs/xfs/xfs_vnodeops.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_vnodeops.c 2007-05-31 17:07:38.421796043 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_vnodeops.c 2007-05-31 17:14:29.188303231 +1000 @@ -197,9 +197,8 @@ xfs_getattr( * realtime extent size or the realtime volume's * extent size. */ - vap->va_blocksize = ip->i_d.di_extsize ? - (ip->i_d.di_extsize << mp->m_sb.sb_blocklog) : - (mp->m_sb.sb_rextsize << mp->m_sb.sb_blocklog); + vap->va_blocksize = + xfs_get_extsz_hint(ip) << mp->m_sb.sb_blocklog; } break; } @@ -4094,22 +4093,16 @@ xfs_alloc_file_space( if (XFS_FORCED_SHUTDOWN(mp)) return XFS_ERROR(EIO); - rt = XFS_IS_REALTIME_INODE(ip); - if (unlikely(rt)) { - if (!(extsz = ip->i_d.di_extsize)) - extsz = mp->m_sb.sb_rextsize; - } else { - extsz = ip->i_d.di_extsize; - } - if ((error = XFS_QM_DQATTACH(mp, ip, 0))) return error; if (len <= 0) return XFS_ERROR(EINVAL); + rt = XFS_IS_REALTIME_INODE(ip); + extsz = xfs_get_extsz_hint(ip); + count = len; - error = 0; imapp = &imaps[0]; nimaps = 1; bmapi_flag = XFS_BMAPI_WRITE | (alloc_type ? XFS_BMAPI_PREALLOC : 0); Index: 2.6.x-xfs-new/fs/xfs/xfs_rw.h =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_rw.h 2007-05-31 17:07:37.000000000 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_rw.h 2007-05-31 17:36:31.711921349 +1000 @@ -77,6 +77,28 @@ xfs_fsb_to_db_io(struct xfs_iocore *io, #define XFS_FREE_EOF_LOCK (1<<0) #define XFS_FREE_EOF_NOLOCK (1<<1) + +/* + * helper function to extract extent size hint from inode + */ +STATIC_INLINE xfs_extlen_t +xfs_get_extsz_hint( + xfs_inode_t *ip) +{ + xfs_extlen_t extsz; + + if (unlikely(ip->i_d.di_flags & XFS_DIFLAG_REALTIME)) { + extsz = (ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE) + ? ip->i_d.di_extsize + : ip->i_mount->m_sb.sb_rextsize; + ASSERT(extsz); + } else { + extsz = (ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE) + ? ip->i_d.di_extsize : 0; + } + return extsz; +} + /* * Prototypes for functions in xfs_rw.c. */ Index: 2.6.x-xfs-new/fs/xfs/xfs_bmap.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_bmap.c 2007-05-29 16:40:12.000000000 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_bmap.c 2007-05-31 17:38:24.429227867 +1000 @@ -2618,8 +2618,7 @@ xfs_bmap_rtalloc( xfs_rtblock_t rtb; mp = ap->ip->i_mount; - align = ap->ip->i_d.di_extsize ? - ap->ip->i_d.di_extsize : mp->m_sb.sb_rextsize; + align = xfs_get_extsz_hint(ap->ip); prod = align / mp->m_sb.sb_rextsize; error = xfs_bmap_extsize_align(mp, ap->gotp, ap->prevp, align, 1, ap->eof, 0, @@ -2727,9 +2726,7 @@ xfs_bmap_btalloc( if (!args) return XFS_ERROR(ENOMEM); mp = ap->ip->i_mount; - align = (ap->userdata && ap->ip->i_d.di_extsize && - (ap->ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE)) ? - ap->ip->i_d.di_extsize : 0; + align = ap->userdata ? xfs_get_extsz_hint(ap->ip) : 0; if (unlikely(align)) { error = xfs_bmap_extsize_align(mp, ap->gotp, ap->prevp, align, 0, ap->eof, 0, ap->conv, @@ -2829,9 +2826,9 @@ xfs_bmap_btalloc( args->total = ap->total; args->minlen = ap->minlen; } - if (unlikely(ap->userdata && ap->ip->i_d.di_extsize && - (ap->ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE))) { - args->prod = ap->ip->i_d.di_extsize; + /* apply extent size hints if obtained earlier */ + if (unlikely(align)) { + args->prod = align; if ((args->mod = (xfs_extlen_t)do_mod(ap->off, args->prod))) args->mod = (xfs_extlen_t)(args->prod - args->mod); } else if (mp->m_sb.sb_blocksize >= NBPP) { @@ -3018,9 +3015,7 @@ xfs_bmap_filestreams( */ mp = ap->ip->i_mount; rt = (ap->ip->i_d.di_flags & XFS_DIFLAG_REALTIME) && ap->userdata; - align = (ap->userdata && ap->ip->i_d.di_extsize && - (ap->ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE)) ? - ap->ip->i_d.di_extsize : 0; + align = ap->userdata ? xfs_get_extsz_hint(ap->ip) : 0; if (align) { error = xfs_bmap_extsize_align(mp, ap->gotp, ap->prevp, align, rt, @@ -3166,9 +3161,9 @@ xfs_bmap_filestreams( args.total = ap->total; args.minlen = ap->minlen; } - if (ap->userdata && ap->ip->i_d.di_extsize && - (ap->ip->i_d.di_flags & XFS_DIFLAG_EXTSIZE)) { - args.prod = ap->ip->i_d.di_extsize; + /* apply extent size hint if found earlier */ + if (align) { + args.prod = align; if ((args.mod = (xfs_extlen_t)(do_mod(ap->off, args.prod)))) args.mod = (xfs_extlen_t)(args.prod - args.mod); } else if (mp->m_sb.sb_blocksize >= NBPP) { @@ -5224,12 +5219,7 @@ xfs_bmapi( xfs_extlen_t extsz; /* Figure out the extent size, adjust alen */ - if (rt) { - if (!(extsz = ip->i_d.di_extsize)) - extsz = mp->m_sb.sb_rextsize; - } else { - extsz = ip->i_d.di_extsize; - } + extsz = xfs_get_extsz_hint(ip); if (extsz) { error = xfs_bmap_extsize_align(mp, &got, &prev, extsz, @@ -6170,8 +6160,7 @@ xfs_getbmap( ip->i_d.di_format != XFS_DINODE_FMT_LOCAL) return XFS_ERROR(EINVAL); if (whichfork == XFS_DATA_FORK) { - if ((ip->i_d.di_extsize && (ip->i_d.di_flags & - (XFS_DIFLAG_REALTIME|XFS_DIFLAG_EXTSIZE))) || + if (xfs_get_extsz_hint(ip) || ip->i_d.di_flags & (XFS_DIFLAG_PREALLOC|XFS_DIFLAG_APPEND)){ prealloced = 1; fixlen = XFS_MAXIOFFSET(mp); Index: 2.6.x-xfs-new/fs/xfs/xfs_iomap.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_iomap.c 2007-05-31 17:07:38.000000000 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_iomap.c 2007-05-31 17:30:22.096110172 +1000 @@ -451,19 +451,14 @@ xfs_iomap_write_direct( return XFS_ERROR(error); rt = XFS_IS_REALTIME_INODE(ip); - if (unlikely(rt)) { - if (!(extsz = ip->i_d.di_extsize)) - extsz = mp->m_sb.sb_rextsize; - } else { - extsz = ip->i_d.di_extsize; - } + extsz = xfs_get_extsz_hint(ip); isize = ip->i_size; if (io->io_new_size > isize) isize = io->io_new_size; - offset_fsb = XFS_B_TO_FSBT(mp, offset); - last_fsb = XFS_B_TO_FSB(mp, ((xfs_ufsize_t)(offset + count))); + offset_fsb = XFS_B_TO_FSBT(mp, offset); + last_fsb = XFS_B_TO_FSB(mp, ((xfs_ufsize_t)(offset + count))); if ((offset + count) > isize) { error = xfs_iomap_eof_align_last_fsb(mp, io, isize, extsz, &last_fsb); @@ -666,13 +661,7 @@ xfs_iomap_write_delay( if (error) return XFS_ERROR(error); - if (XFS_IS_REALTIME_INODE(ip)) { - if (!(extsz = ip->i_d.di_extsize)) - extsz = mp->m_sb.sb_rextsize; - } else { - extsz = ip->i_d.di_extsize; - } - + extsz = xfs_get_extsz_hint(ip); offset_fsb = XFS_B_TO_FSBT(mp, offset); retry: From owner-xfs@oss.sgi.com Sun Jun 3 22:24:14 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 22:24:16 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l545OAWt006541 for ; Sun, 3 Jun 2007 22:24:13 -0700 Received: from [134.14.55.89] (soarer.melbourne.sgi.com [134.14.55.89]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id PAA04971; Mon, 4 Jun 2007 15:24:06 +1000 Message-ID: <4663A283.2030205@sgi.com> Date: Mon, 04 Jun 2007 15:26:27 +1000 From: Vlad Apostolov User-Agent: Thunderbird 1.5.0.10 (X11/20070221) MIME-Version: 1.0 To: David Chinner CC: xfs-dev , xfs-oss Subject: Re: Review: xfs_bmapi does not update previous extent pointer correctly References: <20070604051944.GQ85884050@sgi.com> In-Reply-To: <20070604051944.GQ85884050@sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 11603 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: vapo@sgi.com Precedence: bulk X-list: xfs It is looking good Dave, Regards, Vlad David Chinner wrote: > When looping across multiple extents, xfs_bmapi will fail to > update the previous extent pointer which is used in subsequent > loops. > > As a result, we can end up with the second loop in xfs_bmapi trying > to use an incorrect previous extent pointer and assert failures or > corrupted in-memory extent lists will result. > > Correctly update the previous extent at the end of each loop so that > we DTRT when processing multiple map requests. > > Cheers, > > Dave. > From owner-xfs@oss.sgi.com Sun Jun 3 23:13:25 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 23:13:28 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l546DLWt025059 for ; Sun, 3 Jun 2007 23:13:23 -0700 Received: from boing.melbourne.sgi.com (boing.melbourne.sgi.com [134.14.55.141]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id QAA05911; Mon, 4 Jun 2007 16:13:16 +1000 Date: Mon, 04 Jun 2007 16:13:12 +1000 From: Timothy Shimmin To: David Chinner , xfs-dev cc: xfs-oss Subject: Re: Review: Be smarter about handling ENOSPC during writeback Message-ID: In-Reply-To: <20070604045219.GG86004887@sgi.com> References: <20070604045219.GG86004887@sgi.com> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-archive-position: 11604 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: tes@sgi.com Precedence: bulk X-list: xfs As previously discussed, the idea sounds reasonable to me. I'll look at the patch shortly. --Tim --On 4 June 2007 2:52:19 PM +1000 David Chinner wrote: > > During delayed allocation extent conversion or unwritten extent > conversion, we need to reserve some blocks for transactions > reservations. We need to reserve these blocks in case a btree > split occurs and we need to allocate some blocks. > > Unfortunately, we've only ever reserved the number of data blocks we > are allocating, so in both the unwritten and delalloc case we can > get ENOSPC to the transaction reservation. This is bad because in > both cases we cannot report the failure to the writing application. > > The fix is two-fold: > > 1 - leverage the reserved block infrastructure XFS already > has to reserve a small pool of blocks by default to allow > specially marked transactions to dip into when we are at > ENOSPC. > > Default setting is min(5%, 1024 blocks). > > 2 - convert critical transaction reservations to be allowed > to dip into this pool. Spots changed are delalloc > conversion, unwritten extent conversion and growing a > filesystem at ENOSPC. > > Comments? > > Cheers, > > Dave. > -- > Dave Chinner > Principal Engineer > SGI Australian Software Group > > --- > fs/xfs/xfs_fsops.c | 10 +++++++--- > fs/xfs/xfs_iomap.c | 22 ++++++++-------------- > fs/xfs/xfs_mount.c | 37 +++++++++++++++++++++++++++++++++++-- > 3 files changed, 50 insertions(+), 19 deletions(-) > > Index: 2.6.x-xfs-new/fs/xfs/xfs_fsops.c > =================================================================== > --- 2.6.x-xfs-new.orig/fs/xfs/xfs_fsops.c 2007-05-11 10:35:29.288847149 +1000 > +++ 2.6.x-xfs-new/fs/xfs/xfs_fsops.c 2007-05-11 11:13:34.195363437 +1000 > @@ -179,6 +179,7 @@ xfs_growfs_data_private( > up_write(&mp->m_peraglock); > } > tp = xfs_trans_alloc(mp, XFS_TRANS_GROWFS); > + tp->t_flags |= XFS_TRANS_RESERVE; > if ((error = xfs_trans_reserve(tp, XFS_GROWFS_SPACE_RES(mp), > XFS_GROWDATA_LOG_RES(mp), 0, 0, 0))) { > xfs_trans_cancel(tp, 0); > @@ -500,8 +501,9 @@ xfs_reserve_blocks( > unsigned long s; > > /* If inval is null, report current values and return */ > - > if (inval == (__uint64_t *)NULL) { > + if (!outval) > + return EINVAL; > outval->resblks = mp->m_resblks; > outval->resblks_avail = mp->m_resblks_avail; > return 0; > @@ -564,8 +566,10 @@ retry: > } > } > out: > - outval->resblks = mp->m_resblks; > - outval->resblks_avail = mp->m_resblks_avail; > + if (outval) { > + outval->resblks = mp->m_resblks; > + outval->resblks_avail = mp->m_resblks_avail; > + } > XFS_SB_UNLOCK(mp, s); > > if (fdblks_delta) { > Index: 2.6.x-xfs-new/fs/xfs/xfs_mount.c > =================================================================== > --- 2.6.x-xfs-new.orig/fs/xfs/xfs_mount.c 2007-05-11 10:35:29.292846630 +1000 > +++ 2.6.x-xfs-new/fs/xfs/xfs_mount.c 2007-05-11 11:13:47.229662318 +1000 > @@ -718,7 +718,7 @@ xfs_mountfs( > bhv_vnode_t *rvp = NULL; > int readio_log, writeio_log; > xfs_daddr_t d; > - __uint64_t ret64; > + __uint64_t resblks; > __int64_t update_flags; > uint quotamount, quotaflags; > int agno; > @@ -835,6 +835,7 @@ xfs_mountfs( > */ > if ((mfsi_flags & XFS_MFSI_SECOND) == 0 && > (mp->m_flags & XFS_MOUNT_NOUUID) == 0) { > + __uint64_t ret64; > if (xfs_uuid_mount(mp)) { > error = XFS_ERROR(EINVAL); > goto error1; > @@ -1127,13 +1128,27 @@ xfs_mountfs( > goto error4; > } > > - > /* > * Complete the quota initialisation, post-log-replay component. > */ > if ((error = XFS_QM_MOUNT(mp, quotamount, quotaflags, mfsi_flags))) > goto error4; > > + /* > + * Now we are mounted, reserve a small amount of unused space for > + * privileged transactions. This is needed so that transaction > + * space required for critical operations can dip into this pool > + * when at ENOSPC. This is needed for operations like create with > + * attr, unwritten extent conversion at ENOSPC, etc. Data allocations > + * are not allowed to use this reserved space. > + * > + * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. > + * This may drive us straight to ENOSPC on mount, but that implies > + * we were already there on the last unmount. > + */ > + resblks = min_t(__uint64_t, mp->m_sb.sb_dblocks / 20, 1024); > + xfs_reserve_blocks(mp, &resblks, NULL); > + > return 0; > > error4: > @@ -1172,6 +1187,7 @@ xfs_unmountfs(xfs_mount_t *mp, struct cr > #if defined(DEBUG) || defined(INDUCE_IO_ERROR) > int64_t fsid; > #endif > + __uint64_t resblks; > > /* > * We can potentially deadlock here if we have an inode cluster > @@ -1200,6 +1216,23 @@ xfs_unmountfs(xfs_mount_t *mp, struct cr > xfs_binval(mp->m_rtdev_targp); > } > > + /* > + * Unreserve any blocks we have so that when we unmount we don't account > + * the reserved free space as used. This is really only necessary for > + * lazy superblock counting because it trusts the incore superblock > + * counters to be aboslutely correct on clean unmount. > + * > + * We don't bother correcting this elsewhere for lazy superblock > + * counting because on mount of an unclean filesystem we reconstruct the > + * correct counter value and this is irrelevant. > + * > + * For non-lazy counter filesystems, this doesn't matter at all because > + * we only every apply deltas to the superblock and hence the incore > + * value does not matter.... > + */ > + resblks = 0; > + xfs_reserve_blocks(mp, &resblks, NULL); > + > xfs_log_sbcount(mp, 1); > xfs_unmountfs_writesb(mp); > xfs_unmountfs_wait(mp); /* wait for async bufs */ > Index: 2.6.x-xfs-new/fs/xfs/xfs_iomap.c > =================================================================== > --- 2.6.x-xfs-new.orig/fs/xfs/xfs_iomap.c 2007-05-11 11:13:13.862017149 +1000 > +++ 2.6.x-xfs-new/fs/xfs/xfs_iomap.c 2007-05-11 11:13:34.199362915 +1000 > @@ -489,13 +489,13 @@ xfs_iomap_write_direct( > if (unlikely(rt)) { > resrtextents = qblocks = resaligned; > resrtextents /= mp->m_sb.sb_rextsize; > - resblks = XFS_DIOSTRAT_SPACE_RES(mp, 0); > - quota_flag = XFS_QMOPT_RES_RTBLKS; > - } else { > - resrtextents = 0; > + resblks = XFS_DIOSTRAT_SPACE_RES(mp, 0); > + quota_flag = XFS_QMOPT_RES_RTBLKS; > + } else { > + resrtextents = 0; > resblks = qblocks = XFS_DIOSTRAT_SPACE_RES(mp, resaligned); > - quota_flag = XFS_QMOPT_RES_REGBLKS; > - } > + quota_flag = XFS_QMOPT_RES_REGBLKS; > + } > > /* > * Allocate and setup the transaction > @@ -788,18 +788,12 @@ xfs_iomap_write_allocate( > nimaps = 0; > while (nimaps == 0) { > tp = xfs_trans_alloc(mp, XFS_TRANS_STRAT_WRITE); > + tp->t_flags |= XFS_TRANS_RESERVE; > nres = XFS_EXTENTADD_SPACE_RES(mp, XFS_DATA_FORK); > error = xfs_trans_reserve(tp, nres, > XFS_WRITE_LOG_RES(mp), > 0, XFS_TRANS_PERM_LOG_RES, > XFS_WRITE_LOG_COUNT); > - if (error == ENOSPC) { > - error = xfs_trans_reserve(tp, 0, > - XFS_WRITE_LOG_RES(mp), > - 0, > - XFS_TRANS_PERM_LOG_RES, > - XFS_WRITE_LOG_COUNT); > - } > if (error) { > xfs_trans_cancel(tp, 0); > return XFS_ERROR(error); > @@ -917,8 +911,8 @@ xfs_iomap_write_unwritten( > * from unwritten to real. Do allocations in a loop until > * we have covered the range passed in. > */ > - > tp = xfs_trans_alloc(mp, XFS_TRANS_STRAT_WRITE); > + tp->t_flags |= XFS_TRANS_RESERVE; > error = xfs_trans_reserve(tp, resblks, > XFS_WRITE_LOG_RES(mp), 0, > XFS_TRANS_PERM_LOG_RES, From owner-xfs@oss.sgi.com Sun Jun 3 23:15:25 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 23:15:28 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l546FLWt025769 for ; Sun, 3 Jun 2007 23:15:23 -0700 Received: from chook.melbourne.sgi.com (chook.melbourne.sgi.com [134.14.54.237]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id QAA05952; Mon, 4 Jun 2007 16:15:17 +1000 Received: by chook.melbourne.sgi.com (Postfix, from userid 16346) id AF6EF58C38C1; Mon, 4 Jun 2007 16:15:17 +1000 (EST) To: xfs@oss.sgi.com, sgi.bugs.xfs@engr.sgi.com Subject: TAKE 965631 - xfs_bmapi() fails to update previous extent pointer Message-Id: <20070604061517.AF6EF58C38C1@chook.melbourne.sgi.com> Date: Mon, 4 Jun 2007 16:15:17 +1000 (EST) From: dgc@sgi.com (David Chinner) X-archive-position: 11605 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs xfs_bmapi fails to update the previous extent pointer When processing multiple extent maps, xfs_bmapi needs to keep track of the extent behind the one it is currently working on to be able to trim extent ranges correctly. Failing to update the previous pointer can result in corrupted extent lists in memory and this will result in panics or assert failures. Update the previous pointer correctly when we move to the next extent to process. Date: Mon Jun 4 16:14:47 AEST 2007 Workarea: chook.melbourne.sgi.com:/build/dgc/isms/2.6.x-xfs Inspected by: vapo@sgi.com The following file(s) were checked into: longdrop.melbourne.sgi.com:/isms/linux/2.6.x-xfs-melb Modid: xfs-linux-melb:xfs-kern:28773a fs/xfs/xfs_bmap.c - 1.368 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-linux/xfs_bmap.c.diff?r1=text&tr1=1.368&r2=text&tr2=1.367&f=h - Update the previous extent pointer correctly in xfs_bmapi. From owner-xfs@oss.sgi.com Sun Jun 3 23:33:36 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 23:33:38 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l546XXWt030048 for ; Sun, 3 Jun 2007 23:33:35 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id QAA06451; Mon, 4 Jun 2007 16:33:29 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l546XSAf113366370; Mon, 4 Jun 2007 16:33:29 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l546XSXf113386261; Mon, 4 Jun 2007 16:33:28 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 16:33:28 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss , asg-qa Subject: Review: fix test 004 to account for reserved space Message-ID: <20070604063328.GT85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11606 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs With the changes to use some space by default in only in memory as a reserved pool, df and statfs will now output a fre block count that is slightly different to what is held in the superblock. Update the qa test to account for this change. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- xfstests/004 | 35 +++++++++++++++++++++++++---------- 1 file changed, 25 insertions(+), 10 deletions(-) Index: xfs-cmds/xfstests/004 =================================================================== --- xfs-cmds.orig/xfstests/004 2006-11-14 19:57:39.000000000 +1100 +++ xfs-cmds/xfstests/004 2007-05-04 16:38:03.957537306 +1000 @@ -67,21 +67,36 @@ xfs_db -r -c "freesp -s" $SCRATCH_DEV >$ echo "xfs_db for $SCRATCH_DEV" >>$seq.full cat $tmp.xfs_db >>$seq.full +eval `$XFS_IO_PROG -x -c resblks $SCRATCH_MNT 2>&1 \ + | $AWK_PROG '/available/ { printf "resblks=%u\n", $5 }'` +echo "resblks gave: resblks=$resblks" >>$seq.full + # check the 'blocks' field from freesp command is OK # since 2.6.18, df does not report the 4 blocks per AG that cannot # be allocated, hence we check for that exact mismatch. +# since ~2.6.22, reserved blocks are used by default and df does +# not report them, hence check for an exact mismatch. perl -ne ' - BEGIN { $avail ='$avail' * 512; - $answer="(no xfs_db free blocks line?)" } - /free blocks (\d+)$/ || next; - $freesp = $1 * '$dbsize'; - if ($freesp == $avail) { $answer = "yes"; } - else { + BEGIN { $avail ='$avail' * 512; + $answer="(no xfs_db free blocks line?)" } + /free blocks (\d+)$/ || next; + $freesp = $1 * '$dbsize'; + if ($freesp == $avail) { + $answer = "yes"; + } else { $avail = $avail + (('$agcount' + 1) * '$dbsize' * 4); - if ($freesp == $avail) { $answer = "yes"; } - else { $answer = "no ($freesp != $avail)"; } - } - END { print "$answer\n" } + if ($freesp == $avail) { + $answer = "yes"; + } else { + $avail = $avail + ('$resblks' * '$dbsize'); + if ($freesp == $avail) { + $answer = "yes"; + } else { + $answer = "no ($freesp != $avail)"; + } + } + } + END { print "$answer\n" } ' <$tmp.xfs_db >$tmp.ans ans="`cat $tmp.ans`" echo "Checking blocks column same as df: $ans" From owner-xfs@oss.sgi.com Sun Jun 3 23:58:46 2007 Received: with ECARTIS (v1.0.0; list xfs); Sun, 03 Jun 2007 23:58:49 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l546whWt008370 for ; Sun, 3 Jun 2007 23:58:45 -0700 Received: from chook.melbourne.sgi.com (chook.melbourne.sgi.com [134.14.54.237]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id QAA06833; Mon, 4 Jun 2007 16:58:39 +1000 Received: by chook.melbourne.sgi.com (Postfix, from userid 16346) id 7054B58C38C1; Mon, 4 Jun 2007 16:58:39 +1000 (EST) To: xfs@oss.sgi.com, sgi.bugs.xfs@engr.sgi.com Subject: TAKE 965630 - xfs should flush the block device on unmount Message-Id: <20070604065839.7054B58C38C1@chook.melbourne.sgi.com> Date: Mon, 4 Jun 2007 16:58:39 +1000 (EST) From: dgc@sgi.com (David Chinner) X-archive-position: 11607 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs Flush the block device before closing it on unmount. Date: Mon Jun 4 16:58:12 AEST 2007 Workarea: chook.melbourne.sgi.com:/build/dgc/isms/2.6.x-xfs Inspected by: hch@infradead.org The following file(s) were checked into: longdrop.melbourne.sgi.com:/isms/linux/2.6.x-xfs-melb Modid: xfs-linux-melb:xfs-kern:28774a fs/xfs/linux-2.6/xfs_buf.c - 1.242 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-linux/linux-2.6/xfs_buf.c.diff?r1=text&tr1=1.242&r2=text&tr2=1.241&f=h - Flush the block device before closing it on unmount. From owner-xfs@oss.sgi.com Mon Jun 4 00:18:47 2007 Received: with ECARTIS (v1.0.0; list xfs); Mon, 04 Jun 2007 00:18:50 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l547IiWt020789 for ; Mon, 4 Jun 2007 00:18:46 -0700 Received: from chook.melbourne.sgi.com (chook.melbourne.sgi.com [134.14.54.237]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id RAA07298; Mon, 4 Jun 2007 17:18:40 +1000 Received: by chook.melbourne.sgi.com (Postfix, from userid 16346) id 341A958C38C1; Mon, 4 Jun 2007 17:18:40 +1000 (EST) To: xfs@oss.sgi.com, sgi.bugs.xfs@engr.sgi.com Subject: TAKE 964092 - synchronous direct I/O write calls are incomplete when returning to user space Message-Id: <20070604071840.341A958C38C1@chook.melbourne.sgi.com> Date: Mon, 4 Jun 2007 17:18:40 +1000 (EST) From: dgc@sgi.com (David Chinner) X-archive-position: 11608 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs Block on unwritten extent conversion during synchronous direct I/O. Currently we do not wait on extent conversion to occur, and hence we can return to userspace from a sycnhronous direct I/O write without having completed all teh actions in the write. Hence a read after the write may see zeroes (unwritten extent) rather than the data that was written. Block the I/O completion by triggering a synchronous workqueue flush to ensure that the conversion has occurred before we return to userspace. Date: Mon Jun 4 17:18:01 AEST 2007 Workarea: chook.melbourne.sgi.com:/build/dgc/isms/2.6.x-xfs Inspected by: tes@sgi.com The following file(s) were checked into: longdrop.melbourne.sgi.com:/isms/linux/2.6.x-xfs-melb Modid: xfs-linux-melb:xfs-kern:28775a fs/xfs/linux-2.6/xfs_aops.c - 1.144 - changed http://oss.sgi.com/cgi-bin/cvsweb.cgi/xfs-linux/linux-2.6/xfs_aops.c.diff?r1=text&tr1=1.144&r2=text&tr2=1.143&f=h - Make unwritten extent conversion occur synchronously for synchronous direct I/O to ensure it is completed before we return to userspace. From owner-xfs@oss.sgi.com Mon Jun 4 01:07:33 2007 Received: with ECARTIS (v1.0.0; list xfs); Mon, 04 Jun 2007 01:07:47 -0700 (PDT) Received: from larry.melbourne.sgi.com (larry.melbourne.sgi.com [134.14.52.130]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with SMTP id l5487SWt011430 for ; Mon, 4 Jun 2007 01:07:31 -0700 Received: from snort.melbourne.sgi.com (snort.melbourne.sgi.com [134.14.54.149]) by larry.melbourne.sgi.com (950413.SGI.8.6.12/950213.SGI.AUTOCF) via ESMTP id SAA08282; Mon, 4 Jun 2007 18:07:22 +1000 Received: from snort.melbourne.sgi.com (localhost [127.0.0.1]) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id l5487MAf113202618; Mon, 4 Jun 2007 18:07:22 +1000 (AEST) Received: (from dgc@localhost) by snort.melbourne.sgi.com (SGI-8.12.5/8.12.5/Submit) id l5487Ksc113426956; Mon, 4 Jun 2007 18:07:20 +1000 (AEST) X-Authentication-Warning: snort.melbourne.sgi.com: dgc set sender to dgc@sgi.com using -f Date: Mon, 4 Jun 2007 18:07:20 +1000 From: David Chinner To: xfs-dev Cc: xfs-oss Subject: Review: apply transaction deltas atomically to superblock Message-ID: <20070604080720.GV85884050@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-archive-position: 11609 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: dgc@sgi.com Precedence: bulk X-list: xfs When testing lazy superblock counters and ENOSPC conditions (test 083), I came across semi-regular assert failures in xfs_mod_incore_sb_batch() where the assert failure was occurring as a result of failing to *undo* block reservations for a transaction that had reserved all the blocks it was going to use up front. That is, we failed to apply the transaction delta when it should not have failed, and then we failed to remove the delta's that we had already applied. It turns out that that the problem is an interaction between the per-cpu superblock counters and xfs_trans_unreserve_and_mod_sb(). Prior to the per-cpu superblock counters, transaction delta's were applied under the XFS_SB_LOCK() and so were always applied atomically. The per-cpu superblock counters don't hold the XFS_SB_LOCK() and hence are not applied atomically. This was not thought to be a problem because each cahnge that needed ot be made had already been validated and reserved. It turns out that xfs_trans_unreserve_and_mod_sb() does something incredibly stupid. It applies changes to the free block in *two* separate deltas. The first change puts back the *entire reservation* to the superblock and then it takes away what was actually used. So now we have a window where the transaction reservation is undone and another thread can come along and use that reservation. the result is the second delta to mark the blocks as used fails with ENOSPC, and because the blocks need for the transaction reservation has been taken by something else, we then fail to get them back for the transaction reservation when we try to undo that delta. So, the fix is to simply calculate what the free block delta is and apply it in a single atomic delta. Cheers, Dave. -- Dave Chinner Principal Engineer SGI Australian Software Group --- fs/xfs/xfs_trans.c | 77 +++++++++++++++++++++++++++-------------------------- 1 file changed, 40 insertions(+), 37 deletions(-) Index: 2.6.x-xfs-new/fs/xfs/xfs_trans.c =================================================================== --- 2.6.x-xfs-new.orig/fs/xfs/xfs_trans.c 2007-05-03 15:05:09.000000000 +1000 +++ 2.6.x-xfs-new/fs/xfs/xfs_trans.c 2007-05-09 12:14:18.429409061 +1000 @@ -638,11 +638,23 @@ xfs_trans_apply_sb_deltas( } /* - * xfs_trans_unreserve_and_mod_sb() is called to release unused - * reservations and apply superblock counter changes to the in-core - * superblock. + * xfs_trans_unreserve_and_mod_sb() is called to release unused reservations + * and apply superblock counter changes to the in-core superblock. The + * t_res_fdblocks_delta and t_res_frextents_delta fields are explicitly NOT + * applied to the in-core superblock. The idea is that that has