From BATV+c2a8630ad946b3a996ee+2353+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 1 05:00:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o11B0lHk102574 for ; Mon, 1 Feb 2010 05:00:54 -0600 X-ASG-Debug-ID: 1265022114-1bdd02640000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BE5C813695CC for ; Mon, 1 Feb 2010 03:01:55 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id WHxJ1kckxRg7Fq0O for ; Mon, 01 Feb 2010 03:01:55 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nbu2U-0000Gp-1j; Mon, 01 Feb 2010 11:01:54 +0000 Date: Mon, 1 Feb 2010 06:01:54 -0500 From: Christoph Hellwig To: Nick Piggin Cc: Christoph Hellwig , xfs@oss.sgi.com, linux-mm@kvack.org X-ASG-Orig-Subj: Re: [patch 2/2] xfs: use scalable vmap API Subject: Re: [patch 2/2] xfs: use scalable vmap API Message-ID: <20100201110153.GA588@infradead.org> References: <20081021082735.GB6974@wotan.suse.de> <20081021120932.GB13348@infradead.org> <20081022093018.GD4359@wotan.suse.de> <20100119121505.GA9428@infradead.org> <20100125075445.GD19664@laptop> <20100125081750.GA20012@infradead.org> <20100125083309.GF19664@laptop> <20100125123746.GA24406@laptop> <20100125213403.GA1309@infradead.org> <20100127083819.GA11072@laptop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100127083819.GA11072@laptop> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265022115 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Jan 27, 2010 at 07:38:19PM +1100, Nick Piggin wrote: > > So far I've not run out of vmalloc space yet with quite a few xfstests > > iterations and not encountered any other problems either. > > > > Thanks for looking into this! > > OK thanks for testing. I'll send it upstream if you haven't had any > problems so far. Still working fine, so please send it upstream ASAP. That'll make re-eabling the scalable vmap API in XFS much more easier for 2.6.34. From npiggin@suse.de Mon Feb 1 05:27:20 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.4 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o11BRJtv104831 for ; Mon, 1 Feb 2010 05:27:20 -0600 X-ASG-Debug-ID: 1265023706-6dcd03d20000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx2.suse.de (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 048881C95CFF for ; Mon, 1 Feb 2010 03:28:27 -0800 (PST) Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15]) by cuda.sgi.com with ESMTP id TRayOJ7yMFSWbx6c for ; Mon, 01 Feb 2010 03:28:27 -0800 (PST) Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.221.2]) by mx2.suse.de (Postfix) with ESMTP id 4E27486EE4; Mon, 1 Feb 2010 12:28:26 +0100 (CET) Received: by laptop.local0.net (Postfix, from userid 1000) id 1354A29856; Mon, 1 Feb 2010 22:28:14 +1100 (EST) Date: Mon, 1 Feb 2010 22:28:14 +1100 From: Nick Piggin To: Christoph Hellwig Cc: xfs@oss.sgi.com, linux-mm@kvack.org X-ASG-Orig-Subj: Re: [patch 2/2] xfs: use scalable vmap API Subject: Re: [patch 2/2] xfs: use scalable vmap API Message-ID: <20100201112813.GA17689@laptop> References: <20081021120932.GB13348@infradead.org> <20081022093018.GD4359@wotan.suse.de> <20100119121505.GA9428@infradead.org> <20100125075445.GD19664@laptop> <20100125081750.GA20012@infradead.org> <20100125083309.GF19664@laptop> <20100125123746.GA24406@laptop> <20100125213403.GA1309@infradead.org> <20100127083819.GA11072@laptop> <20100201110153.GA588@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100201110153.GA588@infradead.org> User-Agent: Mutt/1.5.20 (2009-06-14) X-Barracuda-Connect: cantor2.suse.de[195.135.220.15] X-Barracuda-Start-Time: 1265023708 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21356 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Feb 01, 2010 at 06:01:54AM -0500, Christoph Hellwig wrote: > On Wed, Jan 27, 2010 at 07:38:19PM +1100, Nick Piggin wrote: > > > So far I've not run out of vmalloc space yet with quite a few xfstests > > > iterations and not encountered any other problems either. > > > > > > Thanks for looking into this! > > > > OK thanks for testing. I'll send it upstream if you haven't had any > > problems so far. > > Still working fine, so please send it upstream ASAP. That'll make > re-eabling the scalable vmap API in XFS much more easier for 2.6.34. Done. Thanks for testing this. From patrick@news-service.com Mon Feb 1 10:51:31 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o11GpUYA121572 for ; Mon, 1 Feb 2010 10:51:31 -0600 X-ASG-Debug-ID: 1265043155-781b00c20000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from pu01.news-service.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 38A141A1DEF for ; Mon, 1 Feb 2010 08:52:35 -0800 (PST) Received: from pu01.news-service.com (ns1.news-service.com [195.114.240.3]) by cuda.sgi.com with ESMTP id 9Zs2tj1qs8uV2RY4 for ; Mon, 01 Feb 2010 08:52:35 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by pu01.news-service.com (Postfix) with ESMTP id 6531D13E2C; Mon, 1 Feb 2010 17:52:34 +0100 (CET) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Debian amavisd-new at pu01.news-service.com Received: from pu01.news-service.com ([127.0.0.1]) by localhost (pu01.nse [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yAZm-KzBZbHj; Mon, 1 Feb 2010 17:52:31 +0100 (CET) Received: from [172.25.4.10] (fw01.nse [172.25.8.1]) by pu01.news-service.com (Postfix) with ESMTP id 5977313E24; Mon, 1 Feb 2010 17:52:31 +0100 (CET) Message-ID: <4B6706CE.1020207@news-service.com> Date: Mon, 01 Feb 2010 17:52:30 +0100 From: Patrick Schreurs User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Dave Chinner CC: Christoph Hellwig , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) References: <89c4f90c0910270341r7833f490g60810f2817eb0950@mail.gmail.com> <89c4f90c0910280519k759230c1r7b1586932ac792f7@mail.gmail.com> <20091030101601.GA11142@infradead.org> <4AF0422D.1070104@news-service.com> <20091114162126.GB17658@infradead.org> <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> In-Reply-To: <4B504B03.7050604@news-service.com> Content-Type: multipart/mixed; boundary="------------090200000407090003070100" X-Barracuda-Connect: ns1.news-service.com[195.114.240.3] X-Barracuda-Start-Time: 1265043157 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21377 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean This is a multi-part message in MIME format. --------------090200000407090003070100 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hello Dave, I'm afraid we had another crash a few days ago. Have a look at the screenshot. Can you make anything out of it? This server is running 2.6.32.3 with your inode-reclaim patch applied and XFS_DEBUG still enabled. Thanks for looking into this. -Patrick On 15-1-2010 12:01, Patrick Schreurs wrote: > Hi Dave, > > I think it's save to consider this issue fixed. We have currently 9 > servers operational with these patches and they have been stable so far. > For a 100% certainty we'll have to test/wait a little bit longer, but > considering the frequency of crashes we saw earlier i think it's save to > come to a conclusion. > > I hope these patches will be included in 2.6.33 and will be back ported > to at least 2.6.32. > > Many thanks to Dave and to Christoph for fixing this apparently rare and > seldom triggered condition. > > -Patrick > > Dave Chinner wrote: >> Hi Patrick, >> >> I've attached two compendium patches that will hopefully fix >> the inode reclaim problems you've been seeing - one is for 2.6.31, >> the other is for 2.6.32. I've cc'd this to the XFS list Ñ•o that >> anyone else who has been seeing crashes, assert failures and >> general nastiness around inode reclaim can test them as well. >> >> These are not final patches - there's a few changes that Christoph >> has picked up on during review - so there'll be another round of >> patches before checkins and -stable backports can be requested. >> >> I'm hoping that these patches fix your problem, because with them >> I can't make my machines fall over anymore.... >> >> Cheers, >> >> Dave. >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs --------------090200000407090003070100 Content-Type: image/jpeg; name="sb06-20100128.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sb06-20100128.jpg" /9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRof Hh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwh MjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAAR CAHgAoADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAA AgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkK FhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWG h4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl 5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREA AgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYk NOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOE hYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk 5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDz3U9Qik8NzW6alaN4iFugv7lZVJuIct+6 WXo7gGPdg5bGMtjFHizX5YrHT4YtSnnuJNOSC4aDUkliL4IkEkYDbmIY/NuGcgj7tGp6hFJ4 bmt01K0bxELdBf3KyqTcQ5b90svR3AMe7By2MZbGKPFmvyxWOnwxalPPcSackFw0GpJLEXwR IJIwG3MQx+bcM5BH3aSAl1TUUvLLUml1K3t4HsVEUVtexzWrsAmEjtmUPHnHXgoQTXnleh6p qKXllqTS6lb28D2KiKK2vY5rV2ATCR2zKHjzjrwUIJrzyhbAd/qWneCkbVbGzdfNtrV2guvt u4SOioykfwszl2UgZx5fAGar6hp3hqMamIDZf2fHZh7G7S8LXMs2EwGj3nGSWBHlrj2qxqWn eCkbVbGzdfNtrV2guvtu4SOioykfwszl2UgZx5fAGar6hp3hqMamIDZf2fHZh7G7S8LXMs2E wGj3nGSWBHlrj2pLoA3V9M8Nhdek06S08to4ZdLH2vnaNvmggtkNz91/mPOBwaSez8NSxzKg tITJo4vVeO6YmO64/cruYjHB+UgtyeemF1fTPDYXXpNOktPLaOGXSx9r52jb5oILZDc/df5j zgcGkns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnphrYA1K7sNV8EaesI0u1ksxOXia aQPExkUqsalizbgepDAc8riqDx2zfDtCJoFu1v8AeYRefM8e0jeYi+N244yFBwAemSb+pXdh qvgjT1hGl2slmJy8TTSB4mMilVjUsWbcD1IYDnlcVQeO2b4doRNAt2t/vMIvPmePaRvMRfG7 ccZCg4APTJIBfns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnpi+/h/Qn07Ub6OyeKxj 04y2ctx58cjSFVwZCw8tmLHgI2Dno2QRQns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcn npi+/h/Qn07Ub6OyeKxj04y2ctx58cjSFVwZCw8tmLHgI2Dno2QQAeeV3OqajZWXgWwtdMKp 9oEong+2QzMu5lZTKoTLNtXhgFKYAzniuGrudU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWb avDAKUwBnPFMDLeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJIksL/DV7cTwfaE1Xzj CZVDlPKC7gpOSMnt7+hoeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJIksL/DV7cTwf aE1XzjCZVDlPKC7gpOSMnt7+hoA1dZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjN VL6x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1FvWbDwlbWV2bILPCIF+yzxTp5pchc M4M2TzkMPKUjJ6YzVS+sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329QkBfm8PaJI2sa rFbTw6W2nNPYxzRzIIpNq7dztgMxY/KAXByeeBmhfWOhS6FLPZraWci26OgmnEsrN8uVykx+ Y89YlA77eovzeHtEkbWNVitp4dLbTmnsY5o5kEUm1du52wGYsflALg5PPAzQvrHQpdClns1t LORbdHQTTiWVm+XK5SY/MeesSgd9vUCAl1K7sNV8EaesI0u1ksxOXiaaQPExkUqsalizbgep DAc8riptZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjNQ6ld2Gq+CNPWEaXayWYnL xNNIHiYyKVWNSxZtwPUhgOeVxU2s2HhK2srs2QWeEQL9lninTzS5C4ZwZsnnIYeUpGT0xmgC XUtO8FI2q2Nm6+bbWrtBdfbdwkdFRlI/hZnLspAzjy+AM1wFd/qWneCkbVbGzdfNtrV2guvt u4SOioykfwszl2UgZx5fAGa4ChbAdffWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eo 17/+yrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk9MHArIvrHQpdClns1tLORbdHQTTi WVm+XK5SY/MeesSgd9vUa9//AGVbeF/EGnaTdWX2J1tpLMm9BluMbWkZkZ/lbjoFUnpg4FAG RfWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eo5Cu01K7sNV8EaesI0u1ksxOXiaaQP ExkUqsalizbgepDAc8riuLpgFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUA FFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRR RQAUUUUAFFFFABRRRQB32p6hFJ4bmt01K0bxELdBf3KyqTcQ5b90svR3AMe7By2MZbGKPFmv yxWOnwxalPPcSackFw0GpJLEXwRIJIwG3MQx+bcM5BH3aNSv4W8NS20eo2jeIRboL+5WVf8A SIAW/dLL0dwDHuwcsBjLYxR4q16SCw0+CHUZpp301IJzBqKSxb8FZA8YDbmIb724diPu0gJd U1FLyy1JpdSt7eB7FRFFbXsc1q7AJhI7ZlDx5x14KEE155Xoeqail5Zak0upW9vA9ioiitr2 Oa1dgEwkdsyh48468FCCa88oWwHf6lp3gpG1Wxs3Xzba1doLr7buEjoqMpH8LM5dlIGceXwB mq+oad4ajGpiA2X9nx2Yexu0vC1zLNhMBo95xklgR5a49qsalp3gpG1Wxs3Xzba1doLr7buE joqMpH8LM5dlIGceXwBmq+oad4ajGpiA2X9nx2Yexu0vC1zLNhMBo95xklgR5a49qS6AN1fT PDYXXpNOktPLaOGXSx9r52jb5oILZDc/df5jzgcGkns/DUscyoLSEyaOL1XjumJjuuP3K7mI xwflILcnnphdX0zw2F16TTpLTy2jhl0sfa+do2+aCC2Q3P3X+Y84HBpJ7Pw1LHMqC0hMmji9 V47piY7rj9yu5iMcH5SC3J56Ya2ANSu7DVfBGnrCNLtZLMTl4mmkDxMZFKrGpYs24HqQwHPK 4qg8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSb+pXdhqvgjT1hGl2slmJy8TTSB4mM ilVjUsWbcD1IYDnlcVQeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJIBfns/DUscyoL SEyaOL1XjumJjuuP3K7mIxwflILcnnpi+/h/Qn07Ub6OyeKxj04y2ctx58cjSFVwZCw8tmLH gI2Dno2QRQns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnpi+/h/Qn07Ub6OyeKxj04y 2ctx58cjSFVwZCw8tmLHgI2Dno2QQAeeV3OqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtX hgFKYAzniuGrudU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFMDLeO2b4doRN At2t/vMIvPmePaRvMRfG7ccZCg4APTJIksL/AA1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaHj tm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0ySJLC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7 e/oaANXWbDwlbWV2bILPCIF+yzxTp5pchcM4M2TzkMPKUjJ6YzVS+sdCl0KWezW0s5Ft0dBN OJZWb5crlJj8x56xKB329Rb1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM1UvrHQ pdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUJAX5vD2iSNrGqxW08OltpzT2Mc0cyCKTau 3c7YDMWPygFwcnngZoX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qL83h7RJG1jVY raeHS205p7GOaOZBFJtXbudsBmLH5QC4OTzwM0L6x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzH nrEoHfb1AgJdSu7DVfBGnrCNLtZLMTl4mmkDxMZFKrGpYs24HqQwHPK4qbWbDwlbWV2bILPC IF+yzxTp5pchcM4M2TzkMPKUjJ6YzUOpXdhqvgjT1hGl2slmJy8TTSB4mMilVjUsWbcD1IYD nlcVNrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHlKRk9MZoAk1HTvBaNqtlaOvnW1ozQXX 23cJHRUZSP4WZy7KQCceXwBmuBrv9S07wUjarY2br5ttau0F19t3CR0VGUj+FmcuykDOPL4A zXAULYDr76x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1Gvf/2VbeF/EGnaTdWX2J1t pLMm9BluMbWkZkZ/lbjoFUnpg4FZF9Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6jX v/7KtvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0wcCgDO1fTPDYXXpNOktPLaOGXSx9 r52jb5oILZDc/df5jzgcGuKrqLweGBbXOp28fFzD5Vtpm999rNgbnZ8/Mo6j+9uxgbTjl6Fs AUUUUwCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACi iigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigDvtT1 CKTw3NbpqVo3iIW6C/uVlUm4hy37pZejuAY92DlsYy2MUeLNflisdPhi1Kee4k05ILhoNSSW IvgiQSRgNuYhj824ZyCPu0alqEUvhuWBNStG8Q/Z0+33IlXNxBlv3SydHcAx7sHLYxlsYo8V 6/LHYafFFqU888mmpBcNDqKSxF8ESCSMBtzEMfmyOxH3aQEuqail5Zak0upW9vA9ioiitr2O a1dgEwkdsyh48468FCCa88r0PVNRS8stSaXUre3gexURRW17HNauwCYSO2ZQ8ecdeChBNeeU LYDv9S07wUjarY2br5ttau0F19t3CR0VGUj+FmcuykDOPL4AzVe/07w0g1IQfYxp8dkHsbtb wtcyzYTAaPecZJYEbBj261PqOneClbVbKzdfNt7Rmguvtm4SOioykcbWZy7KQOnl8AZqDUNO 8NRjUxAbL+z47MPY3aXha5lmwmA0e84ySwI8tce1JdAG6vpnhwJr0unPaBGjgm0sfa/mCjb5 oILZDc/dfk84HBpJ7Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5SC3J56YXV9M8Nhdek06S08 to4ZdLH2vnaNvmggtkNz91/mPOBwaSez8NSxzKgtITJo4vVeO6YmO64/cruYjHB+UgtyeemG tgDUruw1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKoPHbN8O0ImgW7W/3mEXnzPHt I3mIvjduOMhQcAHpkm/qV3Yar4I09YRpdrJZicvE00geJjIpVY1LFm3A9SGA55XFUHjtm+Ha ETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0ySAX57Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5S C3J56Yvv4f0J9O1G+jsnisY9OMtnLcefHI0hVcGQsPLZix4CNg56NkEUJ7Pw1LHMqC0hMmji 9V47piY7rj9yu5iMcH5SC3J56Yvv4f0J9O1G+jsnisY9OMtnLcefHI0hVcGQsPLZix4CNg56 NkEAHnldzqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54rhq7nVNRsrLwLYWum FU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxTAy3jtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQo OAD0ySJLC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaHjtm+HaETQLdrf7zCLz5nj2kbzEXx u3HGQoOAD0ySJLC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaANTWLHwlbWN21kBND9nX7LN HOnnFyFwXBmyechh5QIycYxmqt9Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6i3rNh 4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHlKRk9MZqpfWOhS6FLPZraWci26OgmnEsrN8uVyk x+Y89YlA77eoSAvzeHtEkbWNVitp4dLbTmnsY5o5kEUm1du52wGYsflALg5PPAzQvrHQpdCl ns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUX5vD2iSNrGqxW08OltpzT2Mc0cyCKTau3c7YDM WPygFwcnngZoX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qBAS6ld2Gq+CNPWEaXa yWYnLxNNIHiYyKVWNSxZtwPUhgOeVxUusWPhK2sbtrICaH7Ov2WaOdPOLkLguDNk85DDygRk 4xjNRald2Gq+CNPWEaXayWYnLxNNIHiYyKVWNSxZtwPUhgOeVxU2s2HhK2srs2QWeEQL9lni nTzS5C4ZwZsnnIYeUpGT0xmgCTUdO8Fo2q2Vo6+dbWjNBdfbdwkdFRlI/hZnLspAJx5fAGa4 Gu/1LTvBSNqtjZuvm21q7QXX23cJHRUZSP4WZy7KQM48vgDNcBQtgOvvrHQpdClns1tLORbd HQTTiWVm+XK5SY/MeesSgd9vUa18dKt/C3iDTtKubP7FIltJZlr0GWfG1pGZGf5W46BVJ6YO BWTfWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eo17/APsq28L+INO0m6svsTrbSWZN 6DLcY2tIzIz/ACtx0CqT0wcCgCrqGj+EoZtWvrbUbN7M2RaytFmcyJIyqIz1yTuD5U/dG0ng 8cHXX3tj4fWzvJLf7J/aQtI2NoLsmCJj98xSZ/eOBtOwsQCTgvjA5CmgCiiigAooooAKKKKA CiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoo ooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA77U9Qik8NzW6alaN4iFug v7lZVJuIct+6WXo7gGPdg5bGMtjFHizX5YrHT4YtSnnuJNOSC4aDUkliL4IkEkYDbmIY/NuG cgj7tGp6hFJ4bmt01K0bxELdBf3KyqTcQ5b90svR3AMe7By2MZbGKPFmvyxWOnwxalPPcSac kFw0GpJLEXwRIJIwG3MQx+bcM5BH3aSAk1LUFu7DUTNqkEELWCiOO3vY5raRgEARLZkDxk46 8FCCa89r0PVNRS8stSaXUre3gexURRW17HNauwCYSO2ZQ8ecdeChBNeeULYDvtR07wXGdUsb WRDNbWjNDdC8ysroqMpX+Es5dlKgnHl8AZqC/wBO8NxrqQhNl9gjsg9jeJdk3E02EwGj3nGS WBGwY9utWNS07wUjarY2br5ttau0F19t3CR0VGUj+FmcuykDOPL4AzVfUNO8NRjUxAbL+z47 MPY3aXha5lmwmA0e84ySwI8tce1JdAGarpfh1ItcewmtGUxQS6YftfzFRt87Klshufutyedo 4NJLZeGpIZUX7LEz6ML1ZEuW3R3Qx+5GWI5wflILfMeemHavpnhsLr0mnSWnltHDLpY+187R t80EFshufuv8x5wODST2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykFuTz0w1sAmoXVhqngb TUh/sy3ls1n8yN5pBJGxkUqsaliWLg9SCo55XFUXjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HG QoOAD0yTf1K7sNV8EaesI0u1ksxOXiaaQPExkUqsalizbgepDAc8riqDx2zfDtCJoFu1v95h F58zx7SN5iL43bjjIUHAB6ZJYF6Wy8NSQyov2WJn0YXqyJctujuhj9yMsRzg/KQW+Y89MaDe H9CfTdQvksXiso9NMtpLP58bmQouC5YCNmLHjY2Dno2RihPZ+GpY5lQWkJk0cXqvHdMTHdcf uV3MRjg/KQW5PPTF9/D+hPp2o30dk8VjHpxls5bjz45GkKrgyFh5bMWPARsHPRsghAeeV3Oq ajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAzniuGrudU1GysvAtha6YVT7QJRPB9s hmZdzKymVQmWbavDAKUwBnPFMDLeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJIksL/ AA1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD 0ySJLC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaANXWbDwlbWV2bILPCIF+yzxTp5pchcM4 M2TzkMPKUjJ6YzVS+sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329Rb1mw8JW1ldmyCz wiBfss8U6eaXIXDODNk85DDylIyemM1UvrHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9 vUJAX5PDuiv/AGvqkVvcRaYdNM1is8csYjk2rjc7YDMWPAUsDk9MDNC+sdCl0KWezW0s5Ft0 dBNOJZWb5crlJj8x56xKB329Rfm8PaJI2sarFbTw6W2nNPYxzRzIIpNq7dztgMxY/KAXByee BmhfWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eoEBJqF1Yap4G01If7Mt5bNZ/Mjea QSRsZFKrGpYli4PUgqOeVxU+s2HhK2srs2QWeEQL9lninTzS5C4ZwZsnnIYeUpGT0xmodSu7 DVfBGnrCNLtZLMTl4mmkDxMZFKrGpYs24HqQwHPK4qbWbDwlbWV2bILPCIF+yzxTp5pchcM4 M2TzkMPKUjJ6YzR1Al1LTvBSNqtjZuvm21q7QXX23cJHRUZSP4WZy7KQM48vgDNcBXf6lp3g pG1Wxs3Xzba1doLr7buEjoqMpH8LM5dlIGceXwBmuAoWwHX31joUuhSz2a2lnItujoJpxLKz fLlcpMfmPPWJQO+3qNe//sq28L+INO0m6svsTrbSWZN6DLcY2tIzIz/K3HQKpPTBwKyL6x0K XQpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1Gvf/wBlW3hfxBp2k3Vl9idbaSzJvQZbjG1p GZGf5W46BVJ6YOBQBXk8O6K/9r6pFb3EWmHTTNYrPHLGI5Nq43O2AzFjwFLA5PTAzwNdlr9h pEOiPJZ2ukx3THzWMGpGUwqXAWJR5h8x8HLNgLgHGTzXG0LYAooopgFFFFABRRRQAUUUUAFF FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ AUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAHfanqEUnhua3TUrRvEQt0F/crKpNx Dlv3Sy9HcAx7sHLYxlsYo8Wa/LFY6fDFqU89xJpyQXDQakksRfBEgkjAbcxDH5twzkEfdo1K /hfw3LbJqNo3iEW6C/uVlX/SIAW/dLL0dwDHuwcsBjLYxR4r16WGw0+GLUZp5301IJ2g1FJY t+CsgkjAbcxDH5tw7EfdpAS6pqKXllqTS6lb28D2KiKK2vY5rV2ATCR2zKHjzjrwUIJrzyvQ 9T1FLux1JpdTgggaxURR217HNbSMAmEjtmQPHnHXgoQTXnlC2A7/AFLTvBSNqtjZuvm21q7Q XX23cJHRUZSP4WZy7KQM48vgDNV9Q07w1GNTEBsv7Pjsw9jdpeFrmWbCYDR7zjJLAjy1x7VY 1LTvBSNqtjZuvm21q7QXX23cJHRUZSP4WZy7KQM48vgDNV7/AE7w3GNSEJsvsEdkHsbxLstc TTYTAaPecZJYEbBj260l0Abq+meGwuvSadJaeW0cMulj7XztG3zQQWyG5+6/zHnA4NJPZ+Gp Y5lQWkJk0cXqvHdMTHdcfuV3MRjg/KQW5PPTC6vpnhsLr0mnSWnltHDLpY+187Rt80EFshuf uv8AMecDg0k1l4alimRBaRNJowvVdLpiY7rj9yuWIwcH5SC3J56Ya2ANSu7DVfBGnrCNLtZL MTl4mmkDxMZFKrGpYs24HqQwHPK4qg8ds3w7QiaBbtb/AHmEXnzPHtI3mIvjduOMhQcAHpkm /qV3Yar4I09YRpdrJZicvE00geJjIpVY1LFm3A9SGA55XFUHjtm+HaETQLdrf7zCLz5nj2kb zEXxu3HGQoOAD0ySAX57Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5SC3J56Yvv4f0J9O1G+j snisY9OMtnLcefHI0hVcGQsPLZix4CNg56NkEUJ7Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH 5SC3J56Yvv4f0J9O1G+jsnisY9OMtnLcefHI0hVcGQsPLZix4CNg56NkEAHnldzqmo2Vl4Fs LXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54rhq7jVNRs7LwJp9rppVDcLKLiAXkMxG5lZT KoTLNtXhgFKYAznimBmPHbN8O0ImgW7W/wB5hF58zx7SN5iL43bjjIUHAB6ZJElhf4avbieD 7Qmq+cYTKocp5QXcFJyRk9vf0NDx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIUHAB6ZJElhf4 avbieD7Qmq+cYTKocp5QXcFJyRk9vf0NAGrrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHl KRk9MZqpfWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eot6zYeErayuzZBZ4RAv2WeK dPNLkLhnBmyechh5SkZPTGaqX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qEgL83h 7RJG1jVYraeHS205p7GOaOZBFJtXbudsBmLH5QC4OTzwM0L6x0KXQpZ7NbSzkW3R0E04llZv lyuUmPzHnrEoHfb1F+bw9okjaxqsVtPDpbac09jHNHMgik2rt3O2AzFj8oBcHJ54GaF9Y6FL oUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6gQEupXdhqvgjT1hGl2slmJy8TTSB4mMilVjU sWbcD1IYDnlcVNrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHlKRk9MZqHUruw1XwRp6wjS 7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKm1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylI yemM0AS6lp3gpG1Wxs3Xzba1doLr7buEjoqMpH8LM5dlIGceXwBmuArv9S07wUjarY2br5tt au0F19t3CR0VGUj+FmcuykDOPL4AzXAULYDr76x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzHnr EoHfb1Gvf/2VbeF/EGnaTdWX2J1tpLMm9BluMbWkZkZ/lbjoFUnpg4FZF9Y6FLoUs9mtpZyL bo6CacSys3y5XKTH5jz1iUDvt6jXv/7KtvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0 wcCgDNvNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4ovNF0IeGLl4bzT31CGKGS KSG4Cedx+8G15Sx4PTZGcjgc4ovNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4r R186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vQA86ooopgFFFFABRRRQAUUUUAFF FFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ AUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAHfanqEUnhua3TUrRvEQt0F/crKpNx Dlv3Sy9HcAx7sHLYxlsYo8Wa/LFY6fDFqU89xJpyQXDQakksRfBEgkjAbcxDH5twzkEfdq9q 11oD3GuaRDqMT6bb2BNjbbU8hJVUFWjlDks5LNnjLZIJOKwr6x0KXQpZ7NbSzkW3R0E04llZ vlyuUmPzHnrEoHfb1CWuoGpqmopeWWpNLqVvbwPYqIora9jmtXYBMJHbMoePOOvBQgmvPK6+ +sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329Rb1mw8JW1ldmyCzwiBfss8U6eaXIXDO DNk85DDylIyemM0LTQCXUtO8FI2q2Nm6+bbWrtBdfbdwkdFRlI/hZnLspAzjy+AM1X1DTvDU Y1MQGy/s+OzD2N2l4WuZZsJgNHvOMksCPLXHtVLxBDomnxxJBZWczXVors9rflza3HG5RhmB Tjo2Sdxw3HFrUruw1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKEAur6Z4bC69Jp0l p5bRwy6WPtfO0bfNBBbIbn7r/MecDg0k9n4aljmVBaQmTRxeq8d0xMd1x+5XcxGOD8pBbk89 MUHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0yTpahp3hqMamIDZf2fHZh7G7S8LXMs2 EwGj3nGSWBHlrj2oWmgDNSu7DVfBGnrCNLtZLMTl4mmkDxMZFKrGpYs24HqQwHPK4qg8ds3w 7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSb89n4aljmVBaQmTRxeq8d0xMd1x+5XcxGOD8p Bbk89McXTA7Say8NSxTIgtImk0YXqul0xMd1x+5XLEYOD8pBbk89MX28P6E+m6hfJYvFZR6a ZbSWfz43MhRcFywEbMWPGxsHPRsjFXVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClM AZzxUV5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFIDi67nVNRsrLwLYWumFU+ 0CUTwfbIZmXcysplUJlm2rwwClMAZzxWW8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAem SRJYX+Gr24ng+0JqvnGEyqHKeUF3BSckZPb39DTAHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3H GQoOAD0ySJLC/wANXtxPB9oTVfOMJlUOU8oLuCk5Iye3v6GtXWbDwlbWV2bILPCIF+yzxTp5 pchcM4M2TzkMPKUjJ6YzVy//ALKtvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0wcClc CnrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHlKRk9MZqpfWOhS6FLPZraWci26OgmnEsrN 8uVykx+Y89YlA77eozbX/kTb7/kC/wCvH+u/4/eqf6v/AGf/ALKsKgDvpvD2iSNrGqxW08Ol tpzT2Mc0cyCKTau3c7YDMWPygFwcnngZoX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO +3qMN/DuuRKzSaNqKKoJYtauAAOueKH8O65ErNJo2ooqgli1q4AA654oQHQ6ld2Gq+CNPWEa XayWYnLxNNIHiYyKVWNSxZtwPUhgOeVxUusWPhK2sbtrICaH7Ov2WaOdPOLkLguDNk85DDyg Rk4xjNVb6w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjAtapqNlZeBbC10wqn2g SieD7ZDMy7mVlMqhMs21eGAUpgDOeKAJtS07wUjarY2br5ttau0F19t3CR0VGUj+FmcuykDO PL4AzXAV199Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6jDfw7rkSs0mjaiiqCWLWr gADrnihAbl9Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6jXv8A+yrbwv4g07Sbqy+x OttJZk3oMtxja0jMjP8AK3HQKpPTBwK5547Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9M k6mqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznigCK80XQh4YuXhvNPfUIYoZI pIbgJ53H7wbXlLHg9NkZyOBzim6tYaFEL8WMenPp6WytaXZ1BhcyPhesYLcklgVMajryuKpJ LC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oav3mi6EPDFy8N5p76hDFDJFJDcBPO4/eDa8pY 8HpsjORwOcUAcXRXob+H9CfTtRvo7J4rGPTjLZy3HnxyNIVXBkLDy2YseAjYOejZBGXfWOhS 6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eoYHIUVpv4d1yJWaTRtRRVBLFrVwAB1zxXQ6l d2Gq+CNPWEaXayWYnLxNNIHiYyKVWNSxZtwPUhgOeVxQBxdFd1rNh4StrK7NkFnhEC/ZZ4p0 80uQuGcGbJ5yGHlKRk9MZqpfWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eoVwOQort NRurDVfA+nJD/ZdtJZrOZImmkDxMZFKrGpYli4PUhgOeVxWbrVzBN4c0iPTp40s4023NrvAk +1fxSMvVgRjDcgDj5elMDnaK6V47Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9Mk6+u6Lo ukaeJLyy+yTXenrNDCon82O5+XKDdlNg/iDEuMn/AGaAODoq+9hBHocV819GbmWcotouGYRg cuxB+XngAgE9RxVCgAorvtT1CKTw3NbpqVo3iIW6C/uVlUm4hy37pZejuAY92DlsYy2MUeLN flisdPhi1Kee4k05ILhoNSSWIvgiQSRgNuYhj824ZyCPu0rgcDRXoeqail5Zak0upW9vA9io iitr2Oa1dgEwkdsyh48468FCCa49/DuuRKzSaNqKKoJYtauAAOueKEwMyius+1P/AMKv+yfb 49/9oeZ9n+1Lv8nbj7mc48znGP8Aax3rpfFOvWF1deJEs9Q+1SNEESGe8VrR4yqEvCMY81SO BnOckZPy0X1sB5dRXpXjPUZ7vWNSk0zWEFo6NljrETwSJ5RDKsA+bcTwOvPPGcjM1u9aeG4b R9Vsrfw+1kqxWMsill5GUMXzN5u/Lb+fXfQndAcRRXb+Lo59WGmOdWspltdMUTl9Rjc+cAxc ABiWZsKMgHJxzWpqt3oElxrmkw6jG+mW9gWsbbaggSVVBUxyb9zOSzZ4y2SCTii4HmlFdZ9q f/hV/wBk+3x7/wC0PM+z/al3+Ttx9zOceZzjH+1jvXW6z4i0e41vxPFEdPEkmnzIt6jL+/Hl psRW3YZstICAOQqD+Gi+tgPJqK9M1a60B7jXNIh1GJ9Nt7AmxttqeQkqqCrRyhyWclmzxlsk EnFS6rqlu2iyR3OpwTN/Yawvuv0uEa5yMjylJJkz0kyQOvOKE7geXUV2/i6OfVhpjnVrKZbX TFE5fUY3PnAMXAAYlmbCjIBycc1peMtRmutX1J9L1dBaMjZP9rxPBJH5RDIsA53E8DrzzxnI LgebUV0WqXMD+EtMt7qeO51RXLRPG4cw2uOI3Yd92SFOSo4+XOKzHsII9DivmvozcyzlFtFw zCMDl2IPy88AEAnqOKYFCiiu51TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8U AcNRXSvHbN8O0ImgW7W/3mEXnzPHtI3mIvjduOMhQcAHpkkSWF/hq9uJ4PtCar5xhMqhynlB dwUnJGT29/Q0Ac1RXdazYeErayuzZBZ4RAv2WeKdPNLkLhnBmyechh5SkZPTGawrX/kTb7/k C/68f67/AI/eqf6v/Z/+yoAwqKK7nVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMA ZzxQBw1FdffWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eoo315baJaTaRpM6TzSDZf ahGeJR3iiP8Azz9T/H9MAgHPUV199YeH0tL17f7GdSW0jY2q3ZNvEx++YpM/vHA2nYWIBJwX xgWtU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFAHDUV199Y6FLoUs9mtpZyL bo6CacSys3y5XKTH5jz1iUDvt6ihfXltolnNpOlTJNPKNl9fxniT1iiP/PP1P8f+7gEA5+iu lSWF/hq9uJ4PtCar5xhMqhynlBdwUnJGT29/Q0PHbN8O0ImgW7W/3mEXnzPHtI3mIvjduOMh QcAHpkkA5qiu0ns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnphdX0zw4E16TTpLQo0c Eul4u+do2+aMFshufuv8x5wODSuBxVFdffWHh9LS9e3+xnUltI2Nqt2TbxMfvmKTP7xwNp2F iAScF8YGFLNopjcQ6fqCSEHYz3yMAexIEQz+YpgZtFeg3k14PD1zBNrem3NxPbbpka8iMECJ jEMMKHBkO0fMFAGMKcnNcbLpnlaNBqP26yfzXKfZklzOnXllxwPl657j1oA73xFqFhdnWGtN V8nTXs1+yxi7ilhfhNsa2u3dGePvcFSM8dK5S+vLbRLSbSNJnSeaQbL7UIzxKO8UR/55+p/j +mAaEs2imNxDp+oJIQdjPfIwB7EgRDP5iiWbRTG4h0/UEkIOxnvkYA9iQIhn8xSSsBu31h4f S0vXt/sZ1JbSNjardk28TH75ikz+8cDadhYgEnBfGBa1TUbKy8C2FrphVPtAlE8H2yGZl3Mr KZVCZZtq8MApTAGc8VzEs2imNxDp+oJIQdjPfIwB7EgRDP5iiWbRTG4h0/UEkIOxnvkYA9iQ Ihn8xQBu31joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qMN/DuuRKzSaNqKKoJYtauA AOueKSWbRTG4h0/UEkIOxnvkYA9iQIhn8xRLNopjcQ6fqCSEHYz3yMAexIEQz+YoA3b6x0KX QpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1F+bw9okjaxqsVtPDpbac09jHNHMgik2rt3O2 AzFj8oBcHJ54GeTlm0UxuIdP1BJCDsZ75GAPYkCIZ/MUSzaKY3EOn6gkhB2M98jAHsSBEM/m KAOins/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnphurWGhRC/FjHpz6elsrWl2dQYXM j4XrGC3JJYFTGo68riuflm0UxuIdP1BJCDsZ75GAPYkCIZ/MUSzaKY3EOn6gkhB2M98jAHsS BEM/mKAGS6Z5WjQaj9usn81yn2ZJczp15ZccD5eue49a6PVrDQohfixj059PS2VrS7OoMLmR 8L1jBbkksCpjUdeVxXPyzaKY3EOn6gkhB2M98jAHsSBEM/mKJZtFMbiHT9QSQg7Ge+RgD2JA iGfzFADHsII9DivmvozcyzlFtFwzCMDl2IPy88AEAnqOKoV2Pii7uLjRtJW01ON7GPS7eO4t 471P9YOoMW7JI+Xtxj2rjqYHSpLC/wANXtxPB9oTVfOMJlUOU8oLuCk5Iye3v6Gh47Zvh2hE 0C3a3+8wi8+Z49pG8xF8btxxkKDgA9Mk3dWsNCiF+LGPTn09LZWtLs6gwuZHwvWMFuSSwKmN R15XFP1fTPDYXXpNOktPLaOGXSx9r52jb5oILZDc/df5jzgcGlcBJ7Pw1LHMqC0hMmji9V47 piY7rj9yu5iMcH5SC3J56Ym1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM1DqV3Y ar4I09YRpdrJZicvE00geJjIpVY1LFm3A9SGA55XFcXQBu2v/Im33/IF/wBeP9d/x+9U/wBX /s//AGVYVeh63LpWqhYdRubSW7t9AVxeLeb3Nyh/1e4MUYk57EnPB6VW1mw8JW1ldmyCzwiB fss8U6eaXIXDODNk85DDylIyemM0XAhXxTfyeHdU1G+1Zru+vmNiLN5AqxxsmWlEY78bQQAA SSc5xWx4i1CwuzrDWmq+Tpr2a/ZYxdxSwvwm2NbXbujPH3uCpGeOlc74gh0TT44kgsrOZrq0 V2e1vy5tbjjcowzApx0bJO44bjjQ1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM0 tANTVtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk15jRXY2t3cJ4C1m3v9Tjk81LX 7HA16kjBQ+SFQMSuBjIwOntTSsBx1FaUs2imNxDp+oJIQdjPfIwB7EgRDP5iiWbRTG4h0/UE kIOxnvkYA9iQIhn8xTAzaK0pZtFMbiHT9QSQg7Ge+RgD2JAiGfzFEs2imNxDp+oJIQdjPfIw B7EgRDP5igDNorSlm0UxuIdP1BJCDsZ75GAPYkCIZ/MUSzaKY3EOn6gkhB2M98jAHsSBEM/m KAM2itKWbRTG4h0/UEkIOxnvkYA9iQIhn8xRLNopjcQ6fqCSEHYz3yMAexIEQz+YoAzaK0pZ tFMbiHT9QSQg7Ge+RgD2JAiGfzFEs2imNxDp+oJIQdjPfIwB7EgRDP5igDNorSlm0UxuIdP1 BJCDsZ75GAPYkCIZ/MUks2imJxFYX6yFTsZr1GAPYkeUMj2yKAM6iiigAooooAKKKKACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKK KKACiiigAooooAKKKKACiiigD0HxI2mX1kk097bXusQaTAhWS7BQMC3mMHVsSSg/wkjOc/P0 qtq9j4UtrC6a0CzR/Z1+zTRTp5pkIXBYGbJGchh5QIyemM1mPHbN8O0ImgW7W/3mEXnzPHtI 3mIvjduOMhQcAHpkm/PZ+GpY5lQWkJk0cXqvHdMTHdcfuV3MRjg/KQW5PPTCAq+IIdE0+OJI LKzma6tFdntb8ubW443KMMwKcdGyTuOG44tT2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykF uTz0xTvB4YFtc6nbx8XMPlW2mb332s2Budnz8yjqP727GBtOLl5ouhDwxcvDeae+oQxQyRSQ 3ATzuP3g2vKWPB6bIzkcDnFCAk1ex8KW1hdNaBZo/s6/Zpop080yELgsDNkjOQw8oEZPTGaz Elhf4avbieD7Qmq+cYTKocp5QXcFJyRk9vf0NCSwv8NXtxPB9oTVfOMJlUOU8oLuCk5Iye3v 6Gh47Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9MksDmq9S1bVYH0aVLrVIJ3/ALEWGTdf pcI1zkZHlKSTJnnzckDrziqd/wD2VbeF/EGnaTdWX2J1tpLMm9BluMbWkZkZ/lbjoFUnpg4F ec0t9QPSNRv7JvEPibWRfWjWd9pTRW5W4Vnd2SMBfLzvByD1AxjmmeILuG7fV57bxDFBpb2y i2sl8uWJ12JhFi37o23D+4NuM5FUdQ07w1GNTEBsv7Pjsw9jdpeFrmWbCYDR7zjJLAjy1x7V Lrui6LpGniS8svsk13p6zQwqJ/Njuflyg3ZTYP4gxLjJ/wBmhAcHXaXmi6EPC9xJDd2DahDF DJE8NwE87j94NrysSQD02RnI4HOKbq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOvK4 qXUNO8NRjUxAbL+z47MPY3aXha5lmwmA0e84ySwI8tce1FwGald2Gq+CNPWEaXayWYnLxNNI HiYyKVWNSxZtwPUhgOeVxXF16LfHSrbwt4g07Srmy+xOltJaE3wMtxja0jMjP8rcdAqk9MHA FU9YsPCdtY3bWYWeIW6/Zp4p0MpkIXDMDNkjOQw8pSMnpjNCATVNRs7LwJp9rppVDcLKLiAX kMxG5lZTKoTLNtXhgFKYAzniq19Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6jNtf+ RNvv+QL/AK8f67/j96p/q/8AZ/8AsqwqYHQX15baJZzaTpUyTTyjZfX8Z4k9Yoj/AM8/U/x/ 7uAb99YeH0tL17f7GdSW0jY2q3ZNvEx++YpM/vHA2nYWIBJwXxgdXrWp6F9m1p4dRiuLK7sD 9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLF RtwOnNSnsBzuqajZ2XgTT7XTSqG4WUXEAvIZiNzKymVQmWbavDAKUwBnPFVr6x0KXQZJ7QWl pItvG6CacSys3y5X5Jj8x56xKB329R1etanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjc WKjbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c00wOGvry20S0m0jS Z0nmkGy+1CM8SjvFEf8Ann6n+P6YBvX1joUugyT2gtLSRbeN0E04llZvlyvyTH5jz1iUDvt6 jkK7HxRd3Fxo2kraanG9jHpdvHcW8d6n+sHUGLdkkfL24x7UwH6tpnh0JrsmnyWe1o4JdLAu +do2+aMM2Q3PCv8AMecDg1Y13RdF0jTxJeWX2Sa709ZoYVE/mx3Py5QbspsH8QYlxk/7NbGs 6voEvhK9hguLYu2mWiRxL97hm2rjzScqeSMkrnJL9K5yYtcfDeO1e404XEd6LgRRzwI5hEOM kKQS2eMHLmkgLeu6LoukaeJLyy+yTXenrNDCon82O5+XKDdlNg/iDEuMn/ZqG+07w4iakIvs QsY7IPZXaXha4lmwmA0e84ySwI2DHt1qK1u7hPAWs29/qccnmpa/Y4GvUkYKHyQqBiVwMZGB 09q46mB1XiCHRNPjiSCys5murRXZ7W/Lm1uONyjDMCnHRsk7jhuOLWpXdhqvgjT1hGl2slmJ y8TTSB4mMilVjUsWbcD1IYDnlcVf1TUUvLLUml1K3t4HsVEUVtexzWrsAmEjtmUPHnHXgoQT WtrPiLR7jW/E8UR08SSafMi3qMv78eWmxFbdhmy0gIA5CoP4alPYDhr68ttEs5tJ0qZJp5Rs vr+M8SesUR/55+p/j/3cA9Le/wBl23hbX9O0u6svsbpbSWhN6DJcEbWkZkZ/lbjoFUnpg4FN 8TeI/s+m21q0y6jLc6UltOft6TxJKCC7lFzmQdn3fTODXnlNAdzq9h4TtrC6NmEnjFuv2aeG dfNMhC4Zg02SM5DDylIyemM1c186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vTzq igDtNW0zw6E12TT5LPa0cEulgXfO0bfNGGbIbnhX+Y84HBq8fD+hvpuoXy2TRWUemmW0ln+0 Ru0hRcFyw8tmLHgI2Dno2QR57RTAvy6Z5WjQaj9usn81yn2ZJczp15ZccD5eue49a6LVbDQ4 lvhZR6a9glqrWt3/AGgwuJJML1jy3JJYFTGo68riuOooA3bX/kTb7/kC/wCvH+u/4/eqf6v/ AGf/ALKtSYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmuOooA9Z1nU9E+yay8eoxXFnd6 efs0IuF8q3YKqxxpb53Btw3Fio24HT5qo3Or2s1pq73lxa/ZpdM8uCG31BZLbzNqBBFbFQ8Z BGeR8uD9a80opWA6f+24/wDhXP8AZP2S2837bnzPk3bcbt2N27d/Du2428ZzXMUUUwCiiigA ooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK KACiiigAooooAKKKKACiiigDpUlhf4avbieD7Qmq+cYTKocp5QXcFJyRk9vf0NX7zRdCHhi5 eG8099QhihkikhuAnncfvBteUseD02RnI4HOKfrd608Vw2japZW/h5rMLDYSyKWXkZjMWC3m 78tvwfXfVzxN4j+z6bbWrTLqMtzpSW05+3pPEkoILuUXOZB2fd9M4NIDKvbHQ5dAkntRaWkq 20br5s4klZ/lyo2TH5jz1iUDvt6ibUruw1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzy uK1dTv7FvEfifWEvrRrG+0tobZluFZ3dkjAHl53g5BzkDGOcV5tQtdQCuvvrHQpdClns1tLO RbdHQTTiWVm+XK5SY/MeesSgd9vUdXrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fi o24HTmvJaFrqB30nh7RZP7X1WO1mg0xtNM9jHLHMgik2rjMj4DMWPygFg2T0wM0L6x0KXQpZ 7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1HIUUWA7nVNRsrLwLYWumFU+0CUTwfbIZmXcyspl UJlm2rwwClMAZzxVS9sdDl0GSe1FpaSrbxugmnEkrN8uV+SY/MeesSgd9vUcjRTAK7jU9Rs7 LwJp9rpxVDcLKLiAXcMpG5lYGVAmWbC8MApTAGc8Vw9FAHSvHbN8O0ImgW7W/wB5hF58zx7S N5iL43bjjIUHAB6ZJElhf4avbieD7Qmq+cYTKocp5QXcFJyRk9vf0Nc1RQB3Or2HhS1sLo2m y4jFuv2a4hmXzTIQuGZWnzgnO5fKBGT0xmsO1/5E2+/5Av8Arx/rv+P3qn+r/wBn/wCyrCoo AK6G+vLbRLSbSNJnSeaQbL7UIzxKO8UR/wCefqf4/pgHnqKAOuvbHQ5dAkntRaWkq20br5s4 klZ/lyo2TH5jz1iUDvt6jQk8O6M/9r6oltNFpjaaZrJJYpo1jk2rjLucFix+VQzg5PPAzwNF AHXXtjocugST2otLSVbaN182cSSs/wAuVGyY/MeesSgd9vUUr68ttEtJtI0mdJ5pBsvtQjPE o7xRH/nn6n+P6YB56igD0W9/su18K6/pul3dibN0tpLRjejzLgja0jNGz/K3HQKpPTBwKpat YeFLXT7o2uydBbr9nnhnXzTIQuGZWmzgnO4eUCMnpjNcPRSA6e6HhlbS41KFATcweVb6aGfd azYG52cnlB1XOd27GBtOGvHbN8O0ImgW7W/3mEXnzPHtI3mIvjduOMhQcAHpknmqKYHY6rp+ hwrfCyTTZLFLVWtbz7ewuJJML1jy3JJYFSi4GeVxmp9WsPClrp90bXZOgt1+zzwzr5pkIXDM rTZwTncPKBGT0xmuHopAdde2Ohy6BJPai0tJVto3XzZxJKz/AC5UbJj8x56xKB329RSvry20 S0m0jSZ0nmkGy+1CM8SjvFEf+efqf4/pgHnqKYHXXtjocugST2otLSVbaN182cSSs/y5UbJj 8x56xKB329Rfbw9o8i6tqq20sOnNppns45I5kWKXamMu+AxLH5QC4OTzwM8FRQB117Y6HLoE k9qLS0lW2jdfNnEkrP8ALlRsmPzHnrEoHfb1E2pXdhqvgjT1hGl2slmJy8TTSB4mMilVjUsW bcD1IYDnlcVxdFAHcatYeFLXT7o2uydBbr9nnhnXzTIQuGZWmzgnO4eUCMnpjNS3+neDUGp2 NtJGbi2tGaK6W93LI6KjAr/CxcuykAnHl8AZrgqKQHQ315baJaTaRpM6TzSDZfahGeJR3iiP /PP1P8f0wDoanYaJDHei0j017JLVWtrwX7C4kkwvWPLcklgVKLx3XGa46imB3mu6LoukaeJL yy+yTXenrNDCon82O5+XKDdlNg/iDEuMn/ZqtqmmeHhDrkljJZYaKCXTAt32G3zhhmyDzwr/ ADHnA4NcZRSsB0N9eW2iWk2kaTOk80g2X2oRniUd4oj/AM8/U/x/TAN/U7DRIo70WkemPZJa o1tdjUGFxJJtTrHluSSwKmNeO64rj6KYHea7oui6Rp4kvLL7JNd6es0MKifzY7n5coN2U2D+ IMS4yf8AZqtqmmeHhDrkljJZYaKCXTAt32G3zhhmyDzwr/MecDg1xlFKwHoP9oEaRqi6hqtt cLJp5SEpdxvbs3ybFjtQqtGwGBkgbSCSK5++vLbRLSbSNJnSeaQbL7UIzxKO8UR/55+p/j+m AeeooA7HVdP0OFb4WSabJYpaq1refb2FxJJheseW5JLAqUXAzyuM0+80XQh4YuXhvNPfUIYo ZIpIbgJ53H7wbXlLHg9NkZyOBziuLopgdhqdhokUd6LSPTHsktUa2uxqDC4kk2p1jy3JJYFT GvHdcVp63LpWqhYdRubSW7t9AVxeLeb3Nyh/1e4MUYk57EnPB6V55RQB3Gq2HhW2065NsEnQ Wy/Z54J18wyELhmDTZIzkMPKUjJ6YzWRfXltolpNpGkzpPNINl9qEZ4lHeKI/wDPP1P8f0wD z1FAHSvHbN8O0ImgW7W/3mEXnzPHtI3mIvjduOMhQcAHpkm/PZ+GpY5lQWkJk0cXqvHdMTHd cfuV3MRjg/KQW5PPTHF0UAeiXf8AZlt4T17TtNurH7K8dtJan7aDJcEbWkYoz/K3HQKpPAwc Cqeq2HhW2065NsEnQWy/Z54J18wyELhmDTZIzkMPKUjJ6YzXD0UAdffWHh9LS9e3+xnUltI2 Nqt2TbxMfvmKTP7xwNp2FiAScF8YGleTXg8PXME2t6bc3E9tumRryIwQImMQwwocGQ7R8wUA Ywpyc159RSA6G+vLbRLSbSNJnSeaQbL7UIzxKO8UR/55+p/j+mAbt7Y6HLoEk9qLS0lW2jdf NnEkrP8ALlRsmPzHnrEoHfb1HI0UwPQ9bl0rVQsOo3NpLd2+gK4vFvN7m5Q/6vcGKMSc9iTn g9KoXmi6EPDFy8N5p76hDFDJFJDcBPO4/eDa8pY8HpsjORwOcVxdFAHVeIIdE0+OJILKzma6 tFdntb8ubW443KMMwKcdGyTuOG44v6xYeE7axu2sws8Qt1+zTxToZTIQuGYGbJGchh5SkZPT Ga4aikB6LfHSrbwt4g07Srmy+xOltJaE3wMtxja0jMjP8rcdAqk9MHAFcY3h7W0Qu2j6gqKM ljbOAB69KzaKYHY6pYaHCl6LNNNksUtVa2vBfsLiSTC9Y8tySWBUouB3XGas63ouiaTpqSXl p9mmu9OSaGNRN5qXPy5Ubvk2DqwYlxk/7NcLRQB2mraX4cWPXZNPktChigl0v/TMnaNvm5Bb Ibn7rjJ52jg1zz+HdciVmk0bUUVQSxa1cAAdc8VmUUAd9qOneC0bVbG0dPOtrRmhuhe7lkdF RlI42szl2Ugf88+AM1VvNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4ri6KQHd6 5omi6Ppyve2ZtZ7vT1liiUTealz8uUG75Ng6sGJcZP8As1wlFFMAooooAKKKKACiiigD1nWf EWj3Gt+J4ojp4kk0+ZFvUZf348tNiK27DNlpAQByFQfw1zuoad4ajGpiA2X9nx2Yexu0vC1z LNhMBo95xklgR5a49qZPZ+GpY5lQWkJk0cXqvHdMTHdcfuV3MRjg/KQW5PPTGhd/2ZbeE9e0 7Tbqx+yvHbSWp+2gyXBG1pGKM/ytx0CqTwMHAqUrWAdf/wBlW3hfxBp2k3Vl9idbaSzJvQZb jG1pGZGf5W46BVJ6YOBWdq+meGwuvSadJaeW0cMulj7XztG3zQQWyG5+6/zHnA4NZ6Swv8NX txPB9oTVfOMJlUOU8oLuCk5Iye3v6GtPVbDwrbadcm2CToLZfs88E6+YZCFwzBpskZyGHlKR k9MZp2AzElhf4avbieD7Qmq+cYTKocp5QXcFJyRk9vf0Naus2HhK2srs2QWeEQL9lninTzS5 C4ZwZsnnIYeUpGT0xms/xBDomnxxJBZWczXVors9rflza3HG5RhmBTjo2Sdxw3HHK0eYHX31 joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qOQr0XXzpmoWSSandWU09voqKLqO+Es5u wfuYVyGBJOW2nqTu9M+807w6kWoBPsIso7FXs7uO8LXEs+1MBo95xliwI2DA9OtCAz0lhf4a vbieD7Qmq+cYTKocp5QXcFJyRk9vf0NDx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIUHAB6ZJ vz2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykFuTz0xoXf8AZlt4T17TtNurH7K8dtJan7aD JcEbWkYoz/K3HQKpPAwcCmBnz2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykFuTz0xd8SNpl 9ZJNPe217rEGkwIVkuwUDAt5jB1bEkoP8JIznPz9Kw3jtm+HaETQLdrf7zCLz5nj2kbzEXxu 3HGQoOAD0yTfns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnphATazYeErayuzZBZ4RA v2WeKdPNLkLhnBmyechh5SkZPTGauX/9lW3hfxBp2k3Vl9idbaSzJvQZbjG1pGZGf5W46BVJ 6YOBUQ8P6I+lX999iMVpHpnm2ss32iN2lKLguWHlsSx4CNg56NkEefUWAK6V47Zvh2hE0C3a 3+8wi8+Z49pG8xF8btxxkKDgA9Mk7F/p3g1BqdjbvH9otrNmiulvdyySIqMCv8LFy7KQM48v gDNJrui6LpGniS8svsk13p6zQwqJ/Njuflyg3ZTYP4gxLjJ/2aLgV57Pw1LHMqC0hMmji9V4 7piY7rj9yu5iMcH5SC3J56YyLX/kTb7/AJAv+vH+u/4/eqf6v/Z/+yrX1K7sNV8EaesI0u1k sxOXiaaQPExkUqsalizbgepDAc8riob2x0OXQJJ7UWlpKttG6+bOJJWf5cqNkx+Y89YlA77e oYHI12l5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFWNd0XRdI08SXll9kmu9P WaGFRP5sdz8uUG7KbB/EGJcZP+zWQ8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSUBd 1aw0KIX4sY9OfT0tla0uzqDC5kfC9YwW5JLAqY1HXlcVa13RdF0jTxJeWX2Sa709ZoYVE/mx 3Py5QbspsH8QYlxk/wCzXB1117Y6HLoEk9qLS0lW2jdfNnEkrP8ALlRsmPzHnrEoHfb1BYCs 8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSc1/DuuRKzSaNqKKoJYtauAAOueK6XVbD wrbadcm2CToLZfs88E6+YZCFwzBpskZyGHlKRk9MZqj4gh0TT44kgsrOZrq0V2e1vy5tbjjc owzApx0bJO44bjgAsatYaFEL8WMenPp6WytaXZ1BhcyPhesYLcklgVMajryuKta7oui6Rp4k vLL7JNd6es0MKifzY7n5coN2U2D+IMS4yf8AZrIeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZ Cg4APTJOhead4djj1ERmx+xR2Iezu4rwtcSz7UwGj3nGSWBHlrgelABq+meGwuvSadJaeW0c Mulj7XztG3zQQWyG5+6/zHnA4NP1TUbOy8Cafa6aVQ3Cyi4gF5DMRuZWUyqEyzbV4YBSmAM5 4rmn8O65ErNJo2ooqgli1q4AA654rX8QQ6Jp8cSQWVnM11aK7Pa35c2txxuUYZgU46NknccN xwLsAeIIdE0+OJILKzma6tFdntb8ubW443KMMwKcdGyTuOG440NZsPCVtZXZsgs8IgX7LPFO nmlyFwzgzZPOQw8pSMnpjNQ3mi6EPDFy8N5p76hDFDJFJDcBPO4/eDa8pY8HpsjORwOcVPfa P4Uhk1W+h1Gwe1NkWtLSOaQukrKoQjJyTuD5U/dG0nrwAEui6JdeHdW1TSLTz4bWGNYNwm8w MdvmNL/CWUZPyHaASSCMVwlFdV4gh0TT44kgsrOZrq0V2e1vy5tbjjcowzApx0bJO44bjhgW NWsNCiF+LGPTn09LZWtLs6gwuZHwvWMFuSSwKmNR15XFWdd0bRNJ05Zbuz+yTXmnrNDEgn8y O5+XKDdlPLHcMS4yf9moJ7Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5SC3J56Yujw/oj6Vf3 32IxWkemebayzfaI3aUouC5YeWxLHgI2Dno2QQkBT1fTPDYXXpNOktPLaOGXSx9r52jb5oIL ZDc/df5jzgcGpNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFcy/h3XIlZpNG 1FFUEsWtXAAHXPFbup2GiRR3otI9MeyS1Rra7GoMLiSTanWPLcklgVMa8d1xQgIfEEOiafHE kFlZzNdWiuz2t+XNrccblGGYFOOjZJ3HDccaGs2HhK2srs2QWeEQL9lninTzS5C4ZwZsnnIY eUpGT0xmorzTvDsceoiM2P2KOxD2d3FeFriWfamA0e84ySwI8tcD0qp4gh0TT44kgsrOZrq0 V2e1vy5tbjjcowzApx0bJO44bjgA0ptF0O78Pavqmj2ZnhtYY1t9wm8wMdvmNL0QsoyfkO0A kkEYrln8O65ErNJo2ooqgli1q4AA654rpdVsPCttp1ybYJOgtl+zzwTr5hkIXDMGmyRnIYeU pGT0xmuHoQHb6hp3hqMamIDZf2fHZh7G7S8LXMs2EwGj3nGSWBHlrj2pk9n4aljmVBaQmTRx eq8d0xMd1x+5XcxGOD8pBbk89MQ3tjocugST2otLSVbaN182cSSs/wAuVGyY/MeesSgd9vUO 1Ow0SKO9FpHpj2SWqNbXY1BhcSSbU6x5bkksCpjXjuuKANa//sq28L+INO0m6svsTrbSWZN6 DLcY2tIzIz/K3HQKpPTBwKp6zYeErayuzZBZ4RAv2WeKdPNLkLhnBmyechh5SkZPTGaqX1jo UugyT2gtLSRbeN0E04llZvlyvyTH5jz1iUDvt6iq8ds3w7QiaBbtb/eYRefM8e0jeYi+N244 yFBwAemSSwG1qWneCkbVbGzdfNtrV2guvtu4SOioykfwszl2UgZx5fAGaqXmi6EPDFy8N5p7 6hDFDJFJDcBPO4/eDa8pY8HpsjORwOcVzz+HdciVmk0bUUVQSxa1cAAdc8V0OpXdhqvgjT1h Gl2slmJy8TTSB4mMilVjUsWbcD1IYDnlcUIAns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflI LcnnpjSv/wCyrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk9MHArD8QQ6Jp8cSQWVnM1 1aK7Pa35c2txxuUYZgU46NknccNxxavNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcj gc4oA4uuleO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJN+ez8NSxzKgtITJo4vVeO6Y mO64/cruYjHB+UgtyeemIr6x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1DAu67oui6 Rp4kvLL7JNd6es0MKifzY7n5coN2U2D+IMS4yf8AZp2o6d4LRtVsbSRDNbWjNDdC83LI6KjK R/CzOXZSATjy+AM03XdF0XSNPEl5ZfZJrvT1mhhUT+bHc/LlBuymwfxBiXGT/s1Dfad4cRdS EX2EWMdkHsruO8L3Es2EwGj3dyWB/drgf3aSAznjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQ oOAD0yTmv4d1yJWaTRtRRVBLFrVwAB1zxWvr8Gi6dFFHBZWk7XNorNJa3+821xxuUYZgU46E EnccNxxZ1C6sNV8D6ckP9mW0lms5kjaaQPGxkUqsaliWLg9SCo55XFAEd9Y6FLoUs9mtpZyL bo6CacSys3y5XKTH5jz1iUDvt6iq8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSc1/D uuRKzSaNqKKoJYtauAAOueK3L6x0KXQpZ7NbSzkW3R0E04llZvlyuUmPzHnrEoHfb1AByFdK 8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSdbXNE0XR9NV720NrPdaeksUYE3mpdfKC o3fIEHVgxLjJ/wBmsl47Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9MksDX13RdF0jTxJe WX2Sa709ZoYVE/mx3Py5QbspsH8QYlxk/wCzTtR07wWjarY2kiGa2tGaG6F5uWR0VGUj+Fmc uykAnHl8AZrga9F186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vRAc+8ds3w7Qia Bbtb/eYRefM8e0jeYi+N244yFBwAemSc1/DuuRKzSaNqKKoJYtauAAOueK6CWy8NSQyov2WJ n0YXqyJctujuhj9yMsRzg/KQW+Y89MRXmn6BFZXjQNaNqaWcbNbC7Jgjc/faKTP7xwNp2biA WbBfGAwNi/8A7KtvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0wcCqes2HhK2srs2QWe EQL9lninTzS5C4ZwZsnnIYeUpGT0xmp7ya8Hh65gm1vTbm4ntt0yNeRGCBExiGGFDgyHaPmC gDGFOTmvPqSA6i8HhgW1zqdvHxcw+VbaZvffazYG52fPzKOo/vbsYG04Y8ds3w7QiaBbtb/e YRefM8e0jeYi+N244yFBwAemSb2oXVhqngbTUh/sy3ls1n8yN5pBJGxkUqsaliWLg9SCo55X FaGty6VqoWHUbm0lu7fQFcXi3m9zcof9XuDFGJOexJzwelAFCez8NSxzKgtITJo4vVeO6YmO 64/cruYjHB+UgtyeemDUruw1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKl1ex8KW1 hdNaBJo/s6/ZpobhTKZCFwWDTZIzkMPJUjJ6Yqhr0Gi6bFDHBZ2k7XNmrNJa3+821wMblGGY FOOhGTuOG44AL+qajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAzniuGrvtQ07wYh1 SxtXQT21ozQ3IvdyyOioyleNrM5dlIH/ADz4AzVW80XQh4YuXhvNPfUIYoZIpIbgJ53H7wbX lLHg9NkZyOBzihAUHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0yTr67oui6Rp4kvLL7 JNd6es0MKifzY7n5coN2U2D+IMS4yf8AZrISWF/hq9uJ4PtCar5xhMqhynlBdwUnJGT29/Q1 zVMDv9S07wUjarY2br5ttau0F19t3CR0VGUj+FmcuykDOPL4AzWK8ds3w7QiaBbtb/eYRefM 8e0jeYi+N244yFBwAemSdbXdE0XSNOV72z+yT3enrNDEon82O5+XKDdlNg/iDEuMn/ZrO16D RdNihjgs7SdrmzVmktb/AHm2uBjcowzApx0Iydxw3HCQGQ/h3XIlZpNG1FFUEsWtXAAHXPFd ZqWneCkbVbGzdfNtrV2guvtu4SOioykfwszl2UgZx5fAGar6hp3hqMamIDZf2fHZh7G7S8LX Ms2EwGj3nGSWBHlrj2qLVbDQohfCxj05tPS2VrW7OoEXLvhesYLcklgVKKOvK4zQAy+sdCl0 KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329RqP4f0J9O1G+jsnisY9OMtnLcefHI0hVcGQsP LZix4CNg56NkEVdYsfCVtY3bWQE0P2dfss0c6ecXIXBcGbJ5yGHlAjJxjGaq31joUuhSz2a2 lnItujoJpxLKzfLlcpMfmPPWJQO+3qADU1qTStUCQahcWkt1baAri7W73ublD/q9wYo2TnjB JzwelULzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKivrHQpdClns1tLORbdHQ TTiWVm+XK5SY/MeesSgd9vUWNQ07w1GNTEBsv7Pjsw9jdpeFrmWbCYDR7zjJLAjy1x7UAUvE EOiafHEkFlZzNdWiuz2t+XNrccblGGYFOOjZJ3HDcccrXofibxH9n022tWmXUZbnSktpz9vS eJJQQXcoucyDs+76ZwawXjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0ySLYC/qV3Yar4 I09YRpdrJZicvE00geJjIpVY1LFm3A9SGA55XFX9bl0rVQsOo3NpLd2+gK4vFvN7m5Q/6vcG KMSc9iTng9Kff/2VbeF/EGnaTdWX2J1tpLMm9BluMbWkZkZ/lbjoFUnpg4Fcta/8ibff8gX/ AF4/13/H71T/AFf+z/8AZUIDc1iw8J2tjdmz2zxC3X7NPFOnmmQhcMymbOCc7h5QIyemM1Q8 QQ6Jp8cSQWVnM11aK7Pa35c2txxuUYZgU46NknccNxxyteg3k14PD1zBNrem3NxPbbpka8iM ECJjEMMKHBkO0fMFAGMKcnNADNR07wWjarY2kiGa2tGaG6F5uWR0VGUj+FmcuykAnHl8AZqr eaLoQ8MXLw3mnvqEMUMkUkNwE87j94NryljwemyM5HA5xWx4i1CwuzrDWmq+Tpr2a/ZYxdxS wvwm2NbXbujPH3uCpGeOlY6+Kb+Tw7qmo32rNd318xsRZvIFWONky0ojHfjaCAACSTnOKSvZ AUElhf4avbieD7Qmq+cYTKocp5QXcFJyRk9vf0NZr+HdciVmk0bUUVQSxa1cAAdc8V0K+Kb+ Tw7qmo32rNd318xsRZvIFWONky0ojHfjaCAACSTnOK17nV7Wa01d7y4tfs0umeXBDb6gslt5 m1AgitioeMgjPI+XB+tPUDDvrHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUX5vD2iS NrGqxW08OltpzT2Mc0cyCKTau3c7YDMWPygFwcnngZ5B7CCPQ4r5r6M3Ms5RbRcMwjA5diD8 vPABAJ6jiqFFgOvvrHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUchXZatYaFEL8WMe nPp6WytaXZ1BhcyPhesYLcklgVMajryuKl1DTvDUY1MQGy/s+OzD2N2l4WuZZsJgNHvOMksC PLXHtQmBxFFd/qWneCkbVbGzdfNtrV2guvtu4SOioykfwszl2UgZx5fAGaNS07wUjarY2br5 ttau0F19t3CR0VGUj+FmcuykDOPL4AzQmBwFFbtr/wAibff8gX/Xj/Xf8fvVP9X/ALP/ANlW FTAKK6++sPD6Wl69v9jOpLaRsbVbsm3iY/fMUmf3jgbTsLEAk4L4wLWqajZWXgWwtdMKp9oE ong+2QzMu5lZTKoTLNtXhgFKYAznigDhqK6++sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56x KB329QX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qADkKK76bw9okjaxqsVtPDpba c09jHNHMgik2rt3O2AzFj8oBcHJ54Gas9n4aljmVBaQmTRxeq8d0xMd1x+5XcxGOD8pBbk89 MK4HF0V2WrWGhRC/FjHpz6elsrWl2dQYXMj4XrGC3JJYFTGo68ritS//ALKtvC/iDTtJurL7 E620lmTegy3GNrSMyM/ytx0CqT0wcCi4HnNFFes63q+gXWveJbiG9gS5/syW2VldTHdhlVgy sOrg/IRzkBcdCKL62A8mort9Q07w1GNTEBsv7Pjsw9jdpeFrmWbCYDR7zjJLAjy1x7UzUruw 1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKLgcXRRXaald2Gq+CNPWEaXayWYnLxNN IHiYyKVWNSxZtwPUhgOeVxTA4uiu0vNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjg c4qg8ds3w7QiaBbtb/eYRefM8e0jeYi+N244yFBwAemSQDmqK7Sez8NSxzKgtITJo4vVeO6Y mO64/cruYjHB+UgtyeemDUruw1XwRp6wjS7WSzE5eJppA8TGRSqxqWLNuB6kMBzyuKAOLoru tZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjNXL/+yrbwv4g07Sbqy+xOttJZk3oM txja0jMjP8rcdAqk9MHApXA85oor0G8mvB4euYJtb025uJ7bdMjXkRggRMYhhhQ4Mh2j5goA xhTk5pgefUVfl0zytGg1H7dZP5rlPsyS5nTryy44Hy9c9x613niHULC6/tg2eq+TpjWai1jF 5FLC+Am2NbUrvjPH3uCpGeKVwPNKKvy6Z5WjQaj9usn81yn2ZJczp15ZccD5eue49a7L+0CN I1RdQ1W2uFk08pCUu43t2b5Nix2oVWjYDAyQNpBJFMDz6ivQbya8Hh65gm1vTbm4ntt0yNeR GCBExiGGFDgyHaPmCgDGFOTmuNl0zytGg1H7dZP5rlPsyS5nTryy44Hy9c9x60AUKK9B/tAj SNUXUNVtrhZNPKQlLuN7dm+TYsdqFVo2AwMkDaQSRXn1ABRXqms6voEvhK9hguLYu2mWiRxL 97hm2rjzScqeSMkrnJL9K5yYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmgDjqKK9D8Te I/s+m21q0y6jLc6UltOft6TxJKCC7lFzmQdn3fTODQB55RXb6hp3hqMamIDZf2fHZh7G7S8L XMs2EwGj3nGSWBHlrj2qXUdH8JQ3Gr31tqNk9kbMtZWiTSGRJGVRGeTkncHyp+6NpPXhXA4O iu61mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM1n+IIdE0+OJILKzma6tFdntb8u bW443KMMwKcdGyTuOG44LgcrRXpXjPUZ7vWNSk0zWEFo6NljrETwSJ5RDKsA+bcTwOvPPGci O/8A7KtvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0wcChO4HnNFehv4f0J9O1G+jsni sY9OMtnLcefHI0hVcGQsPLZix4CNg56NkEVtZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8 pSMnpjNFwOFoorsZi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaYHHUV6Xc6vazWmrveX Fr9ml0zy4IbfUFktvM2oEEVsVDxkEZ5Hy4P1qLUdbn1ODV5dV1KxSGSzLwSabqMv72QhdqeS 0hIBBIYGMd8460rgec0V61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmvJ aE7oAooopgFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAeo+KdesLq68SJZ6h9qkaIIkM94rWjx lUJeEYx5qkcDOc5IyflrmvtT/wDCr/sn2+Pf/aHmfZ/tS7/J24+5nOPM5xj/AGsd66XxTr1h dXXiRLPUPtUjRBEhnvFa0eMqhLwjGPNUjgZznJGT8tUdU1FLyy1JpdSt7eB7FRFFbXsc1q7A JhI7ZlDx5x14KEE1MdkAeJvEf2fTba1aZdRludKS2nP29J4klBBdyi5zIOz7vpnBrmbX/kTb 7/kC/wCvH+u/4/eqf6v/AGf/ALKum8TeI/s+m21q0y6jLc6UltOft6TxJKCC7lFzmQdn3fTO DXM2v/Im33/IF/14/wBd/wAfvVP9X/s//ZU1sBhV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQq rHGlvnIbI3Fio24HTmvJa9a1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzR 1ANa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1y0xa4+G8dq9xpwuI70XAi jngRzCIcZIUgls8YOXNdTrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWt T0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNJbJAGtanoX2bWnh1GK4sruwP2W H7QvlW5CqscaW+chsjcWKjbgdOa4K1/5E2+/5Av+vH+u/wCP3qn+r/2f/sq73WtT0L7NrTw6 jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNcFa/wDIm33/ACBf9eP9d/x+9U/1f+z/APZU 47Aanii7uLjRtJW01ON7GPS7eO4t471P9YOoMW7JI+Xtxj2qhNcwP4Njg02eO32uP7Rt5HAl uHz8jg/xoP7gHynk5+9Wvqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUb Ky8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5 lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2V l4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrK ZVCZZtq8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrL wLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxWrq2qWMkWslb+B7GXTlS2UXSNbs+2PAj tB88TZBwSTtIyaNW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTQgMrVNRsrLwLYW umFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyz bV4YBSmAM54roda1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1y0xa4+G8dq 9xpwuI70XAijngRzCIcZIUgls8YOXNC2AsapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21e GAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFQatYaFEL8WMenPp6 WytaXZ1BhcyPhesYLcklgVMajryuK09fOmahZJJqd1ZTT2+ioouo74Szm7B+5hXIYEk5baep O70EBZ1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2kZNYUxa4+G8dq9xpwuI70XAij ngRzCIcZIUgls8YOXNauo63PqcGry6rqVikMlmXgk03UZf3shC7U8lpCQCCQwMY75x1rKmLX Hw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5oWwBMWuPhvHavcacLiO9FwIo54EcwiHGSFIJb PGDlzWh/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkiqS+Kb+Tw7qmo32rNd318 xsRZvIFWONky0ojHfjaCAACSTnOK0r/+yrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk 9MHAoATXzpmoWSSandWU09voqKLqO+Es5uwfuYVyGBJOW2nqTu9JvEOo2F3/AGw9pqnkaa9m v2WIXcUsL8JtjW127ozx97gqRnisi80XQh4YuXhvNPfUIYoZIpIbgJ53H7wbXlLHg9NkZyOB zium1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzS6gY15NeDw9cwTa3ptzc T226ZGvIjBAiYxDDChwZDtHzBQBjCnJzWzrWqaGbXWXh1GO4sruwP2WD7QvlW5CqscaW4OQ2 RuLbRtwOnNeS0U7AdFqFzA3hWygvp47vUhg2rQuC1tB/clYZDZP3U6r3Iztq/a3dwngLWbe/ 1OOTzUtfscDXqSMFD5IVAxK4GMjA6e1bPjPUZ7vWNSk0zWEFo6NljrETwSJ5RDKsA+bcTwOv PPGcin4r1+WOw0+KLUp555NNSC4aHUUliL4IkEkYDbmIY/NkdiPu0J3VwOBort/F0c+rDTHO rWUy2umKJy+oxufOAYuAAxLM2FGQDk45rV1a60B7jXNIh1GJ9Nt7AmxttqeQkqqCrRyhyWcl mzxlskEnFFwMTxRd3Fxo2kraanG9jHpdvHcW8d6n+sHUGLdkkfL24x7VQmuYH8GxwabPHb7X H9o28jgS3D5+Rwf40H9wD5Tyc/erpfEV3b3cusT2niKKHSXtlFrZDy5YnXYm1Fi37o23D/nm NuM5FR6pqKXllqTS6lb28D2KiKK2vY5rV2ATCR2zKHjzjrwUIJoTAq6pqNlZeBbC10wqn2gS ieD7ZDMy7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUw BnPFYdr/AMibff8AIF/14/13/H71T/V/7P8A9lWFTA7nVNRsrLwLYWumFU+0CUTwfbIZmXcy splUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54roda1PQ ha6y0Oox3FjdWBFrD9pXyrchVWONIM7g2RuLFRtwOnzUa1qehfZtaeHUYriyu7A/ZYftC+Vb kKqxxpb5yGyNxYqNuB05qUwOe1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8U apqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK6HWtT0L7NrTw6jFcWV3YH7LD9 oXyrchVWONLfOQ2RuLFRtwOnNctMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzTWwFjVN RsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLu ZWUyqEyzbV4YBSmAM54roda1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1jX k14PD1zBNrem3NxPbbpka8iMECJjEMMKHBkO0fMFAGMKcnNCAu6tqljJFrJW/gexl05UtlF0 jW7PtjwI7QfPE2QcEk7SMmsKYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmt3VtUsZItZ K38D2MunKlsouka3Z9seBHaD54myDgknaRk1Ru57seHbmGXWtNuLie13TI15F5ECLjEMMKHH mnaPmCgDGFOTmhbAUJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcua0P7QI0jVF1DVba4W TTykJS7je3Zvk2LHahVaNgMDJA2kEkV59RTA9F186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFch gSTltp6k7vSfxFqFhdnWGtNV8nTXs1+yxi7ilhfhNsa2u3dGePvcFSM8dK8zopWA9BvJrweH rmCbW9Nubie23TI15EYIETGIYYUODIdo+YKAMYU5Oa2da1TQ/susvFqMVxZXdgfs0AuV8q3Y KqxxpBncGyNxYqNuB0+avJaKLAdFqFzA3hWygvp47vUhg2rQuC1tB/clYZDZP3U6r3Iztq/a 3dwngLWbe/1OOTzUtfscDXqSMFD5IVAxK4GMjA6e1cdRTAK7S80XQh4YuXhvNPfUIYoZIpIb gJ53H7wbXlLHg9NkZyOBziuLooA6++sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329QX 1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qOQooA7nVNRsrLwLYWumFU+0CUTwfbIZ mXcysplUJlm2rwwClMAZzxUV5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFcXR QB0rx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIUHAB6ZJElhf4avbieD7Qmq+cYTKocp5QXcF JyRk9vf0Nc1RQB0X2vw//wAIZ9m+yyf2p9q3Z3jfjy8bt/l/c3f8s8575rt9Z1PQ/smstFqM VzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOa8moosB197YaBHZ3jQG0OpraRs1qLsmCJj98x SZ/eOBtOzcQCTgvjAdMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzXHUUAdpeaLoQ8MXL w3mnvqEMUMkUkNwE87j94NryljwemyM5HA5xWZfXltolpNpGkzpPNINl9qEZ4lHeKI/88/U/ x/TAPPUUAeta1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmuUvbDQI7O8aA2 h1NbSNmtRdkwRMfvmKTP7xwNp2biAScF8YHIUUkrAdjMWuPhvHavcacLiO9FwIo54EcwiHGS FIJbPGDlzVjWLDwnbWN21mFniFuv2aeKdDKZCFwzAzZIzkMPKUjJ6YzXDUUwCiiigAooooAK KKKACiiigAooooA9R8U69YXV14kSz1D7VI0QRIZ7xWtHjKoS8IxjzVI4Gc5yRk/LUWrXWgPc a5pEOoxPptvYE2NttTyElVQVaOUOSzks2eMtkgk4qXxTr1hdXXiRLPUPtUjRBEhnvFa0eMqh LwjGPNUjgZznJGT8tQa7qdtJpBTU9Str2L+yYoRAlys8n24bv3g2k4xzubIBBA+bpUR2QFPU 9Qik8NzW6alaN4iFugv7lZVJuIct+6WXo7gGPdg5bGMtjFc9a/8AIm33/IF/14/13/H71T/V /wCz/wDZV0Op6hFJ4bmt01K0bxELdBf3KyqTcQ5b90svR3AMe7By2MZbGK561/5E2+/5Av8A rx/rv+P3qn+r/wBn/wCyqlsBhV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio2 4HTmvJa7nVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRbUDoda1TQ/susvFq MVxZXdgfs0AuV8q3YKqxxpBncGyNxYqNuB0+ajWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONL fOQ2RuLFRtwOnNc9qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2 FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UkgOh1rVND+y6y8WoxXFld2B+zQC5Xyrdgq rHGkGdwbI3Fio24HT5qNa1TQ/susvFqMVxZXdgfs0AuV8q3YKqxxpBncGyNxYqNuB0+aue1T UbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vq6tqljJFrJW/gexl05UtlF0jW7P tjwI7QfPE2QcEk7SMmhLYC5rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmj WtT0P7LrLRajFc2V3YH7NCLhfKt2CqscaW+dwbI3ElRtwOnzVT1bVLGSLWSt/A9jLpypbKLp Gt2fbHgR2g+eJsg4JJ2kZNXNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0 LoAa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFld2B+ yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio 24HTmsW7nux4duYZda024uJ7XdMjXkXkQIuMQwwoceado+YKAMYU5OaEtgNnWdT0P7JrLRaj FcWV1p5FrD9pXyrchVWONIM7t2RuLFRtwOnzUazqehi01lotSiubK608i1h+0L5UBCqscaW+ SwbI3Fio24HT5jVTVtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk1Ru57seHbmGXW tNuLie13TI15F5ECLjEMMKHHmnaPmCgDGFOTmhAbOs6noYtNZaLUormyutPItYftC+VAQqrH GlvksGyNxYqNuB0+Y0utanoQtdZaHUY7ixurAi1h+0r5VuQqrHGkGdwbI3Fio24HT5qNa1PQ vs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A/ZYftC+Vb kKqxxpb5yGyNxYqNuB05oXQA1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpz VDxDf2F0NXNpqvk6a1motYxdxSwvgJtjW1I3xnj73BUjPFX9a1PQ/sustDqUdzZXVgRaw/aF 8qAhVWONLcHIYkbi20bcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTm hdAMrXm0zUbFJdSurKaeDRUUXUd8JJzdg/cKq53AknJ2nqTuHbzqvWta1PQvs2tPDqMVxZXd gfssP2hfKtyFVY40t85DZG4sVG3A6c15LTWwHqPijXrC7ufEi2eofapHiVEguLtWtXjKoS8K 4x5qkdM5zkjJ+Wl1bVYH0aVLrVIJ3/sRYZN1+lwjXORkeUpJMmefNyQOvOKTxTr1hdXXiRLP UPtUjRBEhnvFa0eMqhLwjGPNUjgZznJGT8tUdU1FLyy1JpdSt7eB7FRFFbXsc1q7AJhI7ZlD x5x14KEE0o7ICLU9Qik8NzW6alaN4iFugv7lZVJuIct+6WXo7gGPdg5bGMtjFc9a/wDIm33/ ACBf9eP9d/x+9U/1f+z/APZVf+1P/wAKv+yfb49/9oeZ9n+1Lv8AJ24+5nOPM5xj/ax3qha/ 8ibff8gX/Xj/AF3/AB+9U/1f+z/9lTAwq7nVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2r wwClMAZzxXDV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjqBz2qajZW XgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysp lUJlm2rwwClMAZzxXQ61qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9D +y6y0OpR3NldWBFrD9oXyoCFVY40twchiRuLbRtwOnNJMDntU1GysvAtha6YVT7QJRPB9shm ZdzKymVQmWbavDAKUwBnPFaurapYyRayVv4HsZdOVLZRdI1uz7Y8CO0HzxNkHBJO0jJq5rWp 6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfK tyFVY40t85DZG4sVG3A6c0LoBS1bVLKSHWSt/btZS6cqW6i6RrcvtjwI7T78RyDgknaRk1d1 rVND+y6y8WoxXFld2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5qNa1TQ/susvFqMVxZXdgfs0Au V8q3YKqxxpBncGyNxYqNuB0+ajWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwO nNC6AJrOp6H9k1lotRiuLK608i1h+0r5VuQqrHGkGd27I3Fio24HT5qxvt5XR9UTUNVtrhH0 8pCUvEkt2b5Nix2oVWjYAAZIG0gkitrWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuL FRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaE9gOZvNF0IeGLl4b zT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4q7eTXg8PXME2t6bc3E9tumRryIwQImMQwwocGQ 7R8wUAYwpyc1s61qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmjWtT0L7NrTw 6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNCYGLdz3Y8O3MMutabcXE9rumRryLyIEXGI YYUOPNO0fMFAGMKcnNS6+dM1CySTU7qymnt9FRRdR3wlnN2D9zCuQwJJy209Sd3pqazqeh/Z NZaLUYrmyu9PItoftK+VbsFVY40gzkNuG4ttG3A6c0utapof2XWXi1GK4sruwP2aAXK+VbsF VY40gzuDZG4sVG3A6fNQBy+rWGhRC/FjHpz6elsrWl2dQYXMj4XrGC3JJYFTGo68riun1nU9 E+yay8eoxXFnd6efs0IuF8q3YKqxxpb53Btw3Fio24HT5qNZ1LQhaayYdSjubK608i1h+0r5 cBCqscaQZyGJG4sVG3A6c1gxX/hxfAesafYyyxSbISzXESLNcSb88ASHKjA4H3Rk/MTTQG/r Wp6H9l1lotRiubK7sD9mhFwvlW7BVWONLfO4NkbiSo24HT5q8lr0vxDqFhdf2wbPVfJ0xrNR axi8ilhfATbGtqV3xnj73BUjPFeaULYD1HxTr1hdXXiRLPUPtUjRBEhnvFa0eMqhLwjGPNUj gZznJGT8tYut3rTxXDaNqllb+HmswsNhLIpZeRmMxYLebvy2/B9d9bPijXtPurjxGlnqH2qR olRIbi8DWrxlUJeEYx5qkdM5zkjJ+Wodd1O2k0gpqepW17F/ZMUIgS5WeT7cN37wbScY53Nk AggfN0pR2QFPU9Qik8NzW6alaN4iFugv7lZVJuIct+6WXo7gGPdg5bGMtjFc9a/8ibff8gX/ AF4/13/H71T/AFf+z/8AZVf+1P8A8Kv+yfb49/8AaHmfZ/tS7/J24+5nOPM5xj/ax3qha/8A Im33/IF/14/13/H71T/V/wCz/wDZUwMKvWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85D ZG4sVG3A6c15LXcanqNnZeBNPtdOKobhZRcQC7hlI3MrAyoEyzYXhgFKYAznii2twOi1rU9C +za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5Vu QqrHGlvnIbI3Fio24HTmud1PUbOy8Cafa6cVQ3Cyi4gF3DKRuZWBlQJlmwvDAKUwBnPFGp6j Z2XgTT7XTiqG4WUXEAu4ZSNzKwMqBMs2F4YBSmAM54pJWA6LWtT0L7NrTw6jFcWV3YH7LD9o XyrchVWONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdO a53U9Rs7LwJp9rpxVDcLKLiAXcMpG5lYGVAmWbC8MApTAGc8UanqNnZeBNPtdOKobhZRcQC7 hlI3MrAyoEyzYXhgFKYAznihKwHRa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqN uB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzXO6nqNnZeBNPtdOKobh ZRcQC7hlI3MrAyoEyzYXhgFKYAznijU9Rs7LwJp9rpxVDcLKLiAXcMpG5lYGVAmWbC8MApTA Gc8UJWA6LWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtapof2XWXi1GK4 sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNXO6nqNnZeBNPtdOKobhZRcQC7hlI3MrAyoEyzY XhgFKYAznitXVtVsZYdZK38DWUunKluv2pGgL7Y8CO0B3xNkHBJO0jJoStYC7rWqaH9l1l4t RiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVVjjS DO4NkbixUbcDp81ctMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzV/wDtAjR9UXUdVtrh ZNPKQlLuN7dm+TYsdqqq0bAcZI+UgkinawG1rWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7 g2RuLFRtwOnzVlXx0q28LeINO0q5svsTpbSWhN8DLcY2tIzIz/K3HQKpPTBwBVLU9Rs7LwJp 9rpxVDcLKLiAXcMpG5lYGVAmWbC8MApTAGc8VBMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbP GDlzQlYDV1HW59Tg1eXVdSsUhksy8Emm6jL+9kIXanktISAQSGBjHfOOtJr50zULJJNTurKa e30VFF1HfCWc3YP3MK5DAknLbT1J3elWG/8ADo8Baxp9lLJFLshLNPEizXEm/PA3nKjAGB90 ZPzE1V1Sw0OFL0WaabJYpaq1teC/YXEkmF6x5bkksCpRcDuuM0JWAsaxYeE7axu2sws8Qt1+ zTxToZTIQuGYGbJGchh5SkZPTGa4au7iv/Di+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ +YmuEpoAooooAKKKKACiiigAooooAKKKKACiiigAooooA9R8U69YXV14kSz1D7VI0QRIZ7xW tHjKoS8IxjzVI4Gc5yRk/LS6tqsD6NKl1qkE7/2IsMm6/S4RrnIyPKUkmTPPm5IHXnFJ4p16 wurrxIlnqH2qRogiQz3itaPGVQl4RjHmqRwM5zkjJ+WufmLXHw3jtXuNOFxHei4EUc8COYRD jJCkEtnjBy5qY7IDS8TeI/s+m21q0y6jLc6UltOft6TxJKCC7lFzmQdn3fTODXM2v/Im33/I F/14/wBd/wAfvVP9X/s//ZV3utapoZtdZeHUY7iyu7A/ZYPtC+VbkKqxxpbg5DZG4ttG3A6c 1wVr/wAibff8gX/Xj/Xf8fvVP9X/ALP/ANlTWwGFXc6pqNlZeBbC10wqn2gSieD7ZDMy7mVl MqhMs21eGAUpgDOeK4avWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0dQ Oe1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7Z DMy7mVlMqhMs21eGAUpgDOeK6HWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwO nNGtanoQtdZaHUY7ixurAi1h+0r5VuQqrHGkGdwbI3Fio24HT5qSYHPapqNlZeBbC10wqn2g SieD7ZDMy7mVlMqhMs21eGAUpgDOeK1dW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSd pGTVzWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanof2XWWh1KO5srqw ItYftC+VAQqrHGluDkMSNxbaNuB05oXQCnq2qWMkWslb+B7GXTlS2UXSNbs+2PAjtB88TZBw STtIyaua1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9D+y6y0OpR3Nl dWBFrD9oXyoCFVY40twchiRuLbRtwOnNGtanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSN xbaNuB05oXQDGvJrweHrmCbW9Nubie23TI15EYIETGIYYUODIdo+YKAMYU5OaS7nux4duYZd a024uJ7XdMjXkXkQIuMQwwoceado+YKAMYU5Oa2ta1PQ/sustDqUdzZXVgRaw/aF8qAhVWON LcHIYkbi20bcDpzRrWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOaEBz2qajZ WXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznip7ue7Hh25hl1rTbi4ntd0yNeReRAi4 xDDChx5p2j5goAxhTk5ra1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrW p6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmhPYDAiv/Di+A9Y0+xllik2Qlmu IkWa4k354AkOVGBwPujJ+YmiK/8ADi+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ+Ymt/W tT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7Q vlW5CqscaW+chsjcWKjbgdOaEwMvUNbn1O31aXVdRskilsy8EmnajJ+8lIXankmQkA5IYFB3 zjrVSK/8OL4D1jT7GWWKTZCWa4iRZriTfngCQ5UYHA+6Mn5ia39a1PQvs2tPDqMVxZXdgfss P2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNu B05oXQA1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzXktdpeaLoQ8MXLw3m nvqEMUMkUkNwE87j94NryljwemyM5HA5xXF00rID1HxTr1hdXXiRLPUPtUjRBEhnvFa0eMqh LwjGPNUjgZznJGT8tWda1TQza6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05qt4p16 wurrxIlnqH2qRogiQz3itaPGVQl4RjHmqRwM5zkjJ+WufmLXHw3jtXuNOFxHei4EUc8COYRD jJCkEtnjBy5pRWiA3dW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTXHWv/Im33/I F/14/wBd/wAfvVP9X/s//ZVqTFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c1l2v8AyJt9 /wAgX/Xj/Xf8fvVP9X/s/wD2VNKwGFXrWtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsj cWKjbgdOa8lr1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNHUA1rVNDNr rLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchV WONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaNa1PQ vs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1K6AGtanoX2bWnh1GK4sruwP2WH7Q vlW5CqscaW+chsjcWKjbgdOaNa1TQza6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05 o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh +0L5VuQqrHGlvnIbI3Fio24HTmhdADWtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjcW2 jbgdOaNa1TQza6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05o1rU9C+za08OoxXFld 2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3 Fio24HTmmugBrWqaGbXWXh1GO4sruwP2WD7QvlW5CqscaW4OQ2RuLbRtwOnNGtapoZtdZeHU Y7iyu7A/ZYPtC+VbkKqxxpbg5DZG4ttG3A6c0a1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVVjjSD O4NkbixUbcDp81GtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaF0ANa1TQz a6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05o1rU9C+za08OoxXFld2B+yw/aF8q3I VVjjS3zkNkbixUbcDpzRrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1q ehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05oXQA1rU9C+za08OoxXFld2B+yw/ aF8q3IVVjjS3zkNkbixUbcDpzSazqmhm01l4tRiuLK708/ZoftC+VbsFVY40t8khtw3Fto24 HTk0utapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNRrWp6F9m1p4dRiuLK7s D9lh+0L5VuQqrHGlvnIbI3Fio24HTmhdADWtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchs jcW2jbgdOaTWNT0P7HrDRajHcWV3p5FtD9oXyrdgqrHGlvksGyNxYqNuB7ml1rU9C+za08Oo xXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6ELXWWh1GO4sbqwItYftK+VbkKqxxpBn cGyNxYqNuB0+ahdADWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNeS161rW p6ELXWWh1GO4sbqwItYftK+VbkKqxxpBncGyNxYqNuB0+avJacdgPUfFOvWF1deJEs9Q+1SN EESGe8VrR4yqEvCMY81SOBnOckZPy1Z1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbi xUbcDpzVbxTr1hdXXiRLPUPtUjRBEhnvFa0eMqhLwjGPNUjgZznJGT8tc/MWuPhvHavcacLi O9FwIo54EcwiHGSFIJbPGDlzSitEBs3OrWstnq7XlxbfZpdM8uCK31BZLbzNqBBHbFQ8ZBGe fu4P1rkrX/kTb7/kC/68f67/AI/eqf6v/Z/+yrpLya8Hh65gm1vTbm4ntt0yNeRGCBExiGGF DgyHaPmCgDGFOTmubtf+RNvv+QL/AK8f67/j96p/q/8AZ/8Asqa2Awq9a1rU9C+za08OoxXF ld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzXkteta1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxp b5yGyNxYqNuB05o6gGtanoQtdZaHUY7ixurAi1h+0r5VuQqrHGkGdwbI3Fio24HT5qNa1PQv s2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A/ZYftC+Vbk Kqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzUroA a1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFld2B+yw/ aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24H TmjWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmoXQA1rU9CFrrLQ6jHcWN1 YEWsP2lfKtyFVY40gzuDZG4sVG3A6fNRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3 Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtapof2XWXi1GK4 sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNTXQA1rU9CFrrLQ6jHcWN1YEWsP2lfKtyFVY40g zuDZG4sVG3A6fNRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0L7Nr Tw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNJrOqaGbTWXi1CKeyu9PP2aD7SvlW7BVW ONIM5DbhuLbRtwOnNC6ALrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWt T0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtapoZtdZeHUY7iyu7A/ZYPtC+ VbkKqxxpbg5DZG4ttG3A6c0ms6poZtNZeLUIp7K708/ZoPtK+VbsFVY40gzkNuG4ttG3A6c0 JbALrWp6ELXWWh1GO4sbqwItYftK+VbkKqxxpBncGyNxYqNuB0+ajWtT0P7LrLRajFc2V3YH 7NCLhfKt2CqscaW+dwbI3ElRtwOnzUa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxY qNuB05o1rVND+y6y8WoxXFld2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5qF0ANa1PQ/sustFqM VzZXdgfs0IuF8q3YKqxxpb53BsjcSVG3A6fNRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlv nIbI3Fio24HTmjWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmo1rU9C+za0 8OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzQugHM3mi6EPDFy8N5p76hDFDJFJDcBPO4 /eDa8pY8HpsjORwOcVxdeg3k14PD1zBNrem3NxPbbpka8iMECJjEMMKHBkO0fMFAGMKcnNef VSAKKKKACiiigAooooAKKKKACiiigAooooAKKKKAPUfFOvWF1deJEs9Q+1SNEESGe8VrR4yq EvCMY81SOBnOckZPy1z8xa4+G8dq9xpwuI70XAijngRzCIcZIUgls8YOXNXfFmvyxWOnwxal PPcSackFw0GpJLEXwRIJIwG3MQx+bcM5BH3al1TUUvLLUml1K3t4HsVEUVtexzWrsAmEjtmU PHnHXgoQTUrRJAN/tAjSNUXUNVtrhZNPKQlLuN7dm+TYsdqFVo2AwMkDaQSRXN2v/Im33/IF /wBeP9d/x+9U/wBX/s//AGVFr/yJt9/yBf8AXj/Xf8fvVP8AV/7P/wBlWFVAFdzqmo2Vl4Fs LXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54rhq9a1rU9C+za08OoxXFld2B+yw/aF8q3IV VjjS3zkNkbixUbcDpzS6gc9qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1T UbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8V0OtanoX2bWnh1GK4sruwP2WH7Qv lW5CqscaW+chsjcWKjbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0 k9gOe1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vq6tqljJFrJW/gexl05Ut lF0jW7PtjwI7QfPE2QcEk7SMmrmtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbg dOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0LoBT1bVLGSLWSt/A9jLp ypbKLpGt2fbHgR2g+eJsg4JJ2kZNXNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sV G3A6c0a1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFld 2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzSXQDGvJrweHrmCbW9Nubie23TI15EYIETGIYYUODI do+YKAMYU5OaLya8Hh65gm1vTbm4ntt0yNeRGCBExiGGFDgyHaPmCgDGFOTmtnWtT0L7NrTw 6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanof2XWWh1KO5srqwItYftC+VAQqrHGl uDkMSNxbaNuB05pp7AZd/wD2VbeF/EGnaTdWX2J1tpLMm9BluMbWkZkZ/lbjoFUnpg4FRf2g RpGqLqGq21wsmnlISl3G9uzfJsWO1Cq0bAYGSBtIJIrZ1rU9D+y6y0OpR3NldWBFrD9oXyoC FVY40twchiRuLbRtwOnNGtanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSNxbaNuB05oXQD K186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vSnrFj4StrG7ayAmh+zr9lmjnTzi 5C4LgzZPOQw8oEZOMYzXQ61qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmjWt T0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNC6AYEV/wCHF8B6xp9jLLFJshLN cRIs1xJvzwBIcqMDgfdGT8xNb+tapoZtdZeHUY7iyu7A/ZYPtC+VbkKqxxpbg5DZG4ttG3A6 c0a1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmjWtT0P7LrLQ6lHc2V1YEWs P2hfKgIVVjjS3ByGJG4ttG3A6c0LoAa1qmhm11l4dRjuLK7sD9lg+0L5VuQqrHGluDkNkbi2 0bcDpzXktes6xqeifY9YaPUormzutPItoRcL5UDBVWONLfOQxYbixUbcDpzXEa1cwTeHNIj0 6eNLONNtza7wJPtX8UjL1YEYw3IA4+XpTWwHX+KdesLq68SJZ6h9qkaIIkM94rWjxlUJeEYx 5qkcDOc5Iyflrn5i1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcua6jWdT0P7JrLRalFc2V1 p5FtF9oXyoGCqscaW+chiw3E7RtwOnNGs6nof2TWWi1KK5srrTyLaL7QvlQMFVY40t85DFhu J2jbgdOaUdEkBgxX/hxfAesafYyyxSbISzXESLNcSb88ASHKjA4H3Rk/MTWDa/8AIm33/IF/ 14/13/H71T/V/wCz/wDZV3ms6nof2TWWi1KK5srrTyLaL7QvlQMFVY40t85DFhuJ2jbgdOaN Z1PQ/smstFqUVzZXWnkW0X2hfKgYKqxxpb5yGLDcTtG3A6c00wPJq9a1rVND+y6y8WoxXFld 2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5q4fVLmB/CWmW91PHc6orlonjcOYbXHEbsO+7JCnJU cfLnFc7RYD1rWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmo1rVND+y6y8W oxXFld2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5qy/EN1BdSavNa+I4otKa2UW1mBHJFIoRNqL Fv3RtuH9wbcZyKZqeopd2OpNLqcEEDWKiKO2vY5raRgEwkdsyB48468FCCaSWwGvrWqaH9l1 l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVV jjSDO4NkbixUbcDp81cFa/8AIm33/IF/14/13/H71T/V/wCz/wDZVt6pqNnZeBNPtdNKobhZ RcQC8hmI3MrKZVCZZtq8MApTAGc8U7AdDrOp6H9k1lotRiuLK608i1h+0r5VuQqrHGkGd27I 3Fio24HT5qXWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmrntU1GysvAtha 6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLN tXhgFKYAznihIDoNY1TQ/sesPHqMdxZ3enn7NB9pXyrdgqrHGkAOQ24bi20bcDpzS61qmh/Z dZeLUYriyu7A/ZoBcr5VuwVVjjSDO4NkbixUbcDp81c7qmo2dl4E0+100qhuFlFxALyGYjcy splUJlm2rwwClMAZzxWrq2qWMkOs7dQgeyk05Utx9qRoGfbHgR2g+eJsg4JJ2kZNFtQLmsap of2PWHj1GO4s7vTz9mg+0r5VuwVVjjSAHIbcNxbaNuB05o1nU9D+yay0WoxXNld6eRbQ/aV8 q3YKqxxpBnIbcNxbaNuB05qnq2qWMkOs7dQgeyk05Utx9qRoGfbHgR2g+eJsg4JJ2kZNXNZ1 PQ/smstFqUVzZXWnkW0X2hfKgYKqxxpb5yGLDcTtG3A6c0kAazqeh/ZNZaLUYrmyu9PItoft K+VbsFVY40gzkNuG4ttG3A6c0azqeh/ZNZaLUYrmyu9PItoftK+VbsFVY40gzkNuG4ttG3A6 c0azqeh/ZNZaLUormyutPItovtC+VAwVVjjS3zkMWG4naNuB05rHvJrweHrmCbW9Nubie23T I15EYIETGIYYUODIdo+YKAMYU5OaEBsazqeh/ZNZaLUYrmyu9PItoftK+VbsFVY40gzkNuG4 ttG3A6c0azqeh/ZNZaLUYrmyu9PItoftK+VbsFVY40gzkNuG4ttG3A6c1T1bVLGSHWduoQPZ SacqW4+1I0DPtjwI7QfPE2QcEk7SMmqd5NeDw9cwTa3ptzcT226ZGvIjBAiYxDDChwZDtHzB QBjCnJzQgM+YtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmuOr1nWdT0P7JrLRalFc2V1p 5FtF9oXyoGCqscaW+chiw3E7RtwOnNLrWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW 2jbgdOadwKvijXbG6uPEaWuofapWiVFhnvFa0eMqhZ4RjBlUjgZznJGT8tYExa4+G8dq9xpw uI70XAijngRzCIcZIUgls8YOXNdRrOp6H9k1lotSiubK608i2i+0L5UDBVWONLfOQxYbido2 4HTmjWdT0P7JrLRalFc2V1p5FtF9oXyoGCqscaW+chiw3E7RtwOnNJbJAZmoa1PqVvq0mq6n ZLFJZloJdO1CQebKQu1PJLkgHJDAxjvnHWuWtf8AkTb7/kC/68f67/j96p/q/wDZ/wDsq7zW dT0P7JrLRalFc2V1p5FtF9oXyoGCqscaW+chiw3E7RtwOnNcRrVzBN4c0iPTp40s4023NrvA k+1fxSMvVgRjDcgDj5elNbAc7XrOs6nof2TWWi1GK5srvTyLaH7SvlW7BVWONIM5DbhuLbRt wOnNGs6non2TWXj1GK4s7vTz9mhFwvlW7BVWONLfO4NuG4sVG3A6fNRrOp6J9k1l49RiuLO7 08/ZoRcL5VuwVVjjS3zuDbhuLFRtwOnzUAGs6nof2TWWi1GK5srvTyLaH7SvlW7BVWONIM5D bhuLbRtwOnNGs6nof2TWWi1GK5srvTyLaH7SvlW7BVWONIM5DbhuLbRtwOnNGs6non2TWXj1 GK4s7vTz9mhFwvlW7BVWONLfO4NuG4sVG3A6fNRrOp6J9k1l49RiuLO708/ZoRcL5VuwVVjj S3zuDbhuLFRtwOnzUl0ANZ1PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOaNZ1 PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOa4jVLmB/CWmW91PHc6orlonjcOY bXHEbsO+7JCnJUcfLnFc7TsB6zrOp6H9k1lotRiubK708i2h+0r5VuwVVjjSDOQ24bi20bcD pzRrOp6H9k1lotRiubK708i2h+0r5VuwVVjjSDOQ24bi20bcDpzWN4ibS76wjlmvLa91eDSI Iysl2NgYFvMYOrYklB/hJGc5+fpVOez8NSxzKgtITJo4vVeO6YmO64/cruYjHB+UgtyeemCw HS6zqeh/ZNZaLUYrmyu9PItoftK+VbsFVY40gzkNuG4ttG3A6c0axqehiz1hotSjubK608i2 h+0r5UDBVWONIM7g2RuLFRtwOnzVxaSwv8NXtxPB9oTVfOMJlUOU8oLuCk5Iye3v6Guh8Q3U F1Jq89r4iih0p7ZRbWQ8uWJ1CJtRYt+6Ntw6+WNuM5FAGnrOp6H9k1lotRiubK708i2h+0r5 VuwVVjjSDOQ24bi20bcDpzRrGqaH9j1h49RjuLO708/ZoPtK+VbsFVY40gByG3DcW2jbgdOa ydT1FLux1JpdTgggaxURR217HNbSMAmEjtmQPHnHXgoQTXM2v/Im33/IF/14/wBd/wAfvVP9 X/s//ZUJAd5rOp6H9k1lotRiubK708i2h+0r5VuwVVjjSDOQ24bi20bcDpzRrGqaH9j1h49R juLO708/ZoPtK+VbsFVY40gByG3DcW2jbgdOaNZ1PQ/smstFqMVzZXenkW0P2lfKt2CqscaQ ZyG3DcW2jbgdOaNZ1PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOaSANY1TQ/s esPHqMdxZ3enn7NB9pXyrdgqrHGkAOQ24bi20bcDpzRrGqaH9j1h49RjuLO708/ZoPtK+Vbs FVY40gByG3DcW2jbgdOaNZ1PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOaNZ1 PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOaEAaxqmh/Y9YePUY7izu9PP2aD7 SvlW7BVWONIAchtw3Fto24HTmvJq9a1rU9D+y6y0OpR3NldWBFrD9oXyoCFVY40twchiRuLb RtwOnNGtanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSNxbaNuB05ppgeS0V61rWp6H9l1l odSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOaNa1PQ/sustDqUdzZXVgRaw/aF8qAhVWON LcHIYkbi20bcDpzQmB5LRXrWtanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSNxbaNuB05o 1rU9D+y6y0OpR3NldWBFrD9oXyoCFVY40twchiRuLbRtwOnNCYHktFeta1qeh/ZdZaHUo7my urAi1h+0L5UBCqscaW4OQxI3Fto24HTmjWtT0P7LrLQ6lHc2V1YEWsP2hfKgIVVjjS3ByGJG 4ttG3A6c0JgeS0V61rWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOaNa1PQ/s ustDqUdzZXVgRaw/aF8qAhVWONLcHIYkbi20bcDpzQmB5LRXrOsanogs9YaPU4rqzutPItoh cKIoGCqscaW+SQxYbiSo24A45NLrWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbg dOaLgeS0UUUwCiiigDt9bvWniuG0bVLK38PNZhYbCWRSy8jMZiwW83flt+D676uapqKXllqT S6lb28D2KiKK2vY5rV2ATCR2zKHjzjrwUIJqnrd808Nw+j6pZW3h9rJVisJZVLLyMxmLBbzd +W34999bXijXrC7ufEiWmoC5laJUWCe8VrRoyqEvCMYMikcDOc5IyflqV0A4u1/5E2+/5Av+ vH+u/wCP3qn+r/2f/sqwq9BvJrweHrmCbW9Nubie23TI15EYIETGIYYUODIdo+YKAMYU5Oa5 u1/5E2+/5Av+vH+u/wCP3qn+r/2f/sqoDCr1nWdU0NrTWZItQjnsrvTz9mg+0r5duwVVjjSA EkNkbi20bcDpya8mr1rWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmpdQDW dU0M2usvFqEc9ld2B+ywfaF8u3IVVjjS3BJDZG4ttG3A6c1y0xa4+G8dq9xpwuI70XAijngR zCIcZIUgls8YOXNdTrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1qmh/ ZdZeLUYriyu7A/ZoBcr5VuwVVjjSDO4NkbixUbcDp81JbJAGtanoX2bWnh1GK4sruwP2WH7Q vlW5CqscaW+chsjcWKjbgdOa8lr1rWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKj bgdPmrmV8U38nh3VNRvtWa7vr5jYizeQKscbJlpRGO/G0EAAEknOcUR2A1tc1S2l0fbqWpW9 7EdIhiECXKzSfbhu/eDBJGBnc2QCCB83SoNT1FLux1JpdTgggaxURR217HNbSMAmEjtmQPHn HXgoQTVXVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaB KJ4PtkMzLuZWUyqEyzbV4YBSmAM54pgYdr/yJt9/yBf9eP8AXf8AH71T/V/7P/2VYVdovim/ k8O6pqN9qzXd9fMbEWbyBVjjZMtKIx342ggAAkk5zipdU1GysvAtha6YVT7QJRPB9shmZdzK ymVQmWbavDAKUwBnPFAHQ6zqmhm11l4tQjnsruwP2WD7Qvl25CqscaW4JIbI3Fto24HTmjWd U0M2usvFqEc9ld2B+ywfaF8u3IVVjjS3BJDZG4ttG3A6c1l6jrc+pwavLqupWKQyWZeCTTdR l/eyELtTyWkJAIJDAxjvnHWqWsWHhO2sbtrMLPELdfs08U6GUyELhmBmyRnIYeUpGT0xmkls BXmLXHw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5rd1bVLGSLWSt/A9jLpypbKLpGt2fbHgR 2g+eJsg4JJ2kZNUrya8Hh65gm1vTbm4ntt0yNeRGCBExiGGFDgyHaPmCgDGFOTmpNfOmahZJ Jqd1ZTT2+ioouo74Szm7B+5hXIYEk5baepO70fUDzqvSvGeoz3esalJpmsILR0bLHWIngkTy iGVYB824ngdeeeM5EV8dKtvC3iDTtKubL7E6W0loTfAy3GNrSMyM/wArcdAqk9MHAFU9U1Gy svAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFHW4FrVNRS8stSaXUre3gexURRW17HN auwCYSO2ZQ8ecdeChBNcza/8ibff8gX/AF4/13/H71T/AFf+z/8AZV3utapof2XWXi1GK4sr uwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNXktC2AK9a1rU9C+za08OoxXFld2B+yw/aF8q3IVV jjS3zkNkbixUbcDpzXOa3fNPDcPo+qWVt4fayVYrCWVSy8jMZiwW83flt+PffWz4o1+wu7jx Gtpf/aZXiVEgnvFa0aMqhLwjGPNUjgZznJGT8tLqBa1rU9C+za08OoxXFld2B+yw/aF8q3IV VjjS3zkNkbixUbcDpzXLTFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c1of2gRpGqLqGq2 1wsmnlISl3G9uzfJsWO1Cq0bAYGSBtIJIrm7X/kTb7/kC/68f67/AI/eqf6v/Z/+yprRAdjq 2qWMkWslb+B7GXTlS2UXSNbs+2PAjtB88TZBwSTtIya8xor1rWtT0L7NrTw6jFcWV3YH7LD9 oXyrchVWONLfOQ2RuLFRtwOnNG2gHOa3etPFcNo2qWVv4eazCw2Esill5GYzFgt5u/Lb8H13 1teKdesLq68SJZ6h9qkaIIkM94rWjxlUJeEYx5qkcDOc5IyflqzrWqaH9l1l4tRiuLK7sD9m gFyvlW7BVWONIM7g2RuLFRtwOnzUa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqN uB05qVugDWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNcFa/wDIm33/ACBf 9eP9d/x+9U/1f+z/APZV3utapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNRr WqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzU1skB5LXrWtanoX2bWnh1GK4s ruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaNa1TQ/susvFqMVxZXdgfs0AuV8q3YKqxxpBncGy NxYqNuB0+ajWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmo3aYBrWp6F9m1 p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVW ONLfOQ2RuLFRtwOnNY15NeDw9cwTa3ptzcT226ZGvIjBAiYxDDChwZDtHzBQBjCnJzSXc92P DtzDLrWm3FxPa7pka8i8iBFxiGGFDjzTtHzBQBjCnJzQlsBta1qmh/ZdZeLUYriyu7A/ZoBc r5VuwVVjjSDO4NkbixUbcDp81JrOp6H9k1lotRiubK708i2h+0r5VuwVVjjSDOQ24bi20bcD pzWXrzaZqNikupXVlNPBoqKLqO+Ek5uwfuFVc7gSTk7T1J3Dsuo63PqcGry6rqVikMlmXgk0 3UZf3shC7U8lpCQCCQwMY75x1oStYDU1rU9D+y6y0WoxXNld2B+zQi4XyrdgqrHGlvncGyNx JUbcDp81Gtapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNWN/aBGkaouoarbX CyaeUhKXcb27N8mxY7UKrRsBgZIG0gkipL46VbeFvEGnaVc2X2J0tpLQm+BluMbWkZkZ/lbj oFUnpg4AoS2A1NZ1PQ/smstFqMVzZXenkW0P2lfKt2CqscaQZyG3DcW2jbgdOaqatqljJFrJ W/gexl05UtlF0jW7PtjwI7QfPE2QcEk7SMmrmtapof2XWXi1GK4sruwP2aAXK+VbsFVY40gz uDZG4sVG3A6fNRrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzULoAa1qehf ZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9D+y6y0WoxXNld2B+zQi4Xyrdg qrHGlvncGyNxJUbcDp81Gtapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNXkt CWiA9G1DWp9St9Wk1XU7JYpLMtBLp2oSDzZSF2p5JckA5IYGMd8461qa1qehfZtaeHUYriyu 7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05rnNbvmnhuH0fVLK28PtZKsVhLKpZeRmMxYLebvy2/ HvvrZ8Ua9Y3Vx4jS11AXUzRKiwT3itaOhVCzwjGPMUjgZznJGT8tHVAWta1PQ/sustFqMVzZ Xdgfs0IuF8q3YKqxxpb53BsjcSVG3A6fNWN/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsB gZIG0gkioYr/AMOL4D1jT7GWWKTZCWa4iRZriTfngCQ5UYHA+6Mn5iawbX/kTb7/AJAv+vH+ u/4/eqf6v/Z/+yppAdRr50zULJJNTurKae30VFF1HfCWc3YP3MK5DAknLbT1J3ennVFeta1q eh/ZdZaLUYrmyu7A/ZoRcL5VuwVVjjS3zuDZG4kqNuB0+ajyA8lor1rWtT0L7NrTw6jFcWV3 YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjc WKjbgdOaEwPJaK9a1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m 1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmhMDyWivWta1PQvs2tPDqMVxZXdgfssP 2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB 05oTA8lor1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanoX2bWnh1G K4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaEwPJaK9a1rU9C+za08OoxXFld2B+yw/aF8q3 IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmhMD yWivWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7 A/ZYftC+VbkKqxxpb5yGyNxYqNuB05oTA8lor0zxFqFhdnWGtNV8nTXs1+yxi7ilhfhNsa2u 3dGePvcFSM8dKg186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vQTuB51RXrWtano X2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOa8loTugCiiimAUUUUAFFFFABRRRQA UUUUAFFFFABRRRQB6Vrup20mkFNT1K2vYv7JihECXKzyfbhu/eDaTjHO5sgEED5ulT+KdesL q68SJZ6h9qkaIIkM94rWjxlUJeEYx5qkcDOc5IyflqDXdTtpNIKanqVtexf2TFCIEuVnk+3D d+8G0nGOdzZAIIHzdKm8Ua7YXNx4jjtNQ+1StEqJDPeK1o8ZVCZIR081SOBuznJGT8tT1QFr WtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjcW2jbgdOa4K1/5E2+/wCQL/rx/rv+P3qn +r/2f/sq6T+0CNI1RdQ1W2uFk08pCUu43t2b5Nix2oVWjYDAyQNpBJFc3a/8ibff8gX/AF4/ 13/H71T/AFf+z/8AZU1sBhV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HT mvJa9a1rVNDNrrLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto24HTmjqAms6poZtNZeLUYri yu9PP2aH7QvlW7BVWONLfJIbcNxbaNuB05NGs6poZtNZeLUYriyu9PP2aH7QvlW7BVWONLfJ IbcNxbaNuB05NLrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmk1nU9D+yay 0WoxXFldaeRaw/aV8q3IVVjjSDO7dkbixUbcDp81JdADWdU0M2msvFqMVxZXenn7ND9oXyrd gqrHGlvkkNuG4ttG3A6cml1rVNDNrrLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto24HTmjW tU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjcW2jbgdOaTWdU0M2msvFqMVxZXenn7ND9o XyrdgqrHGlvkkNuG4ttG3A6cmhdAF1rVNDNrrLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto 24HTmjWtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjcW2jbgdOaTWdT0P7JrLRalFc2V1 p5FtF9oXyoGCqscaW+chiw3E7RtwOnNLrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3 Fio24HTmhdADWtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjcW2jbgdOaTWdU0M2msvFq MVxZXenn7ND9oXyrdgqrHGlvkkNuG4ttG3A6cml1rU9D+y6y0OpR3NldWBFrD9oXyoCFVY40 twchiRuLbRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaF0ANa1PQ vs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qeh/ZdZaLUYrmyu7A/ZoRcL5Vu wVVjjS3zuDZG4kqNuB0+ak1jVND+x6w8eox3Fnd6efs0H2lfKt2CqscaQA5DbhuLbRtwOnNL rWqaGbXWXh1GO4sruwP2WD7QvlW5CqscaW4OQ2RuLbRtwOnNCVrAGtanoX2bWnh1GK4sruwP 2WH7QvlW5CqscaW+chsjcWKjbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sV G3A6c0ms6nof2TWXi1GO4srvTz9lh+0L5duQqrHGluDkNkbi20bcDpzS61qehfZtaeHUYriy u7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05oXQA1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3z kNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmk1jU9D+x6w0 eox3Nnd6efs0P2hfKt2CqscaW4OQ24bido24HTmjWdT0T7JrLx6jFcWd3p5+zQi4XyrdgqrH GlvncG3DcWKjbgdPmoS2AXWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNeS 161rWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOa8lpx2A9K13U7aTSCmp6lb XsX9kxQiBLlZ5Ptw3fvBtJxjnc2QCCB83Sp/FOvWF1deJEs9Q+1SNEESGe8VrR4yqEvCMY81 SOBnOckZPy1Brup20mkFNT1K2vYv7JihECXKzyfbhu/eDaTjHO5sgEED5ulTeKNdsbq48Rpa 6h9qlaJUWGe8VrR4yqFnhGMGVSOBnOckZPy0uqAwJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpB LZ4wcuay7X/kTb7/AJAv+vH+u/4/eqf6v/Z/+yrrvEOoWF3/AGw1pqgh017Nfssa3kUsL8Jt jW1K74zx97gqRnjpXI2v/Im33/IF/wBeP9d/x+9U/wBX/s//AGVNAYVeta1qehfZtaeHUYri yu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05ryWvWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40 t85DZG4sVG3A6c0dQDWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanoX 2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKty FVY40t85DZG4sVG3A6c0a1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVVjjSDO4NkbixUbcDp81Sug BrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD 9oXyrchVWONLfOQ2RuLFRtwOnNGtapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A 6fNRrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzU0tgDWtT0L7NrTw6jFcW V3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chs jcWKjbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUY riyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05pLoAa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxx pb5yGyNxYqNuB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m 1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmue1TUbKy8C2FrphVPtAlE8H2yGZl3Mr KZVCZZtq8MApTAGc8U0tgOh1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzR rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmue1TUbKy8C2FrphVPtAlE8H2 yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK EtgOh1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7s D9lh+0L5VuQqrHGlvnIbI3Fio24HTmue1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MA pTAGc8V0Otapof2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNQlawBrWp6F9m1p4 dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWON LfOQ2RuLFRtwOnNc9qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C 2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UJbAdDrWp6F9m1p4dRiuLK7sD9lh+0L5Vu QqrHGlvnIbI3Fio24HTmvJa9a1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDp zXktOOwHpWu6nbSaQU1PUra9i/smKEQJcrPJ9uG794NpOMc7myAQQPm6VP4p16wurrxIlnqH 2qRogiQz3itaPGVQl4RjHmqRwM5zkjJ+WoNd1O2k0gpqepW17F/ZMUIgS5WeT7cN37wbScY5 3NkAggfN0qfxRr1hd3PiRLTUBcytEqLBPeK1o0ZVCXhGMGRSOBnOckZPy0uqAzrue7Hh25hl 1rTbi4ntd0yNeReRAi4xDDChx5p2j5goAxhTk5rnLX/kTb7/AJAv+vH+u/4/eqf6v/Z/+yrX XxTfyeHdU1G+1Zru+vmNiLN5AqxxsmWlEY78bQQAASSc5xWRa/8AIm33/IF/14/13/H71T/V /wCz/wDZUwMKvWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c15LXrWtano X2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaOoBrWp6F9m1p4dRiuLK7sD9lh+0L 5VuQqrHGlvnIbI3Fio24HTmjWtT0P7LrLQ6lHc2V1YEWsP2hfKgIVVjjS3ByGJG4ttG3A6c0 a1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05pNZ1TQzaay8WoRT2V3p5+zQf aV8q3YKqxxpBnIbcNxbaNuB05pLoAutanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWK jbgdOaNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu 7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05pNZ1TQzaay8WoRT2V3p5+zQfaV8q3YKqxxpBnIbcN xbaNuB05oS2AXWtT0IWustDqMdxY3VgRaw/aV8q3IVVjjSDO4NkbixUbcDp81JrOpaELTWTD qUdzZXWnkWsP2lfLgIVVjjSDOQxI3Fio24HTml1rVND+y6y8WoxXFld2B+zQC5XyrdgqrHGk GdwbI3Fio24HT5qNa1PQ/sustDqUdzZXVgRaw/aF8qAhVWONLcHIYkbi20bcDpzQugBrWp6E LXWWh1GO4sbqwItYftK+VbkKqxxpBncGyNxYqNuB0+ajWtT0IWustDqMdxY3VgRaw/aV8q3I VVjjSDO4NkbixUbcDp81Gtanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSNxbaNuB05o1rU 9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzQugCazqWhC01kw6lHc2V1p5FrD9 pXy4CFVY40gzkMSNxYqNuB05pda1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6 c0a1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmjWtT0P7LrLQ6lHc2V1YEWs P2hfKgIVVjjS3ByGJG4ttG3A6c0LoAa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxY qNuB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK 7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3Bsj cWKjbgdPmpLoAa1qmhm11l4dRjuLK7sD9lg+0L5VuQqrHGluDkNkbi20bcDpzRrWp6F9m1p4 dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmsCK/8ADi+A9Y0+xllik2QlmuIkWa4k354A kOVGBwPujJ+Ymt/WtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmppbAGtapo f2XWXi1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNXktes6zqeh/ZNZaLUYrmyu9PItof tK+VbsFVY40gzkNuG4ttG3A6c15NTWwBRRRTAKKKKACiiigAooooAKKKKACiiigAooooA9D8 TeI/s+m21q0y6jLc6UltOft6TxJKCC7lFzmQdn3fTODWNr8Oi6dHFHBZWkzXVorM9rf7zbXH G5RhmBTjoQSdxw3HFfVLmB/CWmW91PHc6orlonjcOYbXHEbsO+7JCnJUcfLnFc7SA7fUNO8N RjUxAbL+z47MPY3aXha5lmwmA0e84ySwI8tce1RatYaFEL8WMenPp6WytaXZ1BhcyPhesYLc klgVMajryuK42iiwHdazYeErayuzZBZ4RAv2WeKdPNLkLhnBmyechh5SkZPTGaqX1joUuhSz 2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qOQoosB199Y6FLoUs9mtpZyLbo6CacSys3y5XKTH 5jz1iUDvt6jkKKKYHX31h4fS0vXt/sZ1JbSNjardk28TH75ikz+8cDadhYgEnBfGBa1TUbKy 8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vw1FAHX31joUuhSz2a2lnItujoJpxLKz fLlcpMfmPPWJQO+3qC+sdCl0GSe0FpaSLbxugmnEsrN8uV+SY/MeesSgd9vUchRQB30vh3RZ Dq+qxW08OmNpxnsUmimjEcm1cbnbALFj8oBcHJ54Gas9n4aljmVBaQmTRxeq8d0xMd1x+5Xc xGOD8pBbk89McXRQB2Wq2GhQi+FimnPYJbK1refb2FxI+F6x5bkksCpRQOeVxmtO+/sq28Le INO0q6sjZOltJaMb0ebcY2tIzIz/ACtx0CqT0wcCvOqKQBXRa1cwTeHNIj06eNLONNtza7wJ PtX8UjL1YEYw3IA4+XpXO0UwO813RdF0jTxJeWX2Sa709ZoYVE/mx3Py5QbspsH8QYlxk/7N ZDx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIUHAB6ZJ5qigD0a/8A7KtvC/iDTtJurL7E620l mTegy3GNrSMyM/ytx0CqT0wcCuWtf+RNvv8AkC/68f67/j96p/q/9n/7KsKigAr0G8mvB4eu YJtb025uJ7bdMjXkRggRMYhhhQ4Mh2j5goAxhTk5rz6igD0zxFqFhdnWGtNV8nTXs1+yxi7i lhfhNsa2u3dGePvcFSM8dKx18U38nh3VNRvtWa7vr5jYizeQKscbJlpRGO/G0EAAEknOcVxd FJKwHaJ4ovn8OanqN7qrXd7esbEWTyBUjjZMtKIx9NoIAAJJOc4rXudXtZrTV3vLi1+zS6Z5 cENvqCyW3mbUCCK2Kh4yCM8j5cH615pRRYC+9hBHocV819GbmWcotouGYRgcuxB+XngAgE9R xVCiimB2mpXdhqvgjT1hGl2slmJy8TTSB4mMilVjUsWbcD1IYDnlcVf1uXStVCw6jc2kt3b6 Ari8W83ublD/AKvcGKMSc9iTng9K88opAdV4gh0TT44kgsrOZrq0V2e1vy5tbjjcowzApx0b JO44bjjQ1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM1wtFFgO0vNF0IeGLl4bzT 31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4rSv/AOyrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8 rcdAqk9MHArzmiiwHSvHbN8O0ImgW7W/3mEXnzPHtI3mIvjduOMhQcAHpkm/PZ+GpY5lQWkJ k0cXqvHdMTHdcfuV3MRjg/KQW5PPTHF0UwO51TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZt q8MApTAGc8U3UNO8NRjUxAbL+z47MPY3aXha5lmwmA0e84ySwI8tce1cRRSsAV3OqajZWXgW wtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAzniuGopgdzqmo2Vl4FsLXTCqfaBKJ4PtkMzLu ZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vw1FAH c6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK1dW1Sxki1krfwPYy6cqWyi6R rdn2x4EdoPnibIOCSdpGTXmNFKwHpurapYvDrOy/gexk05Y7ZftStAz7Y8CO0zvibIOCSdpG TV3WtT0IWustDqMdxY3VgRaw/aV8q3IVVjjSDO4NkbixUbcDp81eS0UWA7GYtcfDeO1e404X Ed6LgRRzwI5hEOMkKQS2eMHLml1aw0KIX4sY9OfT0tla0uzqDC5kfC9YwW5JLAqY1HXlcVxt FMD0XXm0zUbFJdSurKaeDRUUXUd8JJzdg/cKq53AknJ2nqTuHZdR1ufU4NXl1XUrFIZLMvBJ puoy/vZCF2p5LSEgEEhgYx3zjrXnNFKwHYzFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c 1teIdQsLv+2GtNUEOmvZr9ljW8ilhfhNsa2pXfGePvcFSM8dK80oosB61rWqaH9l1l4tRiuL K7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVVjjSDO4N kbixUbcDp81eS0UJAeta1qmh/ZdZeLUYriyu7A/ZoBcr5VuwVVjjSDO4NkbixUbcDp81eaPY QR6HFfNfRm5lnKLaLhmEYHLsQfl54AIBPUcVQooSsAV2mpXdhqvgjT1hGl2slmJy8TTSB4mM ilVjUsWbcD1IYDnlcVxdFMDtLzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKiv rHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUchRQB6D4kbTL6ySae9tr3WINJgQrJdg oGBbzGDq2JJQf4SRnOfn6V59RRQB3OqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKY AznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxXDUUAdxqmo2dl4E0+100q huFlFxALyGYjcysplUJlm2rwwClMAZzxS6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eG AUpgDOeK4aigD07VtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk0atqljJFrJW/ge xl05UtlF0jW7PtjwI7QfPE2QcEk7SMmvMaKSQHrWtanoX2bWnh1GK4sruwP2WH7QvlW5Cqsc aW+chsjcWKjbgdOaxbuW7Xw7cwS63ptzcT2uZla9iMECJjEMMKEgynaPmCgDHynJzXn9FCVt APQLuW7Xw7cwS63ptzcT2uZla9iMECJjEMMKEgynaPmCgDHynJzU1/8A2VbeF/EGnaTdWX2J 1tpLMm9BluMbWkZkZ/lbjoFUnpg4Fec0UAdpeaLoQ8MXLw3mnvqEMUMkUkNwE87j94Nryljw emyM5HA5xWtc6tay2erteXFt9ml0zy4IrfUFktvM2oEEdsVDxkEZ5+7g/WvNaKYHaXmi6EPD Fy8N5p76hDFDJFJDcBPO4/eDa8pY8HpsjORwOcUJ4ov5PDmp6jfas95e3rGxFm8oCxxsmWl8 sd+NoIAAJJOc4ri6KAPS/EOoWF2NXa01TyNNezX7LGLyKWF8BNsa2u3dGePvcFSM8Vf1rU9C FrrLQ6jHcWN1YEWsP2lfKtyFVY40gzuDZG4sVG3A6fNXktFKwHoeqail5Zak0upW9vA9ioii tr2Oa1dgEwkdsyh48468FCCaoald2Gq+CNPWEaXayWYnLxNNIHiYyKVWNSxZtwPUhgOeVxXF 161rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjbQDmbzRdCHhi5eG8099Q hihkikhuAnncfvBteUseD02RnI4HOKbq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOv K4rqNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A /ZYftC+VbkKqxxpb5yGyNxYqNuB05pJ7AY3iRtMvrJJp722vdYg0mBCsl2CgYFvMYOrYklB/ hJGc5+fpXN2v/Im33/IF/wBeP9d/x+9U/wBX/s//AGVd7rWp6F9m1p4dRiuLK7sD9lh+0L5V uQqrHGlvnIbI3Fio24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNCe wHktFeta1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFl d2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzTTA8lor1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVW ONLfOQ2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaEwPJaK 9a1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6F9m1p4dRiuLK7sD9l h+0L5VuQqrHGlvnIbI3Fio24HTmhMDyWivWta1PQ/sustDqUdzZXVgRaw/aF8qAhVWONLcHI Ykbi20bcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmi4HktFeta1qe hfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rU9C+za08OoxXFld2B+yw/aF8q 3IVVjjS3zkNkbixUbcDpzQmB5LRXrWtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWK jbgdOa8loTugCiu51TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBb C10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKYHDUV3OqajZWXgWwtdMKp9oEong+2QzMu5 lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxQBw1Fd zqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yG Zl3MrKZVCZZtq8MApTAGc8UAcNRXc6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpg DOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFAHDUV6bq2qWMkOs7dQgey k05Utx9qRoGfbHgR2g+eJsg4JJ2kZNYcxa4+G8dq9xpwuI70XAijngRzCIcZIUgls8YOXNID jqK7GYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmtD+0CNI1RdQ1W2uFk08pCUu43t2b5 Nix2oVWjYDAyQNpBJFMDz6iu0vNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4oT xRfP4c1PUb3VWu729Y2IsnkCpHGyZaURj6bQQAASSc5xQBxdFeg/2gRpGqLqGq21wsmnlISl 3G9uzfJsWO1Cq0bAYGSBtIJIqXUdbn1ODV5dV1KxSGSzLwSabqMv72QhdqeS0hIBBIYGMd84 60rgec0UV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmi+oHktFeta1qe hfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05o1rVNDNrrLw6jHcWV3YH7LB9oXyr chVWONLcHIbI3Fto24HTmhMDyWivWta1TQza6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxba NuB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRcDyWivWta1TQza6y8 Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05o1rU9C+za08OoxXFld2B+yw/aF8q3IVVjj S3zkNkbixUbcDpzRcDyWivWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0 a1qmhm11l4dRjuLK7sD9lg+0L5VuQqrHGluDkNkbi20bcDpzQmB5LRXrWtapoZtdZeHUY7iy u7A/ZYPtC+VbkKqxxpbg5DZG4ttG3A6c0ms6nof2TWXi1GO4srvTz9lh+0L5duQqrHGluDkN kbi20bcDpzQmB5NRXrOs6nof2TWXi1GO4srvTz9lh+0L5duQqrHGluDkNkbi20bcDpzS61qe h/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmi4HktFeta1qeh/ZdZaHUo7myurAi 1h+0L5UBCqscaW4OQxI3Fto24HTmjWtT0P7LrLQ6lHc2V1YEWsP2hfKgIVVjjS3ByGJG4ttG 3A6c0JgeS0V61rWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOa8loWoBRXc6p qNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKTVNRs7LwJp9rppVDcLKLiAXkMxG 5lZTKoTLNtXhgFKYAznimBw9Fdzqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM5 4o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAcNRXpuq6pZSQawRqED2Umn Kluouka3Z9seBHaD54myDgknaRk0atqljJDrO3UIHspNOVLcfakaBn2x4EdoPnibIOCSdpGT SuB5lRXrOs6nof2TWWi1GK5srvTyLaH7SvlW7BVWONIM5DbhuLbRtwOnNY95NeDw9cwTa3pt zcT226ZGvIjBAiYxDDChwZDtHzBQBjCnJzRcDz6ivQbya8Hh65gm1vTbm4ntt0yNeRGCBExi GGFDgyHaPmCgDGFOTmq+qajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznigDhqK9 BvJrweHrmCbW9Nubie23TI15EYIETGIYYUODIdo+YKAMYU5Oahhv/Do8Baxp9lLJFLshLNPE izXEm/PA3nKjAGB90ZPzE0wOEoru4b/w6PAWsafZSyRS7ISzTxIs1xJvzwN5yowBgfdGT8xN W9Q1qfUrfVpNV1OyWKSzLQS6dqEg82UhdqeSXJAOSGBjHfOOtIDzmiu7hv8Aw6PAWsafZSyR S7ISzTxIs1xJvzwN5yowBgfdGT8xNb2s6nof2TWWi1GK5srvTyLaH7SvlW7BVWONIM5DbhuL bRtwOnNAHk1FFFMAooooAKKKKACiiigAooooAKKKKAOimuYH8GxwabPHb7XH9o28jgS3D5+R wf40H9wD5Tyc/eruNa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1w81zA/g 2ODTZ47fa4/tG3kcCW4fPyOD/Gg/uAfKeTn71dxrWp6ELXWWh1GO4sbqwItYftK+VbkKqxxp BncGyNxYqNuB0+al1A4fULmBvCtlBfTx3epDBtWhcFraD+5KwyGyfup1XuRnbXO10WoXMDeF bKC+nju9SGDatC4LW0H9yVhkNk/dTqvcjO2udpgeta1qehfZtaeHUYriyu7A/ZYftC+VbkKq xxpb5yGyNxYqNuB05o1rVNDNrrLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto24HTmue1TUb Ky8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mV lMqhMs21eGAUpgDOeKlIDoda1TQza6y8Oox3Fld2B+ywfaF8q3IVVjjS3ByGyNxbaNuB05o1 rVNDNrrLw6jHcWV3YH7LB9oXyrchVWONLcHIbI3Fto24HTmue1TUbKy8C2FrphVPtAlE8H2y GZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKE gOg1nU9D+yay8Wox3Fld6efssP2hfLtyFVY40twchsjcW2jbgdOaXWtT0P7LrLQ6lHc2V1YE WsP2hfKgIVVjjS3ByGJG4ttG3A6c1z2qajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFK YAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxTSA6HWtT0P7LrLQ6lHc2 V1YEWsP2hfKgIVVjjS3ByGJG4ttG3A6c0a1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI 3Fto24HTmue1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wq n2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKSQHQ61qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4 OQxI3Fto24HTmjWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNUtW1SxeHWd l/A9jJpyx2y/alaBn2x4EdpnfE2QcEk7SMmsOYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eM HLmhLYDqda1PQ/sustDqUdzZXVgRaw/aF8qAhVWONLcHIYkbi20bcDpzRrWp6H9l1lodSjub K6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOa5aYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHL mr/28ro+qJqGq21wj6eUhKXiSW7N8mxY7UKrRsAAMkDaQSRTsBta1qmhm11l4dRjuLK7sD9l g+0L5VuQqrHGluDkNkbi20bcDpzRrWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio2 4HTmuZvNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4rX8Q6hYXX9sGz1XydMazU WsYvIpYXwE2xrald8Z4+9wVIzxSS2Av61qehC11lodRjuLG6sCLWH7SvlW5CqscaQZ3BsjcW KjbgdPmo1rU9D+y6y0OpR3NldWBFrD9oXyoCFVY40twchiRuLbRtwOnNc9qmo2Vl4FsLXTCq faBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54q5rzaZqNikupXVlNPBoqKLqO+Ek5uwfuFVc7gSTk 7T1J3DsJAedV3OqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAzniuGrudU1GysvA tha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFUAapqNlZeBbC10wqn2gSieD7ZDMy7mVlM qhMs21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgW wtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJ lm2rwwClMAZzxSANU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWw tdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJl m2rwwClMAZzxWpq2qWLw6zsv4HsZNOWO2X7UrQM+2PAjtM74myDgknaRk0AZeqajZWXgWwtd MKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2 rwwClMAZzxWpq2qWLw6zsv4HsZNOWO2X7UrQM+2PAjtM74myDgknaRk1d1rU9CFrrLQ6jHcW N1YEWsP2lfKtyFVY40gzuDZG4sVG3A6fNQmBz2qajZWXgWwtdMKp9oEong+2QzMu5lZTKoTL NtXhgFKYAznitTVtUsZIdZ26hA9lJpypbj7UjQM+2PAjtB88TZBwSTtIyaw5i1x8N47V7jTh cR3ouBFHPAjmEQ4yQpBLZ4wcuaXVrDQohfixj059PS2VrS7OoMLmR8L1jBbkksCpjUdeVxR1 ASYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmiYtcfDeO1e404XEd6LgRRzwI5hEOMkKQ S2eMHLmtTXzpmoWSSandWU09voqKLqO+Es5uwfuYVyGBJOW2nqTu9F1DWp9St9Wk1TUrJIZL MtBJp2oy/vZCF2p5LSEgEEhgYx3zjrQBF/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZ IG0gkiqV5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFMmLXHw3jtXuNOFxHei4 EUc8COYRDjJCkEtnjBy5p6eJ75/Dep6he6s13e3rGwFm8gVY4ygLSiMd+NoIAAJJOc4oAl1T UbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8VpXOr2s1pq73lxa/ZpdM8uCG31BZ LbzNqBBFbFQ8ZBGeR8uD9aztZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjNTfbyu j6omoarbXCPp5SEpeJJbs3ybFjtQqtGwAAyQNpBJFCAl186ZqFkkmp3VlNPb6Kii6jvhLObs H7mFchgSTltp6k7vTzqu7iv/AA4vgPWNPsZZYpNkJZriJFmuJN+eAJDlRgcD7oyfmJrhKEB6 1rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmjWtT0P7LrLQ6lHc2V1YEWsP 2hfKgIVVjjS3ByGJG4ttG3A6c0a1qehC11lodRjuLG6sCLWH7SvlW5CqscaQZ3BsjcWKjbgd Pmo1rU9CFrrLQ6jHcWN1YEWsP2lfKtyFVY40gzuDZG4sVG3A6fNSXQA1rU9C+za08OoxXFld 2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjc W2jbgdOaNa1PQha6y0Oox3FjdWBFrD9pXyrchVWONIM7g2RuLFRtwOnzUa1qehC11lodRjuL G6sCLWH7SvlW5CqscaQZ3BsjcWKjbgdPmoXQA1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3 zkNkbixUbcDpzRrWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOaNa1PQha6y0 Oox3FjdWBFrD9pXyrchVWONIM7g2RuLFRtwOnzVy+rWGhRC/FjHpz6elsrWl2dQYXMj4XrGC 3JJYFTGo68rihdAOo1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWp6H9 l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOayte/su/sUk1K5spprfRUUXUV8JJjdq f9XhXIYEk5O09Sd3pFdy3a+HbmCXW9Nubie1zMrXsRggRMYhhhQkGU7R8wUAY+U5OaEgNrWt T0P7LrLQ6lHc2V1YEWsP2hfKgIVVjjS3ByGJG4ttG3A6c0a1qehfZtaeHUYriyu7A/ZYftC+ VbkKqxxpb5yGyNxYqNuB05o1rU9CFrrLQ6jHcWN1YEWsP2lfKtyFVY40gzuDZG4sVG3A6fNR rWp6ELXWWh1GO4sbqwItYftK+VbkKqxxpBncGyNxYqNuB0+ahdADWtT0P7LrLQ6lHc2V1YEW sP2hfKgIVVjjS3ByGJG4ttG3A6c0a1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto2 4HTmjWtT0IWustDqMdxY3VgRaw/aV8q3IVVjjSDO4NkbixUbcDp81GtanoQtdZaHUY7ixurA i1h+0r5VuQqrHGkGdwbI3Fio24HT5qF0ANa1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZ G4sVG3A6c1i3c92PDtzDLrWm3FxPa7pka8i8iBFxiGGFDjzTtHzBQBjCnJzW1rWp6ELXWWh1 GO4sbqwItYftK+VbkKqxxpBncGyNxYqNuB0+ak1nU9C+yay0Oox3Fld6efssH2lfLtyFVY40 gySGyNxYqNuB05NC2QFTVtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk1Ru57seHb mGXWtNuLie13TI15F5ECLjEMMKHHmnaPmCgDGFOTmtnWdT0P7JrLRajFcWV1p5FrD9pXyrch VWONIM7t2RuLFRtwOnzVjXct2vh25gl1vTbm4ntczK17EYIETGIYYUJBlO0fMFAGPlOTmmtg PP69a1rVND+y6y8WoxXFld2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5q8lr1rWtU0P7LrLxajF cWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmo6gGtapof2XWXi1GK4sruwP2aAXK+VbsFVY40 gzuDZG4sVG3A6fNRrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUa1qehfZ taeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNuB05pNZ1TQzaay8WoRT2V3p5+zQfaV8q3YK qxxpBnIbcNxbaNuB05pLoAus6poZtdZeLUYrizu7A/ZoBcr5VuwVVjjSDO4NuG4sVG3A6fNR rWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUmsanof2PWHi1COeyu9PP2aD 7Spit2CqscaQZ3bsjcWKjbgdOaNZ1TQzaay8WoRT2V3p5+zQfaV8q3YKqxxpBnIbcNxbaNuB 05oS2AXWtU0P7LrLxajFcWV3YH7NALlfKt2CqscaQZ3BsjcWKjbgdPmo1rVND+y6y8WoxXFl d2B+zQC5XyrdgqrHGkGdwbI3Fio24HT5qTV9T0T7FrDR6lHc2d1p+LaH7SvlwMFVY40gByG3 DcW2gLgdOaNY1PRBZ6w0epR3NndaeRbRfaF8qBgqrHGluDkMWG4ttG3A6c0JbAY95NeDw9cw Ta3ptzcT226ZGvIjBAiYxDDChwZDtHzBQBjCnJzVzVdVspYNYYX0DWUunKluguka3L7Y8CO0 HzxNkHBJO0jJq5rGp6ILPWGj1KO5s7rTyLaL7QvlQMFVY40twchiw3Fto24HTmjV9T0T7FrD R6lHc2d1p+LaH7SvlwMFVY40gByG3DcW2gLgdOaaA5eYtcfDeO1e404XEd6LgRRzwI5hEOMk KQS2eMHLmr/9oEaPqi6jqttcLJp5SEpdxvbs3ybFjtVVWjYDjJHykEkVsatqeiCx1do9RS5s 7rTsW0JuV8u3YKqxxpADkPuG4ttG3A6c0us6poZtNZeLUIp7K708/ZoPtK+VbsFVY40gzkNu G4ttG3A6c0Ac/qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54rRudXtprPV3vL i2FvNpmyCG31BZLbzNqBBHbFQ8ZBGefu4P1q/rOqaGbTWXi1CKeyu9PP2aD7SvlW7BVWONIM 5DbhuLbRtwOnNGs6poZtNZeLUIp7K708/ZoPtK+VbsFVY40gzkNuG4ttG3A6c0IDL186ZqFk kmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vSxq2q2MsOslb+BrKXTlS3X7UjQF9seBHaA7 4myDgknaRk1c1nVNDNprLxahFPZXenn7NB9pXyrdgqrHGkGchtw3Fto24HTmjWdU0M2msvFq EU9ld6efs0H2lfKt2CqscaQZyG3DcW2jbgdOaSAXWtU0M2usvDqMdxZXdgfssH2hfKtyFVY4 0twchsjcW2jbgdOa8lruNXsfCltYXTWgWaP7Ov2aaKdPNMhC4LAzZIzkMPKBGT0xmuHppWAK KKKYBRRRQAUUUUAFFFFAHb63etPFcNo2qWVv4eazCw2Esill5GYzFgt5u/Lb8H130y80XQh4 YuXhvNPfUIYoZIpIbgJ53H7wbXlLHg9NkZyOBziuLrsfFF3cXGjaStpqcb2Mel28dxbx3qf6 wdQYt2SR8vbjHtSAppLC/wANXtxPB9oTVfOMJlUOU8oLuCk5Iye3v6GhJYX+Gr24ng+0Jqvn GEyqHKeUF3BSckZPb39DTJrmB/BscGmzx2+1x/aNvI4Etw+fkcH+NB/cA+U8nP3q1tU1Gzsv Amn2umlUNwsouIBeQzEbmVlMqhMs21eGAUpgDOeKYHD1239t3v8AwrfZ/akf2j7V5fkfaE3/ AGXyfK2+XnOM9sZ/i96fqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbK y8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAUPEEOiafHEkFlZzNdWiuz2t+XNrcc blGGYFOOjZJ3HDccWrzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKl1TUbKy8C 2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqh Ms21eGAUpgDOeKQEGrWGhRC/FjHpz6elsrWl2dQYXMj4XrGC3JJYFTGo68rirGs2HhK2srs2 QWeEQL9lninTzS5C4ZwZsnnIYeUpGT0xmk1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8 MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKEBh2v/Im33/IF/14 /wBd/wAfvVP9X/s//ZVhV3GqajZ2XgTT7XTSqG4WUXEAvIZiNzKymVQmWbavDAKUwBnPFa2r apYyRayVv4HsZdOVLZRdI1uz7Y8CO0HzxNkHBJO0jJoA8xorsZi1x8N47V7jThcR3ouBFHPA jmEQ4yQpBLZ4wcuaJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaYHHUV6D/aBGkaouoa rbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkiqV5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWP B6bIzkcDnFAHF0V2i+Kb+Tw7qmo32rNd318xsRZvIFWONky0ojHfjaCAACSTnOKu/wBoEaRq i6hqttcLJp5SEpdxvbs3ybFjtQqtGwGBkgbSCSKQHn1Fejajrc+pwavLqupWKQyWZeCTTdRl /eyELtTyWkJAIJDAxjvnHWub1C5gbwrZQX08d3qQwbVoXBa2g/uSsMhsn7qdV7kZ20IDnaK7 G1u7hPAWs29/qccnmpa/Y4GvUkYKHyQqBiVwMZGB09q46mAUV1SeJLp/D9/JeTtdPcgWcdqz RrBAmAd6wg5DDaApChQeck8VytABRXX31h4fS0vXt/sZ1JbSNjardk28TH75ikz+8cDadhYg EnBfGBa1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAcNRXX31joUuhSz2a2 lnItujoJpxLKzfLlcpMfmPPWJQO+3qMN/DuuRKzSaNqKKoJYtauAAOueKAMyiuq8QQ6Jp8cS QWVnM11aK7Pa35c2txxuUYZgU46NknccNxxoazYeErayuzZBZ4RAv2WeKdPNLkLhnBmyechh 5SkZPTGaVwOForstVsNChF8LFNOewS2VrW8+3sLiR8L1jy3JJYFSigc8rjNUklhf4avbieD7 Qmq+cYTKocp5QXcFJyRk9vf0NMDmqK7S80XQh4YuXhvNPfUIYoZIpIbgJ53H7wbXlLHg9NkZ yOBziqDx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIUHAB6ZJAOaorv9S07wUjarY2br5ttau0 F19t3CR0VGUj+FmcuykDOPL4AzXJv4d1yJWaTRtRRVBLFrVwAB1zxSTuBmUV0rx2zfDtCJoF u1v95hF58zx7SN5iL43bjjIUHAB6ZJvz2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykFuTz0 wwOLortr7TvDca6kITZfYI7IPZXcd4WuJZsJgNHvOMksCPLXHtWhf/2VbeF/EGnaTdWX2J1t pLMm9BluMbWkZkZ/lbjoFUnpg4FK4HnNFd1rNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHl KRk9MZrmH8O65ErNJo2ooqgli1q4AA654ouBmUV2k9n4aljmVBaQmTRxeq8d0xMd1x+5XcxG OD8pBbk89MP1DTvDUY1MQGy/s+OzD2N2l4WuZZsJgNHvOMksCPLXHtRcDiKK7zUdH8JQ3Gr3 1tqNk9kbMtZWiTSGRJGVRGeTkncHyp+6NpPXjF+1+H/+EM+zfZZP7U+1bs7xvx5eN2/y/ubv +Wec980J3A52ivWta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1yl9YeH0t L17f7GdSW0jY2q3ZNvEx++YpM/vHA2nYWIBJwXxgCdwOQor0H+0CNI1RdQ1W2uFk08pCUu43 t2b5Nix2oVWjYDAyQNpBJFUrzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKYHF 0V0N9eW2iWk2kaTOk80g2X2oRniUd4oj/wA8/U/x/TAPc61qehfZtaeHUYriyu7A/ZYftC+V bkKqxxpb5yGyNxYqNuB05pXA8lorr76w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCT gvjA39W1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTRcDzGivRdfOmahZJJqd1ZTT 2+ioouo74Szm7B+5hXIYEk5baepO7086pgFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAB RRRQAUUUUAFFFFABRRRQAUUUUAFdj4ou7i40bSVtNTjexj0u3juLeO9T/WDqDFuySPl7cY9q 46ux8UXdxcaNpK2mpxvYx6Xbx3FvHep/rB1Bi3ZJHy9uMe1AFCa5gfwbHBps8dvtcf2jbyOB LcPn5HB/jQf3APlPJz96tfVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxWRNc wP4Njg02eO32uP7Rt5HAluHz8jg/xoP7gHynk5+9Wvqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWU yqEyzbV4YBSmAM54pAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrL wLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUy qEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXg WwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplU Jlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2F rphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoT LNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLX TCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZt q8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznitXVtUsZItZK38 D2MunKlsouka3Z9seBHaD54myDgknaRk1lapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21e GAUpgDOeK1dW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTR1Awpi1x8N47V7jThc R3ouBFHPAjmEQ4yQpBLZ4wcuaJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaJi1x8N47 V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaYG h/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkiqV5ouhDwxcvDeae+oQxQyRSQ3A TzuP3g2vKWPB6bIzkcDnFXf7QI0jVF1DVba4WTTykJS7je3Zvk2LHahVaNgMDJA2kEkVSvNF 0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4pICXVNRsrLwLYWumFU+0CUTwfbIZmX cysplUJlm2rwwClMAZzxWlc6vazWmrveXFr9ml0zy4IbfUFktvM2oEEVsVDxkEZ5Hy4P1pfE WoWF2dYa01XydNezX7LGLuKWF+E2xra7d0Z4+9wVIzx0rM1TUbKy8C2FrphVPtAlE8H2yGZl 3MrKZVCZZtq8MApTAGc8ULuBc186ZqFkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vTnN QuYG8K2UF9PHd6kMG1aFwWtoP7krDIbJ+6nVe5GdtdHr50zULJJNTurKae30VFF1HfCWc3YP 3MK5DAknLbT1J3enOahcwN4VsoL6eO71IYNq0LgtbQf3JWGQ2T91Oq9yM7aFsBftbu4TwFrN vf6nHJ5qWv2OBr1JGCh8kKgYlcDGRgdPauOrsbW7uE8Bazb3+pxyealr9jga9SRgofJCoGJX AxkYHT2rjqYHVJ4kun8P38l5O109yBZx2rNGsECYB3rCDkMNoCkKFB5yTxXK11UfiS6bw9fv eTtdPcqLOO1d41ggQAHesQOdw2gKQoUHnJPFcrQB199YeH0tL17f7GdSW0jY2q3ZNvEx++Yp M/vHA2nYWIBJwXxgWtU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFVb6w8Ppa Xr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjAtapqNlZeBbC10wqn2gSieD7ZDMy7mVlM qhMs21eGAUpgDOeKSAq31joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qMN/DmuRhi+i 6ioXkk2rjH6VuX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qMN/DuuRKzSaNqKKoJ YtauAAOueKANJ47Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9Mk357Pw1LHMqC0hMmji9V 47piY7rj9yu5iMcH5SC3J56Yq+IIdE0+OJILKzma6tFdntb8ubW443KMMwKcdGyTuOG440NZ sPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjNADNQ07w1GNTEBsv7Pjsw9jdpeFrmWb CYDR7zjJLAjy1x7Vfv8A+yrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8AK3HQKpPTBwKy9WsN CiF+LGPTn09LZWtLs6gwuZHwvWMFuSSwKmNR15XFUklhf4avbieD7Qmq+cYTKocp5QXcFJyR k9vf0NCA1dZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMnpjNcw/hzXIwxfRdRULySbV xj9K6G80XQh4YuXhvNPfUIYoZIpIbgJ53H7wbXlLHg9NkZyOBziqDx2zfDtCJoFu1v8AeYRe fM8e0jeYi+N244yFBwAemSRAX7zRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKv v4f0J9O1G+jsnisY9OMtnLcefHI0hVcGQsPLZix4CNg56NkER6lp3gpG1Wxs3Xzba1doLr7b uEjoqMpH8LM5dlIGceXwBmuTfw7rkSs0mjaiiqCWLWrgADrnihagD+HNcjDF9F1FQvJJtXGP 0rqdR0fwlDcavfW2o2T2Rsy1laJNIZEkZVEZ5OSdwfKn7o2k9eMh47Zvh2hE0C3a3+8wi8+Z 49pG8xF8btxxkKDgA9Mk357Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5SC3J56YAC80XQh4Y uXhvNPfUIYoZIpIbgJ53H7wbXlLHg9NkZyOBziqCSwv8NXtxPB9oTVfOMJlUOU8oLuCk5Iye 3v6GtLUNO8NRjUxAbL+z47MPY3aXha5lmwmA0e84ySwI8tce1X7/APsq28L+INO0m6svsTrb SWZN6DLcY2tIzIz/ACtx0CqT0wcChMCLxI2mX1kk097bXusQaTAhWS7BQMC3mMHVsSSg/wAJ IznPz9KwPtfh/wD4Qz7N9lk/tT7VuzvG/Hl43b/L+5u/5Z5z3zWxrNh4StrK7NkFnhEC/ZZ4 p080uQuGcGbJ5yGHlKRk9MZrmH8Oa5GGL6LqKheSTauMfpQgPR9a1PQ/sustDqUdzZXVgRaw /aF8qAhVWONLcHIYkbi20bcDpzXKXthoEdneNAbQ6mtpGzWouyYImP3zFJn944G07NxAJOC+ MCWez8NSxzKgtITJo4vVeO6YmO64/cruYjHB+UgtyeemH6hp3hqMamIDZf2fHZh7G7S8LXMs 2EwGj3nGSWBHlrj2oWmgGp4h1Cwuv7Ya01XytNazUWqC7ilhkwE2xra7d0Z4+9wVIzx0p2ra pYyQ6zt1CB7KTTlS3H2pGgZ9seBHaD54myDgknaRk1najo/hKG41e+ttRsnsjZlrK0SaQyJI yqIzyck7g+VP3RtJ68Yv2vw//wAIZ9m+yyf2p9q3Z3jfjy8bt/l/c3f8s8575pLYBL68ttEt JtI0mdJ5pBsvtQjPEo7xRH/nn6n+P6YB7nWtT0P7LrLQ6lHc2V1YEWsP2hfKgIVVjjS3ByGJ G4ttG3A6c0a1qehC11lodRjuLG6sCLWH7SvlW5CqscaQZ3BsjcWKjbgdPmrlL6w8PpaXr2/2 M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjAF0YBfWHh9LS9e3+xnUltI2Nqt2TbxMfvmKTP7 xwNp2FiAScF8YEt5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFdNrWqaH9l1l4 tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzVw19eW2iWk2kaTOk80g2X2oRniUd4oj/zz 9T/H9MAtAdBeTXg8PXME2t6bc3E9tumRryIwQImMQwwocGQ7R8wUAYwpyc159XrWtanoX2bW nh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOa5S+sPD6Wl69v9jOpLaRsbVbsm3iY/fMU mf3jgbTsLEAk4L4wBPQDkKK9O1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2kZNVtf OmahZJJqd1ZTT2+ioouo74Szm7B+5hXIYEk5baepO70EwPOqKKKYBRRRQAUUUUAFFFFABRRR QAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXY+KLu4uNG0lbTU43sY9Lt47i3j vU/1g6gxbskj5e3GPauOrsfFF3cXGjaStpqcb2Mel28dxbx3qf6wdQYt2SR8vbjHtQBQmuYH 8GxwabPHb7XH9o28jgS3D5+Rwf40H9wD5Tyc/erX1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZV CZZtq8MApTAGc8VkTXMD+DY4NNnjt9rj+0beRwJbh8/I4P8AGg/uAfKeTn71a+qajZWXgWwt dMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznikAapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhM s21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtd MKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2 rwwClMAZzxQAapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6Y VT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtX hgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxQAapqNlZeBbC10wq n2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavD AKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0 CUTwfbIZmXcysplUJlm2rwwClMAZzxQAapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGA UpgDOeK1dW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTWVqmo2Vl4FsLXTCqfaBK J4PtkMzLuZWUyqEyzbV4YBSmAM54rV1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2k ZNHUDCmLXHw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5omLXHw3jtXuNOFxHei4EUc8COYRD jJCkEtnjBy5omLXHw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5omLXHw3jtXuNOFxHei4EUc 8COYRDjJCkEtnjBy5pgaH9oEaRqi6hqttcLJp5SEpdxvbs3ybFjtQqtGwGBkgbSCSKk186Zq Fkkmp3VlNPb6Kii6jvhLObsH7mFchgSTltp6k7vSP+0CNI1RdQ1W2uFk08pCUu43t2b5Nix2 oVWjYDAyQNpBJFUrzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOKSA6bWtT0L7N rTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNY15NeDw9cwTa3ptzcT226ZGvIjBAiYx DDChwZDtHzBQBjCnJzVvxFqFhdnWGtNV8nTXs1+yxi7ilhfhNsa2u3dGePvcFSM8dKzNU1Gy svAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFKOyAu6jrc+pwavLqupWKQyWZeCTTdR l/eyELtTyWkJAIJDAxjvnHWub1C5gbwrZQX08d3qQwbVoXBa2g/uSsMhsn7qdV7kZ210evnT NQskk1O6spp7fRUUXUd8JZzdg/cwrkMCScttPUnd6c5qFzA3hWygvp47vUhg2rQuC1tB/clY ZDZP3U6r3IztprYC/a3dwngLWbe/1OOTzUtfscDXqSMFD5IVAxK4GMjA6e1cdXY2t3cJ4C1m 3v8AU45PNS1+xwNepIwUPkhUDErgYyMDp7Vx1MDqk8SXT+H7+S8na6e5As47VmjWCBMA71hB yGG0BSFCg85J4rla6pPEl0/h+/kvJ2unuQLOO1Zo1ggTAO9YQchhtAUhQoPOSeK5WgDr76w8 PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjAtapqNlZeBbC10wqn2gSieD7ZDMy7m VlMqhMs21eGAUpgDOeKq31h4fS0vXt/sZ1JbSNjardk28TH75ikz+8cDadhYgEnBfGBa1TUb Ky8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UkBVvrHQpdClns1tLORbdHQTTiWVm+ XK5SY/MeesSgd9vUYbeG9dXO7RdRGOubV+P0rcvrHQpdClns1tLORbdHQTTiWVm+XK5SY/Me esSgd9vUYb+HdciVmk0bUUVQSxa1cAAdc8UAaTx2zfDtCJoFu1v95hF58zx7SN5iL43bjjIU HAB6ZJvz2fhqWOZUFpCZNHF6rx3TEx3XH7ldzEY4PykFuTz0xV8QQ6Jp8cSQWVnM11aK7Pa3 5c2txxuUYZgU46NknccNxxoazYeErayuzZBZ4RAv2WeKdPNLkLhnBmyechh5SkZPTGaAGahp 3hqMamIDZf2fHZh7G7S8LXMs2EwGj3nGSWBHlrj2q/f/ANlW3hfxBp2k3Vl9idbaSzJvQZbj G1pGZGf5W46BVJ6YOBWXq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOvK4qkksL/DV7 cTwfaE1XzjCZVDlPKC7gpOSMnt7+hoQGrrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ5yGHlKR k9MZrmH8Oa5GGL6LqKheSTauMfpXQ3mi6EPDFy8N5p76hDFDJFJDcBPO4/eDa8pY8HpsjORw OcVQeO2b4doRNAt2t/vMIvPmePaRvMRfG7ccZCg4APTJIgL95ouhDwxcvDeae+oQxQyRSQ3A TzuP3g2vKWPB6bIzkcDnFX38P6E+najfR2TxWMenGWzluPPjkaQquDIWHlsxY8BGwc9GyCIt R07wWjarZWjr51taM0F19t3CR0VGUj+FmcuykAnHl8AZrlH8O65ErNJo2ooqgli1q4AA654o QA/hzXIwxfRdRULySbVxj9K0njtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0ySPHbN8O0 ImgW7W/3mEXnzPHtI3mIvjduOMhQcAHpkm/PZ+GpY5lQWkJk0cXqvHdMTHdcfuV3MRjg/KQW 5PPTDANSu7DVfBGnrCNLtZLMTl4mmkDxMZFKrGpYs24HqQwHPK4qbWbDwlbWV2bILPCIF+yz xTp5pchcM4M2TzkMPKUjJ6YzTNQ07w1GNTEBsv7Pjsw9jdpeFrmWbCYDR7zjJLAjy1x7Vfv/ AOyrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk9MHApAYfiCHRNPjiSCys5murRXZ7W/ Lm1uONyjDMCnHRsk7jhuOLV5ouhDwxcvDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFTazY eErayuzZBZ4RAv2WeKdPNLkLhnBmyechh5SkZPTGa5h/DuuRKzSaNqKKoJYtauAAOueKEB0O pXdhqvgjT1hGl2slmJy8TTSB4mMilVjUsWbcD1IYDnlcU/UNO8NRjUxAbL+z47MPY3aXha5l mwmA0e84ySwI8tce1Mns/DUscyoLSEyaOL1XjumJjuuP3K7mIxwflILcnnph+oad4ajGpiA2 X9nx2Yexu0vC1zLNhMBo95xklgR5a49qAKXiCHRNPjiSCys5murRXZ7W/Lm1uONyjDMCnHRs k7jhuOK/2vw//wAIZ9m+yyf2p9q3Z3jfjy8bt/l/c3f8s8575ra1HR/CUNxq99bajZPZGzLW Vok0hkSRlURnk5J3B8qfujaT14xftfh//hDPs32WT+1PtW7O8b8eXjdv8v7m7/lnnPfNC2A7 jWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNcpfWHh9LS9e3+xnUltI2Nqt 2TbxMfvmKTP7xwNp2FiAScF8YHV61qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5yGyNxYqNu B05rlL6w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjAUdkBr6jrc+pwavLqupWK QyWZeCTTdRl/eyELtTyWkJAIJDAxjvnHWtTWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ 2RuLFRtwOnNGtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOa4a+vLbRLSbSN JnSeaQbL7UIzxKO8UR/55+p/j+mARdAC+vLbRLSbSNJnSeaQbL7UIzxKO8UR/wCefqf4/pgH uda1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0a1qehfZtaeHUYriyu7A/ZY ftC+VbkKqxxpb5yGyNxYqNuB05rlL6w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTg vjAFrZgF9YeH0tL17f7GdSW0jY2q3ZNvEx++YpM/vHA2nYWIBJwXxgSr4pv5PDuqajfas13f XzGxFm8gVY42TLSiMd+NoIAAJJOc4ra1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2 kZNVtebTNRsUl1K6spp4NFRRdR3wknN2D9wqrncCScnaepO4dmgKms2HhK2srs2QWeEQL9ln inTzS5C4ZwZsnnIYeUpGT0xmuFoopoAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKK ACiiigAooooAKKKKACiiigArsfFF3cXGjaStpqcb2Mel28dxbx3qf6wdQYt2SR8vbjHtXHV2 Pii7uLjRtJW01ON7GPS7eO4t471P9YOoMW7JI+Xtxj2oAoTXMD+DY4NNnjt9rj+0beRwJbh8 /I4P8aD+4B8p5OfvVr6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKyJrmB/B scGmzx2+1x/aNvI4Etw+fkcH+NB/cA+U8nP3q19U1GysvAtha6YVT7QJRPB9shmZdzKymVQm WbavDAKUwBnPFIA1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC 10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmW bavDAKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznigA1TUbKy8C2Fr phVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs2 1eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtdMK p9oEong+2QzMu5lZTKoTLNtXhgFKYAznigA1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq 8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6YVT 7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhg FKYAznigA1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vq6tqljJFrJW/gexl 05UtlF0jW7PtjwI7QfPE2QcEk7SMmsrVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwCl MAZzxWrq2qWMkWslb+B7GXTlS2UXSNbs+2PAjtB88TZBwSTtIyaOoGFMWuPhvHavcacLiO9F wIo54EcwiHGSFIJbPGDlzRMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzRMWuPhvHavca cLiO9FwIo54EcwiHGSFIJbPGDlzRMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzTAv/by uj6omoarbXCPp5SEpeJJbs3ybFjtQqtGwAAyQNpBJFS6+dM1CySTU7qymnt9FRRdR3wlnN2D 9zCuQwJJy209Sd3pH/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkiqV5ouhDwxc vDeae+oQxQyRSQ3ATzuP3g2vKWPB6bIzkcDnFIDpta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY 40t85DZG4sVG3A6c1T1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2kZNR3Or20tnq7 XlzbG3l0zy4IrfUFktjJtQII7YqHjIIzyPlwfrVa8mvB4euYJtb025uJ7bdMjXkRggRMYhhh Q4Mh2j5goAxhTk5pJbAV9U1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFZGoXM DeFbKC+nju9SGDatC4LW0H9yVhkNk/dTqvcjO2r8xa4+G8dq9xpwuI70XAijngRzCIcZIUgl s8YOXNUNQuYG8K2UF9PHd6kMG1aFwWtoP7krDIbJ+6nVe5GdtUBftbu4TwFrNvf6nHJ5qWv2 OBr1JGCh8kKgYlcDGRgdPauOrsbW7uE8Bazb3+pxyealr9jga9SRgofJCoGJXAxkYHT2rjqA OqTxJdP4fv5Lydrp7kCzjtWaNYIEwDvWEHIYbQFIUKDzkniuVrqk8SXT+H7+S8na6e5As47V mjWCBMA71hByGG0BSFCg85J4rlaAOvvrDw+lpevb/YzqS2kbG1W7Jt4mP3zFJn944G07CxAJ OC+MC1qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54qrfWHh9LS9e3+xnUltI2 Nqt2TbxMfvmKTP7xwNp2FiAScF8YFrVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClM AZzxSQFW+sdCl0KWezW0s5Ft0dBNOJZWb5crlJj8x56xKB329Rhv4d1yJWaTRtRRVBLFrVwA B1zxW5fWOhS6FLPZraWci26OgmnEsrN8uVykx+Y89YlA77eoL6x0KXQpZ7NbSzkW3R0E04ll ZvlyuUmPzHnrEoHfb1AAX1joUuhSz2a2lnItujoJpxLKzfLlcpMfmPPWJQO+3qL83h7RJG1j VYraeHS205p7GOaOZBFJtXbudsBmLH5QC4OTzwMk3h7RJG1jVYraeHS205p7GOaOZBFJtXbu dsBmLH5QC4OTzwM1Z7Pw1LHMqC0hMmji9V47piY7rj9yu5iMcH5SC3J56YEAT2fhqWOZUFpC ZNHF6rx3TEx3XH7ldzEY4PykFuTz0w3VrDQohfixj059PS2VrS7OoMLmR8L1jBbkksCpjUde VxRq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOvK4rUv/7KtvC/iDTtJurL7E620lmT egy3GNrSMyM/ytx0CqT0wcCgDjH8O65ErNJo2ooqgli1q4AA654rqdR0fwlDcavfW2o2T2Rs y1laJNIZEkZVEZ5OSdwfKn7o2k9eMX7X4f8A+EM+zfZZP7U+1bs7xvx5eN2/y/ubv+Wec981 3GtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOaOoHOahp3hqMamIDZf2fHZh 7G7S8LXMs2EwGj3nGSWBHlrj2ql4gh0TT44kgsrOZrq0V2e1vy5tbjjcowzApx0bJO44bjiW +sPD6Wl69v8AYzqS2kbG1W7Jt4mP3zFJn944G07CxAJOC+MB0xa4+G8dq9xpwuI70XAijngR zCIcZIUgls8YOXNCA0PEjaZfWSTT3tte6xBpMCFZLsFAwLeYwdWxJKD/AAkjOc/P0qDWbDwl bWV2bILPCIF+yzxTp5pchcM4M2TzkMPKUjJ6YzU95NeDw9cwTa3ptzcT226ZGvIjBAiYxDDC hwZDtHzBQBjCnJzR/aBGkaouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkihAZHiCHRNPj iSCys5murRXZ7W/Lm1uONyjDMCnHRsk7jhuOK/2vw/8A8IZ9m+yyf2p9q3Z3jfjy8bt/l/c3 f8s8575pL68ttEtJtI0mdJ5pBsvtQjPEo7xRH/nn6n+P6YB7nWtT0P7LrLRajFc2V3YH7NCL hfKt2CqscaW+dwbI3ElRtwOnzUXANa1PQ/sustFqMVzZXdgfs0IuF8q3YKqxxpb53BsjcSVG 3A6fNXKX1h4fS0vXt/sZ1JbSNjardk28TH75ikz+8cDadhYgEnBfGAXthoEdneNAbQ6mtpGz WouyYImP3zFJn944G07NxAJOC+MB0xa4+G8dq9xpwuI70XAijngRzCIcZIUgls8YOXNCQGhe TXg8PXME2t6bc3E9tumRryIwQImMQwwocGQ7R8wUAYwpyc1a8Q39hdDVzaar5OmtZqLWMXcU sL4CbY1tSN8Z4+9wVIzxTtW1Sxkh1nbqED2UmnKluPtSNAz7Y8CO0HzxNkHBJO0jJq7rWp6H 9l1lotRiubK7sD9mhFwvlW7BVWONLfO4NkbiSo24HT5qSA4a+vLbRLSbSNJnSeaQbL7UIzxK O8UR/wCefqf4/pgHuda1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c15LRTsB 199YeH0tL17f7GdSW0jY2q3ZNvEx++YpM/vHA2nYWIBJwXxga+oa1PqVvq0mqalZJDJZloJN O1GX97IQu1PJaQkAgkMDGO+cda85oosB6Bdy3a+HbmCXW9Nubie1zMrXsRggRMYhhhQkGU7R 8wUAY+U5Oa8/oopgFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRR RQAUUUUAFFFFABRRRQAUUUUAFdj4ou7i40bSVtNTjexj0u3juLeO9T/WDqDFuySPl7cY9q46 ux8UXdxcaNpK2mpxvYx6Xbx3FvHep/rB1Bi3ZJHy9uMe1AFCa5gfwbHBps8dvtcf2jbyOBLc Pn5HB/jQf3APlPJz96tfVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxWRNcwP 4Njg02eO32uP7Rt5HAluHz8jg/xoP7gHynk5+9Wvqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyq EyzbV4YBSmAM54pAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwL YWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqE yzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXgWw tdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJl m2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2Frp hVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLN tXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTC qfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8 MApTAGc8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznitXVtUsZItZK38D2 MunKlsouka3Z9seBHaD54myDgknaRk1lapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGA UpgDOeK1NW1Sykh1krf27WUunKluouka3L7Y8CO0+/Ecg4JJ2kZNHUDDmLXHw3jtXuNOFxHe i4EUc8COYRDjJCkEtnjBy5omLXHw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5omLXHw3jtXu NOFxHei4EUc8COYRDjJCkEtnjBy5rqda1TQ/susvFqMVxZXdgfs0AuV8q3YKqxxpBncGyNxY qNuB0+agDG/tAjSNUXUNVtrhZNPKQlLuN7dm+TYsdqFVo2AwMkDaQSRVK80XQh4YuXhvNPfU IYoZIpIbgJ53H7wbXlLHg9NkZyOBzirv9oEaRqi6hqttcLJp5SEpdxvbs3ybFjtQqtGwGBkg bSCSKr6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeKEBEvim/k8O6pqN9qzXd 9fMbEWbyBVjjZMtKIx342ggAAkk5zipdU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAK UwBnPFXdR1ufU4NXl1XUrFIZLMvBJpuoy/vZCF2p5LSEgEEhgYx3zjrWXqthoUQvhYx6c9gl srWt3/aDfaHfC9Y8tkklgVMaDryuKEBY1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIy emM1j6hcwN4VsoL6eO71IYNq0LgtbQf3JWGQ2T91Oq9yM7a6O+OlW3hbxBp2lXNl9idLaS0J vgZbjG1pGZGf5W46BVJ6YOAK5zULmBvCtlBfTx3epDBtWhcFraD+5KwyGyfup1XuRnbQtgL9 rd3CeAtZt7/U45PNS1+xwNepIwUPkhUDErgYyMDp7Vx1dja3dwngLWbe/wBTjk81LX7HA16k jBQ+SFQMSuBjIwOntXHUwOqTxJdP4fv5Lydrp7kCzjtWaNYIEwDvWEHIYbQFIUKDzkniuVrq k8SXT+H7+S8na6e5As47VmjWCBMA71hByGG0BSFCg85J4rlaAOvvrDw+lpevb/YzqS2kbG1W 7Jt4mP3zFJn944G07CxAJOC+MC1qmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM5 4qx4kbTL6ySae9tr3WINJgQrJdgoGBbzGDq2JJQf4SRnOfn6VS1G6sNV8D6ckP8AZdtJZrOZ ImmkDxMZFKrGpYli4PUhgOeVxSQEV9Y6FLoUs9mtpZyLbo6CacSys3y5XKTH5jz1iUDvt6gv rHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vUamty6VqoWHUbm0lu7fQFcXi3m9zcof9 XuDFGJOexJzwelY3iCHRNPjiSCys5murRXZ7W/Lm1uONyjDMCnHRsk7jhuOADUl8O6K51fVI raeHS204zWKTRyxiOTau3c7YDMWPAUuDk+gzVmsvDUsUyILSJpNGF6rpdMTHdcfuVyxGDg/K QW5PPTEusWHhO2sbtrMLPELdfs08U6GUyELhmBmyRnIYeUpGT0xmorzRdCHhi5eG8099Qhih kikhuAnncfvBteUseD02RnI4HOKEA3VrDQohfixj059PS2VrS7OoMLmR8L1jBbkksCpjUdeV xWpf/wBlW3hfxBp2k3Vl9idbaSzJvQZbjG1pGZGf5W46BVJ6YOBSXx0q28LeINO0q5svsTpb SWhN8DLcY2tIzIz/ACtx0CqT0wcAVzn2vw//AMIZ9m+yyf2p9q3Z3jfjy8bt/l/c3f8ALPOe +aEAfa/D/wDwhn2b7LJ/an2rdneN+PLxu3+X9zd/yzznvmu41rU9C+za08OoxXFld2B+yw/a F8q3IVVjjS3zkNkbixUbcDpzRrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOn zVyl7Y+H1s7yS3+yf2kLSNjaC7JgiY/fMUmf3jgbTsLEAk4L4wDfUAvbDQI7O8aA2h1NbSNm tRdkwRMfvmKTP7xwNp2biAScF8YHV61qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto 24HTmuZXxTfyeHdU1G+1Zru+vmNiLN5AqxxsmWlEY78bQQAASSc5xWvc6vazWmrveXFr9ml0 zy4IbfUFktvM2oEEVsVDxkEZ5Hy4P1oA5S9vbbRbOXSdJmWaaVdl9fp0kHeKI9o/U/x/TAPd a1qeh/ZdZaHUo7myurAi1h+0L5UBCqscaW4OQxI3Fto24HTmuGvry20S0m0jSZ0nmkGy+1CM 8SjvFEf+efqf4/pgHuda1TQ/susvFqMVxZXdgfs0AuV8q3YKqxxpBncGyNxYqNuB0+ajqByl 7YaBHZ3jQG0OpraRs1qLsmCJj98xSZ/eOBtOzcQCTgvjAdMWuPhvHavcacLiO9FwIo54Ecwi HGSFIJbPGDlzTb6w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjA3tV1Sykg1gjU IHspNOVLdRdI1uz7Y8CO0HzxNkHBJO0jJoANW1Sxkh1nbqED2UmnKluPtSNAz7Y8CO0HzxNk HBJO0jJq7rWp6H9l1lodSjubK6sCLWH7QvlQEKqxxpbg5DEjcW2jbgdOa5aYtcfDeO1e404X Ed6LgRRzwI5hEOMkKQS2eMHLmuOosAUUUUwCiiigAooooAKKKKACiiigAooooAKKKKACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACux8UXdxcaNpK 2mpxvYx6Xbx3FvHep/rB1Bi3ZJHy9uMe1cdXY+KLu4uNG0lbTU43sY9Lt47i3jvU/wBYOoMW 7JI+Xtxj2oAoTXMD+DY4NNnjt9rj+0beRwJbh8/I4P8AGg/uAfKeTn71a2qajZ2XgTT7XTSq G4WUXEAvIZiNzKymVQmWbavDAKUwBnPFZM1zA/g2ODTZ47fa4/tG3kcCW4fPyOD/ABoP7gHy nk5+9Wvqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54oANU1GysvAtha6YVT7Q JRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFK YAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ 4PtkMzLuZWUyqEyzbV4YBSmAM54pAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKY AznijVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4 PtkMzLuZWUyqEyzbV4YBSmAM54o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc 8UAGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwf bIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54 o1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8UAGqajZWXgWwtdMKp9oEong+2 QzMu5lZTKoTLNtXhgFKYAznitXVtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk1la pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK1NW1Sykh1krf27WUunKluouka3 L7Y8CO0+/Ecg4JJ2kZNHUDDmLXHw3jtXuNOFxHei4EUc8COYRDjJCkEtnjBy5rqda1PQvs2t PDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1y0xa4+G8dq9xpwuI70XAijngRzCIcZIU gls8YOXNdTrWqaH9l1l4tRiuLK7sD9mgFyvlW7BVWONIM7g2RuLFRtwOnzUdQDWtT0L7NrTw 6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNZWvnTNQskk1O6spp7fRUUXUd8JZzdg/cwr kMCScttPUnd6R/2gRpGqLqGq21wsmnlISl3G9uzfJsWO1Cq0bAYGSBtIJIqvqmo2Vl4FsLXT CqfaBKJ4PtkMzLuZWUyqEyzbV4YBSmAM54pJAQatYaFEL8WMenPp6WytaXZ1BhcyPhesYLck lgVMajryuK1L/wDsq28L+INO0m6svsTrbSWZN6DLcY2tIzIz/K3HQKpPTBwKNR1ufU4NXl1X UrFIZLMvBJpuoy/vZCF2p5LSEgEEhgYx3zjrRqOtz6nBq8uq6lYpDJZl4JNN1GX97IQu1PJa QkAgkMDGO+cdaFfQDNvNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4rN1C5gbwr ZQX08d3qQwbVoXBa2g/uSsMhsn7qdV7kZ21sw3/h0eAtY0+ylkil2QlmniRZriTfngbzlRgD A+6Mn5iaxtQuYG8K2UF9PHd6kMG1aFwWtoP7krDIbJ+6nVe5GdtUBftbu4TwFrNvf6nHJ5qW v2OBr1JGCh8kKgYlcDGRgdPauOrsbW7uE8Bazb3+pxyealr9jga9SRgofJCoGJXAxkYHT2rj qAOqTxJdP4fv5Lydrp7kCzjtWaNYIEwDvWEHIYbQFIUKDzkniuVrqk8SXT+H7+S8na6e5As4 7VmjWCBMA71hByGG0BSFCg85J4rlaAOy1aw0KIX4sY9OfT0tla0uzqDC5kfC9YwW5JLAqY1H XlcVo63LpWqhYdRubSW7t9AVxeLeb3Nyh/1e4MUYk57EnPB6U3xI2mX1kk097bXusQaTAhWS 7BQMC3mMHVsSSg/wkjOc/P0qlqV3Yar4I09YRpdrJZicvE00geJjIpVY1LFm3A9SGA55XFJA UHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0ySJLC/w1e3E8H2hNV84wmVQ5Tygu4KTk jJ7e/oa3tbl0rVQsOo3NpLd2+gK4vFvN7m5Q/wCr3BijEnPYk54PSsbxBDomnxxJBZWczXVo rs9rflza3HG5RhmBTjo2Sdxw3HABuX/9lW3hfxBp2k3Vl9idbaSzJvQZbjG1pGZGf5W46BVJ 6YOBXGP4d1yJWaTRtRRVBLFrVwAB1zxXTaxYeE7axu2sws8Qt1+zTxToZTIQuGYGbJGchh5S kZPTGaivNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4oQFXX4dF06OKOCytJmur RWZ7W/3m2uONyjDMCnHQgk7jhuOInjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0yT0F8 dKtvC3iDTtKubL7E6W0loTfAy3GNrSMyM/ytx0CqT0wcAVzn2vw//wAIZ9m+yyf2p9q3Z3jf jy8bt/l/c3f8s8575oQGzruiaLpGnK97Z/ZJ7vT1mhiUT+bHc/LlBuymwfxBiXGT/s1jfa/D /wDwhn2b7LJ/an2rdneN+PLxu3+X9zd/yzznvmu41rVND+y6y8WoxXFld2B+zQC5XyrdgqrH GkGdwbI3Fio24HT5q5S+sPD6Wl69v9jOpLaRsbVbsm3iY/fMUmf3jgbTsLEAk4L4wBMDq9a1 PQ/sustDqUdzZXVgRaw/aF8qAhVWONLcHIYkbi20bcDpzXKXthoEdneNAbQ6mtpGzWouyYIm P3zFJn944G07NxAJOC+MDq9a1TQ/susvFqMVxZXdgfs0AuV8q3YKqxxpBncGyNxYqNuB0+au Gvry20S0m0jSZ0nmkGy+1CM8SjvFEf8Ann6n+P6YBS2QGneaLoQ8MXLw3mnvqEMUMkUkNwE8 7j94NryljwemyM5HA5xWZfXltolpNpGkzpPNINl9qEZ4lHeKI/8APP1P8f0wD3Otapof2XWX i1GK4sruwP2aAXK+VbsFVY40gzuDZG4sVG3A6fNXKX1h4fS0vXt/sZ1JbSNjardk28TH75ik z+8cDadhYgEnBfGA0B1etanof2XWWh1KO5srqwItYftC+VAQqrHGluDkMSNxbaNuB05rlL2w 0COzvGgNodTW0jZrUXZMETH75ikz+8cDadm4gEnBfGBvatqllJDrJW/t2spdOVLdRdI1uX2x 4Edp9+I5BwSTtIyaw5i1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaFsBoXk14PD1zBNre m3NxPbbpka8iMECJjEMMKHBkO0fMFAGMKcnNbOtanof2XWWh1KO5srqwItYftC+VAQqrHGlu DkMSNxbaNuB05ryWiiwBRRRTAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo AKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAK7HxRd3Fxo2kraanG9jHpdv HcW8d6n+sHUGLdkkfL24x7VNrd608Vw2japZW/h5rMLDYSyKWXkZjMWC3m78tvwfXfRqGneG oxqYgNl/Z8dmHsbtLwtcyzYTAaPecZJYEeWuPalcDKmuYH8GxwabPHb7XH9o28jgS3D5+Rwf 40H9wD5Tyc/erX1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZtq8MApTAGc8Vdv/AOyrbwv4 g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk9MHAqnrNh4StrK7NkFnhEC/ZZ4p080uQuGcGbJ 5yGHlKRk9MZouAmqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYW umFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxWHa/8ibff8gX/Xj/AF3/AB+9U/1f+z/9lWFT A7nVNRsrLwLYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4Pt kMzLuZWUyqEyzbV4YBSmAM54p0V/4cXwHrGn2MssUmyEs1xEizXEm/PAEhyowOB90ZPzE1p6 tqljJFrJW/gexl05UtlF0jW7PtjwI7QfPE2QcEk7SMmkBlapqNlZeBbC10wqn2gSieD7ZDMy 7mVlMqhMs21eGAUpgDOeKNU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFcNXZ atYaFEL8WMenPp6WytaXZ1BhcyPhesYLcklgVMajryuKAJ9U1GysvAtha6YVT7QJRPB9shmZ dzKymVQmWbavDAKUwBnPFaurapYyRayVv4HsZdOVLZRdI1uz7Y8CO0HzxNkHBJO0jJrFXxTf yeHdU1G+1Zru+vmNiLN5AqxxsmWlEY78bQQAASSc5xXF0WA7GYtcfDeO1e404XEd6LgRRzwI 5hEOMkKQS2eMHLmup1rU9C+za08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzXL6tYaF EL8WMenPp6WytaXZ1BhcyPhesYLcklgVMajryuKcvim/k8O6pqN9qzXd9fMbEWbyBVjjZMtK Ix342ggAAkk5zijfUDpta1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c1la+d M1CySTU7qymnt9FRRdR3wlnN2D9zCuQwJJy209Sd3p51XdxX/hxfAesafYyyxSbISzXESLNc Sb88ASHKjA4H3Rk/MTQlYCtq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOvK4rUv/7K tvC/iDTtJurL7E620lmTegy3GNrSMyM/ytx0CqT0wcCsqYtcfDeO1e404XEd6LgRRzwI5hEO MkKQS2eMHLmuOosB61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fio24HTmuH1C5g bwrZQX08d3qQwbVoXBa2g/uSsMhsn7qdV7kZ21v/ANoEaRqi6hqttcLJp5SEpdxvbs3ybFjt QqtGwGBkgbSCSK2da1PQvs2tPDqMVxZXdgfssP2hfKtyFVY40t85DZG4sVG3A6c0lpZActa3 dwngLWbe/wBTjk81LX7HA16kjBQ+SFQMSuBjIwOntXHUV6DeTXg8PXME2t6bc3E9tumRryIw QImMQwwocGQ7R8wUAYwpyc1QGQniS6fw/fyXk7XT3IFnHas0awQJgHesIOQw2gKQoUHnJPFc rXc6pqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK4agDstWsNCiF+LGPTn09LZ WtLs6gwuZHwvWMFuSSwKmNR15XFaOty6VqoWHUbm0lu7fQFcXi3m9zcof9XuDFGJOexJzwel W9W1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTWFMWuPhvHavcacLiO9FwIo54Ecw iHGSFIJbPGDlzSWwFN47Zvh2hE0C3a3+8wi8+Z49pG8xF8btxxkKDgA9MkiSwv8ADV7cTwfa E1XzjCZVDlPKC7gpOSMnt7+hrmq7rWbDwlbWV2bILPCIF+yzxTp5pchcM4M2TzkMPKUjJ6Yz QBcv/wCyrbwv4g07Sbqy+xOttJZk3oMtxja0jMjP8rcdAqk9MHArjH8O65ErNJo2ooqgli1q 4AA654ru9W1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoPnibIOCSdpGTXmNC2A6rxBDomnxxJBZW czXVors9rflza3HG5RhmBTjo2Sdxw3HETx2zfDtCJoFu1v8AeYRefM8e0jeYi+N244yFBwAe mSdaK/8ADi+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ+YmtC51e1mtNXe8uLX7NLpnlwQ 2+oLJbeZtQIIrYqHjIIzyPlwfrQBQ13RdF0jTxJeWX2Sa709ZoYVE/mx3Py5QbspsH8QYlxk /wCzWL9r8P8A/CGfZvssn9qfat2d4348vG7f5f3N3/LPOe+a52u0vNF0IeGLl4bzT31CGKGS KSG4Cedx+8G15Sx4PTZGcjgc4oA6bWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFR twOnNcpfWHh9LS9e3+xnUltI2Nqt2TbxMfvmKTP7xwNp2FiAScF8YEt5ouhDwxcvDeae+oQx QyRSQ3ATzuP3g2vKWPB6bIzkcDnFcXQlYD1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfO Q2RuLFRtwOnNcNfXltolpNpGkzpPNINl9qEZ4lHeKI/88/U/x/TAPc61qehfZtaeHUYriyu7 A/ZYftC+VbkKqxxpb5yGyNxYqNuB05rlpi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcuaU dkB1OtanoX2bWnh1GK4sruwP2WH7QvlW5CqscaW+chsjcWKjbgdOa5S+sPD6Wl69v9jOpLaR sbVbsm3iY/fMUmf3jgbTsLEAk4L4wOQr1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2 RuLFRtwOnNC0sgMa8mvB4euYJtb025uJ7bdMjXkRggRMYhhhQ4Mh2j5goAxhTk5rZ1rU9C+z a08OoxXFld2B+yw/aF8q3IVVjjS3zkNkbixUbcDpzRrWqaGbXWXh1GO4sruwP2WD7QvlW5Cq scaW4OQ2RuLbRtwOnNeS0JdQCiu7iv8Aw4vgPWNPsZZYpNkJZriJFmuJN+eAJDlRgcD7oyfm JrT1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2kZNO4HmNFFenatqljJFrJW/gexl0 5UtlF0jW7PtjwI7QfPE2QcEk7SMmgDzGivQf7QI0jVF1DVba4WTTykJS7je3Zvk2LHahVaNg MDJA2kEkV59TAKK7LVrDQohfixj059PS2VrS7OoMLmR8L1jBbkksCpjUdeVxVmK/8OL4D1jT 7GWWKTZCWa4iRZriTfngCQ5UYHA+6Mn5iaVwOEoorsZi1x8N47V7jThcR3ouBFHPAjmEQ4yQ pBLZ4wcuaYHHUV6bq2qWLw6zsv4HsZNOWO2X7UrQM+2PAjtM74myDgknaRk15lQgCiuy1aw0 KIX4sY9OfT0tla0uzqDC5kfC9YwW5JLAqY1HXlcVf/tAjSNUXUNVtrhZNPKQlLuN7dm+TYsd qFVo2AwMkDaQSRSuB59RRXp2rapYyRayVv4HsZdOVLZRdI1uz7Y8CO0HzxNkHBJO0jJoA8xo r0u51e1mtNXe8uLX7NLpnlwQ2+oLJbeZtQIIrYqHjIIzyPlwfrXmlNAFFejX/wDZVt4X8Qad pN1ZfYnW2ksyb0GW4xtaRmRn+VuOgVSemDgVlTFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzx g5c0AcdRRXdxX/hxfAesafYyyxSbISzXESLNcSb88ASHKjA4H3Rk/MTQBwlFenatqljJFrJW /gexl05UtlF0jW7PtjwI7QfPE2QcEk7SMmvMaS1AKK9O1bVLGSLWSt/A9jLpypbKLpGt2fbH gR2g+eJsg4JJ2kZNUv7QI0jVF1DVba4WTTykJS7je3Zvk2LHahVaNgMDJA2kEkUJgefUUV3W s2HhK2srs2QWeEQL9lninTzS5C4ZwZsnnIYeUpGT0xmmBwtFdlq1hoUQvxYx6c+npbK1pdnU GFzI+F6xgtySWBUxqOvK4rjaEAUV61rWp6F9m1p4dRiuLK7sD9lh+0L5VuQqrHGlvnIbI3Fi o24HTmqerapYyRayVv4HsZdOVLZRdI1uz7Y8CO0HzxNkHBJO0jJpJgeY0UV3cV/4cXwHrGn2 MssUmyEs1xEizXEm/PAEhyowOB90ZPzE0wOEor03VdTsng1jbqMMllJpqx24+1I0DPtjwI7Q fPE2QcEn5SCTXmVCAKK9N1XU7J4NY26jDJZSaasduPtSNAz7Y8CO0HzxNkHBJ+Ugk0y41a2k stWa9urY28umeXDHBqCSWxk2oE8u22h4zkZ5Hy4P1pXA81rudU1GysvAtha6YVT7QJRPB9sh mZdzKymVQmWbavDAKUwBnPFcNXdxX/hxfAesafYyyxSbISzXESLNcSb88ASHKjA4H3Rk/MTT AmvJrweHrmCbW9Nubie23TI15EYIETGIYYUODIdo+YKAMYU5Oa8+r0u51e1mtNXe8uLX7NLp nlwQ2+oLJbeZtQIIrYqHjIIzyPlwfrXmlJbAegC/KaPqkeoarbXCPp5SEreJJbs3ybFjtQqt GwAAyR8pBJFUJi1x8N47V7jThcR3ouBFHPAjmEQ4yQpBLZ4wcua3dW1Sxki1krfwPYy6cqWy i6Rrdn2x4EdoPnibIOCSdpGTTLnV7Wa01d7y4tfs0umeXBDb6gslt5m1AgitioeMgjPI+XB+ tCYHmldlq1hoUQvxYx6c+npbK1pdnUGFzI+F6xgtySWBUxqOvK4rja9Gv/7KtvC/iDTtJurL 7E620lmTegy3GNrSMyM/ytx0CqT0wcCjqBqa1qehfZtaeHUYriyu7A/ZYftC+VbkKqxxpb5y GyNxYqNuB05ryWvRtQ1qbUrfVpNU1KxSGSzLQSabqEoEkhC7U8gyEgEEhgYx3zjrXnNC2A7u K/8ADi+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ+YmtPVtUsZItZK38D2MunKlsouka3Z 9seBHaD54myDgknaRk1mRX/hxfAesafYyyxSbISzXESLNcSb88ASHKjA4H3Rk/MTVWYtcfDe O1e404XEd6LgRRzwI5hEOMkKQS2eMHLmjzA46vTtW1Sxki1krfwPYy6cqWyi6Rrdn2x4EdoP nibIOCSdpGTXmNejahrU2pW+rSapqVikMlmWgk03UJQJJCF2p5BkJAIJDAxjvnHWjqBLc6va zWmrveXFr9ml0zy4IbfUFktvM2oEEVsVDxkEZ5Hy4P1rzSu7iv8Aw4vgPWNPsZZYpNkJZriJ FmuJN+eAJDlRgcD7oyfmJrhKEB3OqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAz nil1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM0sV/4cXwHrGn2MssUmyEs1xEiz XEm/PAEhyowOB90ZPzE1oXOr2s1pq73lxa/ZpdM8uCG31BZLbzNqBBFbFQ8ZBGeR8uD9aAPN K7uK/wDDi+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ+YmuEr07VtUsZItZK38D2MunKls ouka3Z9seBHaD54myDgknaRk0dQDVtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk1 5jXpdzq9rNaau95cWv2aXTPLght9QWS28zagQRWxUPGQRnkfLg/WvNKFsB2WrWGhRC/FjHpz 6elsrWl2dQYXMj4XrGC3JJYFTGo68rinL4pv5PDuqajfas13fXzGxFm8gVY42TLSiMd+NoIA AJJOc4purWGhRC/FjHpz6elsrWl2dQYXMj4XrGC3JJYFTGo68rijVrDQohfixj059PS2VrS7 OoMLmR8L1jBbkksCpjUdeVxQgONrstWsNCiF+LGPTn09LZWtLs6gwuZHwvWMFuSSwKmNR15X FcbXo2oa1NqVvq0mqalYpDJZloJNN1CUCSQhdqeQZCQCCQwMY75x1o6gZq+Kb+Tw7qmo32rN d318xsRZvIFWONky0ojHfjaCAACSTnOK4uvRtQ1qbUrfVpNU1KxSGSzLQSabqEoEkhC7U8gy EgEEhgYx3zjrXnNC2A7uK/8ADi+A9Y0+xllik2QlmuIkWa4k354AkOVGBwPujJ+Ymreoa1Nq Nvq0mqalZJDJZloJNO1GX97IQu1PJaQkAgkMCg7k461Y1bVLGSLWSt/A9jLpypbKLpGt2fbH gR2g+eJsg4JJ2kZNMudXtZrTV3vLi1+zS6Z5cENvqCyW3mbUCCK2Kh4yCM8j5cH60kB5pXoA vymj6pHqGq21wj6eUhK3iSW7N8mxY7UKrRsAAMkfKQSRXn9dzqmo2Vl4FsLXTCqfaBKJ4Ptk MzLuZWUyqEyzbV4YBSmAM54qgOh1nU9CFrrLQ6jHcWN1YEWsP2lfLtyFVY40gzuDZG4sVG3A 6fNXktejX/8AZVt4X8QadpN1ZfYnW2ksyb0GW4xtaRmRn+VuOgVSemDgV5zSWwHoN5NeDw9c wTa3ptzcT226ZGvIjBAiYxDDChwZDtHzBQBjCnJzVfVNRsrLwLYWumFU+0CUTwfbIZmXcysp lUJlm2rwwClMAZzxV3UNam1K31aTVNSsUhksy0Emm6hKBJIQu1PIMhIBBIYGMd8461F/aBGk aouoarbXCyaeUhKXcb27N8mxY7UKrRsBgZIG0gkihAefV6dq2qWMkWslb+B7GXTlS2UXSNbs +2PAjtB88TZBwSTtIya8xr07VtUsZItZK38D2MunKlsouka3Z9seBHaD54myDgknaRk0dQMK YtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmuOr0H+0CNI1RdQ1W2uFk08pCUu43t2b5Ni x2oVWjYDAyQNpBJFefUwO61mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDylIyemM1p6rqdi 8OsbL+B7GTTljtl+1q8DPtjwI7TO+Jsg4JJ2kEmvMqKVgCu0XxTfyeHdU1G+1Zru+vmNiLN5 AqxxsmWlEY78bQQAASSc5xXF0UwPQbya8Hh65gm1vTbm4ntt0yNeRGCBExiGGFDgyHaPmCgD GFOTmvPqKKAPTPEWoWF2dYa01XydNezX7LGLuKWF+E2xra7d0Z4+9wVIzx0qndz3Y8O3MMut abcXE9rumRryLyIEXGIYYUOPNO0fMFAGMKcnNef0UkrAFdjMWuPhvHavcacLiO9FwIo54Ecw iHGSFIJbPGDlzXHUUwPQbya8Hh65gm1vTbm4ntt0yNeRGCBExiGGFDgyHaPmCgDGFOTmvPqK KAPWdZ1TQzaay8WoxXFld6efs0P2hfKt2CqscaW+SQ24bi20bcDpyaNY1LQxaawYdSiubK60 8i1h+0r5UBCqscaQZLBiRuLFRtwOnzGvJqKVgCuimuYH8GxwabPHb7XH9o28jgS3D5+Rwf40 H9wD5Tyc/ernaKYHrOsanoYs9YaLUY7iyutPItYftK+XbkKqxxpBksGyNxYqNuB/tGue1TUb Oy8Cafa6aVQ3Cyi4gF5DMRuZWUyqEyzbV4YBSmAM54rh6KAOi1C5gbwrZQX08d3qQwbVoXBa 2g/uSsMhsn7qdV7kZ21ztFFAHrOr6non2PWGj1GK4s7vT/8ARoPtK+VbsFVY40gyWDbhuLFR twOnzGsrXf7Kv9PjfULixllt9EjRbmK9Ek32pTxHtVyGBycnaepO4dvO6KACu41ex8KW1hdN aBZo/s6/Zpop080yELgsDNkjOQw8oEZPTGa4eigD0TXf7Lv7COTUbmylmt9FRVuY70STG6U/ 6varkMCScnaepO4dvO6KKAO41TUbOy8Cafa6aVQ3Cyi4gF5DMRuZWUyqEyzbV4YBSmAM54o1 ax8KW2n3TWoSaMW6/Z54Z1MpkIXBYNNkjOQw8pSMnpjNcPRQAV3cV/4cXwHrGn2MssUmyEs1 xEizXEm/PAEhyowOB90ZPzE1wlFAHYzFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c1x1F FAHouoazLqFtqz6nqNhHA9kWt30zUJAJJCFCp5BkJCkEhgYx3zjrUviDUbG7GrvbaoINOeyX 7NEt3FLC5wmI1tdu6M8fe4KkZ4rzWilYAru4r/w4vgPWNPsZZYpNkJZriJFmuJN+eAJDlRgc D7oyfmJrhKKYHYzFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c1x1FFAHot/rM2o2uqvqe oWMUD2Ra3fTdQkAkkIXbH5BkOAQSGBjHfOOtGoazLqFtqz6nqNhHA9kWt30zUJAJJCFCp5Bk JCkEhgYx3zjrXnVFKwHqPinXrC6uvEiWeofapGiCJDPeK1o8ZVCXhGMeapHAznOSMn5a5r7U /wDwq/7J9vj3/wBoeZ9n+1Lv8nbj7mc48znGP9rHeuTr1HxRr1hd3PiRbS/+0yvEqJBcXita tGVQl4R081SOmeuSMn5aS0sgKPibxH9n022tWmXUZbnSktpz9vSeJJQQXcoucyDs+76Zwa88 r0PVNRS8stSaXUre3gexURRW17HNauwCYSO2ZQ8ecdeChBNHibxH9n022tWmXUZbnSktpz9v SeJJQQXcoucyDs+76ZwaaA88rsfFF3cXGjaStpqcb2Mel28dxbx3qf6wdQYt2SR8vbjHtXHV 0U1zA/g2ODTZ47fa4/tG3kcCW4fPyOD/ABoP7gHynk5+9TAJrmB/BscGmzx2+1x/aNvI4Etw +fkcH+NB/cA+U8nP3q19U1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZW XgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznijVNRsrLwLYWumFU+0CUTwfbIZmXcysp lUJlm2rwwClMAZzxSANU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFGqajZWX gWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznik1TUbOy8Cafa6aVQ3Cyi4gF5DMRuZWUy qEyzbV4YBSmAM54o1TUbOy8Cafa6aVQ3Cyi4gF5DMRuZWUyqEyzbV4YBSmAM54oAXVNRsrLw LYWumFU+0CUTwfbIZmXcysplUJlm2rwwClMAZzxRqmo2Vl4FsLXTCqfaBKJ4PtkMzLuZWUyq EyzbV4YBSmAM54rV1bVLGSLWSt/A9jLpypbKLpGt2fbHgR2g+eJsg4JJ2kZNJq2qWLw6zsv4 HsZNOWO2X7UrQM+2PAjtM74myDgknaRk0IDL1TUbKy8C2FrphVPtAlE8H2yGZl3MrKZVCZZt q8MApTAGc8UapqNlZeBbC10wqn2gSieD7ZDMy7mVlMqhMs21eGAUpgDOeK6HWtT0L7NrTw6j FcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNctMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbP GDlzQtgLGqajZWXgWwtdMKp9oEong+2QzMu5lZTKoTLNtXhgFKYAznitXVtUsZItZK38D2Mu nKlsouka3Z9seBHaD54myDgknaRk1h6tYaFEL8WMenPp6WytaXZ1BhcyPhesYLcklgVMajry uK09fOmahZJJqd1ZTT2+ioouo74Szm7B+5hXIYEk5baepO70AMuYtcfDeO1e404XEd6LgRRz wI5hEOMkKQS2eMHLmiYtcfDeO1e404XEd6LgRRzwI5hEOMkKQS2eMHLmtXUdbn1ODV5dV1Kx SGSzLwSabqMv72QhdqeS0hIBBIYGMd8461lTFrj4bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5 c0IDQ/tAjSNUXUNVtrhZNPKQlLuN7dm+TYsdqFVo2AwMkDaQSRUmvnTNQskk1O6spp7fRUUX Ud8JZzdg/cwrkMCScttPUnd6Zy+Kb+Tw7qmo32rNd318xsRZvIFWONky0ojHfjaCAACSTnOK 0b46XbeFtf07S7my+xultJaE3wMk5G1pGZGf5W46BVJ6YOAKAJ/EWoWF2dYa01XydNezX7LG LuKWF+E2xra7d0Z4+9wVIzx0qpeTXg8PXME2t6bc3E9tumRryIwQImMQwwocGQ7R8wUAYwpy c1SvNF0IeGLl4bzT31CGKGSKSG4Cedx+8G15Sx4PTZGcjgc4rpdY1PRPsmsNHqMVxZ3enn7N CLlfKt2CqscaQZyG3DcWKjbgdOaS00AXWtU0M2usvDqMdxZXdgfssH2hfKtyFVY40twchsjc W2jbgdOa4fULmBvCtlBfTx3epDBtWhcFraD+5KwyGyfup1XuRnbRqFzA3hWygvp47vUhg2rQ uC1tB/clYZDZP3U6r3IztrnaaVgOxtbu4TwFrNvf6nHJ5qWv2OBr1JGCh8kKgYlcDGRgdPau OoopgdpeaLoQ8MXLw3mnvqEMUMkUkNwE87j94NryljwemyM5HA5xUV9Y6FLoUs9mtpZyLbo6 CacSys3y5XKTH5jz1iUDvt6jkKKAOvvrHQpdClns1tLORbdHQTTiWVm+XK5SY/MeesSgd9vU WtU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFcNRQB2l5ouhDwxcvDeae+oQx QyRSQ3ATzuP3g2vKWPB6bIzkcDnFUHjtm+HaETQLdrf7zCLz5nj2kbzEXxu3HGQoOAD0yTzV FAHSpLC/w1e3E8H2hNV84wmVQ5Tygu4KTkjJ7e/oaZ9r8P8A/CGfZvssn9qfat2d4348vG7f 5f3N3/LPOe+a52igD1rWtT0L7NrTw6jFcWV3YH7LD9oXyrchVWONLfOQ2RuLFRtwOnNcpfWH h9LS9e3+xnUltI2Nqt2TbxMfvmKTP7xwNp2FiAScF8YHIUUkrAdjMWuPhvHavcacLiO9FwIo 54EcwiHGSFIJbPGDlzT7zRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOK4uimB0N 9eW2iWk2kaTOk80g2X2oRniUd4oj/wA8/U/x/TAPc61qehfZtaeHUYriyu7A/ZYftC+VbkKq xxpb5yGyNxYqNuB05ryWilYDr76w8PpaXr2/2M6ktpGxtVuybeJj98xSZ/eOBtOwsQCTgvjA dMWuPhvHavcacLiO9FwIo54EcwiHGSFIJbPGDlzXHUUwO61mw8JW1ldmyCzwiBfss8U6eaXI XDODNk85DDylIyemM1wtFFCAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA0pZtFMbiHT9QSQg7Ge+RgD2J AiGfzFEs2imNxDp+oJIQdjPfIwB7EgRDP5is2igDSlm0UxuIdP1BJCDsZ75GAPYkCIZ/MVv+ KLu4uNG0lbTU43sY9Lt47i3jvU/1g6gxbskj5e3GPauOooAK7rWbDwlbWV2bILPCIF+yzxTp 5pchcM4M2TzkMPKUjJ6YzXC0UAdpNZeGpYpkQWkTSaML1XS6YmO64/crliMHB+UgtyeemNK/ /sq28L+INO0m6svsTrbSWZN6DLcY2tIzIz/K3HQKpPTBwK85ooAK7TUruw1XwRp6wjS7WSzE 5eJppA8TGRSqxqWLNuB6kMBzyuK4uigDqvEEOiafHEkFlZzNdWiuz2t+XNrccblGGYFOOjZJ 3HDccWrzRdCHhi5eG8099QhihkikhuAnncfvBteUseD02RnI4HOK4uigDstWsNCiF+LGPTn0 9LZWtLs6gwuZHwvWMFuSSwKmNR15XFWNZsPCVtZXZsgs8IgX7LPFOnmlyFwzgzZPOQw8pSMn pjNcLRSsAV6NqGtT6lb6tJqmpWSQyWZaCTTtRl/eyELtTyWkJAIJDAxjvnHWvOaKYHYzFrj4 bx2r3GnC4jvRcCKOeBHMIhxkhSCWzxg5c1Z1mw8JW1ldmyCzwiBfss8U6eaXIXDODNk85DDy lIyemM1wtFAHouoazNqNtqz6pqVikL2RaCTTdRl/eSELtTyWkJAIJDAxjvnHWovt5XR9UTUN VtrhH08pCUvEkt2b5Nix2oVWjYAAZIG0gkivP6KQHZatYaFEL8WMenPp6WytaXZ1BhcyPhes YLcklgVMajryuK09Q1mbUbbVn1TUrFIXsi0Emm6jL+8kIXanktISAQSGBjHfOOtedUUWA9F1 DWZtRttWfVNSsUheyLQSabqMv7yQhdqeS0hIBBIYGMd8461marYaFEL4WMenPYJbK1rd/wBo N9od8L1jy2SSWBUxoOvK4rjaKLAeta1qehC11lodRjuLG6sCLWH7SvlW5CqscaQZ3BsjcWKj bgdPmrF+3ldH1RNQ1W2uEfTykJS8SS3Zvk2LHahVaNgABkgbSCSK8/ooSsB2WrWGhRC/FjHp z6elsrWl2dQYXMj4XrGC3JJYFTGo68rip9U1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavD AKUwBnPFcNRRYDudU1GysvAtha6YVT7QJRPB9shmZdzKymVQmWbavDAKUwBnPFT/AG8ro+qJ qGq21wj6eUhKXiSW7N8mxY7UKrRsAAMkDaQSRXn9FMD0XUNZm1G21Z9U1KxSF7ItBJpuoy/v JCF2p5LSEgEEhgYx3zjrXnVFFAHY2t3cJ4C1m3v9Tjk81LX7HA16kjBQ+SFQMSuBjIwOntWB LNopjcQ6fqCSEHYz3yMAexIEQz+YrNooA0pZtFMbiHT9QSQg7Ge+RgD2JAiGfzFEs2imNxDp +oJIQdjPfIwB7EgRDP5is2igAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig AooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA// 2Q== --------------090200000407090003070100-- From sgi-linux-xfs@lo.gmane.org Mon Feb 1 11:29:20 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o11HTJQk123353 for ; Mon, 1 Feb 2010 11:29:20 -0600 X-ASG-Debug-ID: 1265045425-30ce00a30000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from lo.gmane.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E973C1520B5B for ; Mon, 1 Feb 2010 09:30:25 -0800 (PST) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by cuda.sgi.com with ESMTP id cHf2Cfxj10UhFCHm for ; Mon, 01 Feb 2010 09:30:25 -0800 (PST) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Nc06O-0004nq-AT for linux-xfs@oss.sgi.com; Mon, 01 Feb 2010 18:30:20 +0100 Received: from vm15c-113.broadinstitute.org ([69.173.114.122]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 01 Feb 2010 18:30:20 +0100 Received: from nico by vm15c-113.broadinstitute.org with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 01 Feb 2010 18:30:20 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: linux-xfs@oss.sgi.com From: Nicolas Stransky X-ASG-Orig-Subj: Re: Filesystem corrupted: "Sorry, could not find valid secondary superblock" Subject: Re: Filesystem corrupted: "Sorry, could not find valid secondary superblock" Date: Mon, 01 Feb 2010 12:29:55 -0500 Lines: 73 Message-ID: References: <4B63010D.1080608@sandeen.net> <4B631704.8080902@sandeen.net> <4B6325E2.2040703@sandeen.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: vm15c-113.broadinstitute.org User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.7) Gecko/20100129 Shredder/3.0.2pre In-Reply-To: <4B6325E2.2040703@sandeen.net> Sender: news X-Barracuda-Connect: lo.gmane.org[80.91.229.12] X-Barracuda-Start-Time: 1265045426 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21378 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello, Unfortunately I am still having big problems with this filesystem. I can mount it OK but there are problems reading files: du: cannot access `./DCIS 2 AB': Structure needs cleaning The previous goes together with a lot of these in dmesg: [ 431.976517] 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 431.988072] Filesystem "sda1": XFS internal error xfs_da_do_buf(2) at line 2085 of file fs/xfs/xfs_da_btree.c. Caller 0xffffffffa016749f [ 431.993185] Pid: 4246, comm: du Not tainted 2.6.26-2-amd64 #1 [ 431.998308] [ 431.998309] Call Trace: [ 432.010348] [] :xfs:xfs_da_read_buf+0x24/0x29 [ 432.015530] [] :xfs:xfs_da_do_buf+0x54e/0x636 [ 432.020696] [] :xfs:xfs_da_read_buf+0x24/0x29 [ 432.025860] [] :xfs:_xfs_buf_lookup_pages+0x298/0x2ca [ 432.031974] [] :xfs:xfs_da_read_buf+0x24/0x29 [ 432.037173] [] :xfs:xfs_dir2_leaf_getdents+0x381/0x61a [ 432.042391] [] :xfs:xfs_dir2_leaf_getdents+0x381/0x61a [ 432.047617] [] :xfs:xfs_hack_filldir+0x0/0x5b [ 432.052673] [] :xfs:xfs_hack_filldir+0x0/0x5b [ 432.057517] [] :xfs:xfs_readdir+0xa6/0xb5 [ 432.062197] [] :xfs:xfs_file_readdir+0xff/0x14c [ 432.066835] [] filldir+0x0/0xb7 [ 432.071987] [] filldir+0x0/0xb7 [ 432.076557] [] vfs_readdir+0x75/0xa7 [ 432.081137] [] sys_getdents+0x75/0xbd [ 432.085716] [] sys_fcntl+0x2eb/0x2f7 [ 432.090267] [] system_call_after_swapgs+0x8a/0x8f The server has 2GB of RAM and 4GB of swap. I have tried several times to xfs_repair but it always fails because of memory problems, even if I use 'xfs_repair -m1500 /dev/sda1' to limit the memory usage. xfs_repair produces an awful lot of output. I could of course send it but it's 130k lines long. Here is what it is currently printing, I have started "xfs_repair -t15 -m500 /dev/sda1" again a few minutes ago. It is currently in Phase 3. bp=(bno 1211109312, len 8192 bytes) key=(bno 1211109312, len 4096 bytes) 429f7950: Badness in key lookup (length) bp=(bno 1211109328, len 8192 bytes) key=(bno 1211109328, len 4096 bytes) bad directory block magic # 0x2f49cdbc in block 2 for directory inode 2952790372 corrupt block 2 in directory inode 2952790372 will junk block bad directory block magic # 0 in block 5 for directory inode 2952790372 corrupt block 5 in directory inode 2952790372 will junk block no . entry for directory 2952790372 no .. entry for directory 2952790372 bad directory block magic # 0 in block 0 for directory inode 2952790373 corrupt block 0 in directory inode 2952790373 will junk block bad directory block magic # 0 in block 1 for directory inode 2952790373 corrupt block 1 in directory inode 2952790373 will junk block doubling cache size to 66768 doubling cache size to 133536 doubling cache size to 267072 doubling cache size to 534144 doubling cache size to 1068288 Any ideas on how I could repair this filesystem? Thanks a lot! -- Nico From BATV+c2a8630ad946b3a996ee+2353+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 1 16:07:07 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o11M75cf136973 for ; Mon, 1 Feb 2010 16:07:07 -0600 X-ASG-Debug-ID: 1265062093-264b01660000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0429B136BC8C for ; Mon, 1 Feb 2010 14:08:13 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id XdvGi3IpadFmJoY3 for ; Mon, 01 Feb 2010 14:08:13 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nc4RJ-00014e-Eo for xfs@oss.sgi.com; Mon, 01 Feb 2010 22:08:13 +0000 Date: Mon, 1 Feb 2010 17:08:13 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100201220813.GA3519@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265062094 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Make sure all log buffers have made it to disk before we free them. Otherwise we might reference freed buffers or a NULL mp->m_log from the I/O completion handlers. This fixes kernel.org bz #15150. Signed-off-by: Christoph Hellwig Reported-by: Ed Cashin Reported-by: ghani Tested-by: ghani Index: linux-2.6/fs/xfs/xfs_log.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_log.c 2009-11-09 22:09:08.858026060 +0100 +++ linux-2.6/fs/xfs/xfs_log.c 2009-11-09 22:13:13.958255857 +0100 @@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log) xlog_in_core_t *iclog, *next_iclog; int i; + xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1); + iclog = log->l_iclog; for (i=0; il_iclog_bufs; i++) { sv_destroy(&iclog->ic_force_wait); From edward.shishkin@gmail.com Mon Feb 1 19:39:19 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o121cwVQ147614 for ; Mon, 1 Feb 2010 19:39:09 -0600 X-ASG-Debug-ID: 1265074783-2ee200e20000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from ey-out-1920.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0FA221A3FE6 for ; Mon, 1 Feb 2010 17:39:43 -0800 (PST) Received: from ey-out-1920.google.com (ey-out-1920.google.com [74.125.78.145]) by cuda.sgi.com with ESMTP id IEEgjKxGlXpjBfrn for ; Mon, 01 Feb 2010 17:39:43 -0800 (PST) Received: by ey-out-1920.google.com with SMTP id 26so1474966eyw.22 for ; Mon, 01 Feb 2010 17:39:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:to:subject:date :user-agent:mime-version:message-id:cc:content-type :content-transfer-encoding; bh=8phbelOUN7gAqV0QeYNcaf/vko4/bnGeLVk6flfcHls=; b=YTO2SyokFMgUJnwIJMClHKJjDzLW6q7gSkhqWzZq+bBSTI08QYBJCiKbAx979XBhSO 0GQCODLYL2D9qRo6M6ft8zG6SI3KPdaWbp9UYhGmKqsstyJGmuO9vAhEa8Ferp6z5PEn ejcQevzhv4728TwzcnL6GgTFZyoJL2T5R2Jns= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:message-id:cc :content-type:content-transfer-encoding; b=MC35OFo/6xQU4DOKq68CGzN0LuKeP2K//SPOKD+0tA8NghBn/46NjfNByHjcm/VvhZ bb5WAjdSm60yFEe89vXnsD1dJ8KFX3J7DQfGtQ2lFpT6Zqr+R/8qtZiBeAO3H8+KafOA S2wCpCAzJpe3ti22Pu/ARD9BRervtSBv4nho4= Received: by 10.213.100.68 with SMTP id x4mr44449ebn.33.1265074781877; Mon, 01 Feb 2010 17:39:41 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 16sm4132961ewy.14.2010.02.01.17.39.40 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 17:39:41 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 16sm4096070ewy.2.2010.02.01.16.14.25 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 16:14:25 -0800 (PST) From: Edward Shishkin To: Andrew Morton , ReiserFS Development List X-ASG-Orig-Subj: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: [patch 0/7] per-bdi flushing model improvements. reiser4 Date: Tue, 2 Feb 2010 02:39:27 +0100 User-Agent: KMail/1.12.3 (Linux/2.6.27.41-170.2.117.fc10.i686; KDE/4.3.3; i686; ; ) MIME-Version: 1.0 Message-Id: <201002020239.27368.edward.shishkin@gmail.com> Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ey-out-1920.google.com[74.125.78.145] X-Barracuda-Start-Time: 1265074788 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21411 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello. Andrew Morton wrote: > reiser4 is currently disabled in -mm (via reiser4-disable.patch) > because recent changes to fs/fs/writeback.c wrecked the build. I fixed > it about ten times as the underlying code was churning, then gave up. It > would be nice if you take a look at that sometime please. > > I have taken a look at fs/fs-writeback.c and found that per-superblock flushing interface is eliminated. However migrating to per-bdi flushing model doesn't necessarily means that such interface doesn't exist or is not needed anymore. Flushing in accordance with the scheme "data-inode- data-inode-..." would be very suboptimal for reiser4. Also xfs people were unhappy with such flushing model: http://article.gmane.org/gmane.linux.file-systems/30153 Moreover, the current stuff looks rather ugly. Why do we pin/unpin superblock for every inode? It would be more reasonable to pin it for the whole group of inodes and call a flushing handler for them. The patch 4 introduces such handler writeback_sb_inodes (which resembles dropped sync_sb_inodes, the difference is that the newer version doesn't flush necessarily all inodes of the superblock). Please, consider pushing this patch to mainline. The patch 5 adds super operation .writeback_inodes (former .sync_inodes) which allows a file system to make optimizations. It can happen that reiser4 will flush a bit more inodes then generic implementation suggests. "a bit more" doesn't mean "all dirty inodes of the superblock" (see a comment about atoms in the header of patch 6). Finally, some file systems have its own means for periodical writeout of dirty data. Since b_io contains inodes of many superblocks we need to evict our inodes back to dirty list when flushing is going on with for_kupdate flag installed. The new library function writeback_skip_sb_inodes() provides such possibility. Patch 7 fixes a race in checkin-checkout jnodes for entd task (reiser4). Please, apply. Thanks, Edward. From edward.shishkin@gmail.com Mon Feb 1 19:55:47 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.1 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, HEADER_ESQ autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o121tRO3148061 for ; Mon, 1 Feb 2010 19:55:37 -0600 X-ASG-Debug-ID: 1265075779-2ee7012c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 81F341A3D57 for ; Mon, 1 Feb 2010 17:56:19 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id 3cDKARroc4Iw00w7 for ; Mon, 01 Feb 2010 17:56:19 -0800 (PST) Received: by ewy27 with SMTP id 27so915298ewy.18 for ; Mon, 01 Feb 2010 17:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:to:subject:date :user-agent:mime-version:message-id:cc:content-type :content-transfer-encoding; bh=c3zD/LlZNEeBkuHpWllRmZolDyQkFE7bA0yGA23VOjo=; b=YZVZ5UYv6yyweDU1ka5n0/Yb0vUTQCw6ym+6gbPWXEVWU2qLunQte8+zg0/Jda2OP0 xhdP+ZYS6DnFBlYe2MDa85dO9t1hBoP8im4vLN8qRrVEob8vVi3IQjhzsDSjXWWPxt1b T/YZ8yme2rPzuiy9q+HJDYEsQHurWJOCApQOc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:message-id:cc :content-type:content-transfer-encoding; b=mSNopl94VL/shJkT79l6HGUudldBRccHdL/NePvff+PQa92HVKDpmLwX6Mj0cQ5juf QOiZJcEqjNEWECLU2FvWTqmlhC9FFRw4P/IqfwILyehSOI3xzi2oBdhWW2H4mWzy3+mm nfPywJNwd25o6U1IxYHiPA3/fOkBVcFwzIX9U= Received: by 10.213.38.11 with SMTP id z11mr54600ebd.36.1265075778625; Mon, 01 Feb 2010 17:56:18 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 16sm4138426ewy.10.2010.02.01.17.56.18 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 17:56:18 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 14sm4074067ewy.15.2010.02.01.16.15.03 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 16:15:03 -0800 (PST) From: Edward Shishkin To: Andrew Morton , ReiserFS Development List X-ASG-Orig-Subj: [patch 6/7] reiser4: writeback_inodes implementation Subject: [patch 6/7] reiser4: writeback_inodes implementation Date: Tue, 2 Feb 2010 02:56:27 +0100 User-Agent: KMail/1.12.3 (Linux/2.6.27.41-170.2.117.fc10.i686; KDE/4.3.3; i686; ; ) MIME-Version: 1.0 Message-Id: <201002020256.27208.edward.shishkin@gmail.com> Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265075782 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21413 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean . add reiser4 implementation of ->writeback_inodes() super operation; . cleanup comments. Signed-off-by: Edward Shishkin --- fs/reiser4/context.c | 3 +- fs/reiser4/context.h | 2 - fs/reiser4/entd.c | 7 ++++-- fs/reiser4/page_cache.c | 12 +++++------ fs/reiser4/super_ops.c | 49 +++++++++++++++++++++++++++++------------------- fs/reiser4/txnmgr.c | 2 - 6 files changed, 45 insertions(+), 30 deletions(-) Index: linux-2.6.33-rc5-mm1/fs/reiser4/context.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/context.c +++ linux-2.6.33-rc5-mm1/fs/reiser4/context.c @@ -151,7 +151,8 @@ static void reiser4_throttle_write_at(re */ if (sbinfo != NULL && sbinfo->fake != NULL && context->nr_marked_dirty != 0 && - !(current->flags & PF_MEMALLOC)) + !(current->flags & PF_MEMALLOC) && + !current_is_flush_bd_task()) /* FIXME-EDWARD: throttle with nr_marked_dirty? */ reiser4_throttle_write(sbinfo->fake, 1); } Index: linux-2.6.33-rc5-mm1/fs/reiser4/context.h =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/context.h +++ linux-2.6.33-rc5-mm1/fs/reiser4/context.h @@ -66,7 +66,7 @@ struct reiser4_context { /* count non-trivial jnode_set_dirty() calls */ unsigned long nr_marked_dirty; - /* reiser4_sync_inodes calls (via generic_sync_sb_inodes) + /* reiser4_writeback_inodes calls (via generic_writeback_sb_inodes) * reiser4_writepages for each of dirty inodes. Reiser4_writepages * captures pages. When number of pages captured in one * reiser4_sync_inodes reaches some threshold - some atoms get Index: linux-2.6.33-rc5-mm1/fs/reiser4/entd.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/entd.c +++ linux-2.6.33-rc5-mm1/fs/reiser4/entd.c @@ -236,16 +236,19 @@ static void entd_flush(struct super_bloc rq->wbc->range_end = rq->wbc->range_start + (ENTD_CAPTURE_APAGE_BURST << PAGE_CACHE_SHIFT); tmp = rq->wbc->nr_to_write; + + assert("edward-1561", super == rq->wbc->sb); + rq->mapping->a_ops->writepages(rq->mapping, rq->wbc); if (rq->wbc->nr_to_write > 0) { rq->wbc->range_start = 0; rq->wbc->range_end = LLONG_MAX; - generic_sync_sb_inodes(rq->wbc); + writeback_inodes_wbc(rq->wbc); } rq->wbc->nr_to_write = ENTD_CAPTURE_APAGE_BURST; - reiser4_writeout(super, rq->wbc); + reiser4_writeout(super, rq->wbc); context_set_commit_async(&ctx); reiser4_exit_context(&ctx); } Index: linux-2.6.33-rc5-mm1/fs/reiser4/page_cache.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/page_cache.c +++ linux-2.6.33-rc5-mm1/fs/reiser4/page_cache.c @@ -486,15 +486,15 @@ static int can_hit_entd(reiser4_context int reiser4_writepage(struct page *page, struct writeback_control *wbc) { - struct super_block *s; - reiser4_context *ctx; - + /* + * assert("edward-1562", + * can_hit_entd(get_current_context_check(), sb)); + */ assert("vs-828", PageLocked(page)); - s = page->mapping->host->i_sb; - ctx = get_current_context_check(); + wbc->sb = page->mapping->host->i_sb; + wbc->bdi = page->mapping->backing_dev_info; - /* assert("", can_hit_entd(ctx, s)); */ return write_page_by_ent(page, wbc); } Index: linux-2.6.33-rc5-mm1/fs/reiser4/super_ops.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/super_ops.c +++ linux-2.6.33-rc5-mm1/fs/reiser4/super_ops.c @@ -379,48 +379,59 @@ static void reiser4_clear_inode(struct i } /** - * reiser4_sync_inodes - sync_inodes of super operations + * reiser4_writeback_inodes - writeback_inodes of super operations * @super: + * @wb: * @wbc: * * This method is called by background and non-backgound writeback. Reiser4's - * implementation uses generic_sync_sb_inodes to call reiser4_writepages for - * each of dirty inodes. Reiser4_writepages handles pages dirtied via shared - * mapping - dirty pages get into atoms. Writeout is called to flush some - * atoms. + * implementation uses generic_writeback_sb_inodes to call reiser4_writepages + * for each of dirty inodes. reiser4_writepages handles pages dirtied via shared + * mapping - dirty pages get into atoms. Writeout is called to flush some atoms. */ -static void reiser4_sync_inodes(struct super_block *super, - struct writeback_control *wbc) +static int reiser4_writeback_inodes(struct super_block *super, + struct bdi_writeback *wb, + struct writeback_control *wbc) { - reiser4_context *ctx; + int ret; long to_write; + reiser4_context *ctx; if (wbc->for_kupdate) /* reiser4 has its own means of periodical write-out */ - return; - - to_write = wbc->nr_to_write; + goto skip; assert("vs-49", wbc->older_than_this == NULL); + spin_unlock(&inode_lock); ctx = reiser4_init_context(super); if (IS_ERR(ctx)) { warning("vs-13", "failed to init context"); - return; + spin_lock(&inode_lock); + goto skip; } - + to_write = wbc->nr_to_write; /* - * call reiser4_writepages for each of dirty inodes to turn dirty pages - * into transactions if they were not yet. + * call reiser4_writepages for each of dirty inodes to turn + * dirty pages into transactions if they were not yet. */ - generic_sync_sb_inodes(wbc); + spin_lock(&inode_lock); + ret = generic_writeback_sb_inodes(super, wb, wbc); + spin_unlock(&inode_lock); - /* flush goes here */ wbc->nr_to_write = to_write; + + /* flush goes here */ reiser4_writeout(super, wbc); - /* avoid recursive calls to ->sync_inodes */ + /* avoid recursive calls to ->writeback_inodes */ context_set_commit_async(ctx); reiser4_exit_context(ctx); + spin_lock(&inode_lock); + + return wbc->nr_to_write <= 0 ? 1 : ret; + skip: + writeback_skip_sb_inodes(super, wb); + return 0; } /** @@ -458,7 +469,7 @@ struct super_operations reiser4_super_op .write_super = reiser4_write_super, .statfs = reiser4_statfs, .clear_inode = reiser4_clear_inode, - .sync_inodes = reiser4_sync_inodes, + .writeback_inodes = reiser4_writeback_inodes, .show_options = reiser4_show_options }; Index: linux-2.6.33-rc5-mm1/fs/reiser4/txnmgr.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/reiser4/txnmgr.c +++ linux-2.6.33-rc5-mm1/fs/reiser4/txnmgr.c @@ -1410,7 +1410,7 @@ flush_some_atom(jnode * start, long *nr_ * Write throttling is case of no one atom can be * flushed/committed. */ - if (!wbc->nonblocking) { + if (!wbc->nonblocking && !current_is_flush_bd_task()) { list_for_each_entry(atom, &tmgr->atoms_list, atom_link) { spin_lock_atom(atom); /* Repeat the check from the above. */ From edward.shishkin@gmail.com Mon Feb 1 19:56:01 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.4 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o121tfb5148068 for ; Mon, 1 Feb 2010 19:55:51 -0600 X-ASG-Debug-ID: 1265075774-33c803840000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D12661C978D6 for ; Mon, 1 Feb 2010 17:56:14 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id xUjXQr4GeMEZhLPp for ; Mon, 01 Feb 2010 17:56:14 -0800 (PST) Received: by ewy27 with SMTP id 27so915224ewy.18 for ; Mon, 01 Feb 2010 17:56:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:to:subject:date :user-agent:mime-version:message-id:cc:content-type :content-transfer-encoding; bh=8XRM8+zUYcaWPP+4tswejqVP/EH9DBkNfX+Nx4FroHw=; b=DQ62v01F0+kYEr7R+pw4C4EGOpkzf6IAhOBFj6PKnnhVZ40jWcjimiLsHsa7e1bbyW mEOMl2g13ZBdXYcKM4QGNkzsiOjpZ6v1vjx2A/zuIA+7GzKKozG1ktos78sWJPs/5ukb 2fzfxs8LSF1lPqV/MC6RlIsGTUYEN039hwZmA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:message-id:cc :content-type:content-transfer-encoding; b=uaPxGXUwo+4deC6xCXDM8sUfKl7OUnqefFo4hBoXxquoEtfuSxtbw/A2uEmmBFsUHr CUDxhxY9Vmeh4lhTamAVM1BMbjVdeTxmNCqPc71NUEcdHLGcBO2L6MAurg1oS1HR4sLE JZ204wBFdO0L1RLQIPMpkIsOTgmE25SDnZ6uU= Received: by 10.213.2.79 with SMTP id 15mr1317664ebi.96.1265075770522; Mon, 01 Feb 2010 17:56:10 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 13sm4118001ewy.9.2010.02.01.17.56.09 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 17:56:10 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 13sm4090804ewy.13.2010.02.01.16.14.52 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 16:14:52 -0800 (PST) From: Edward Shishkin To: Andrew Morton , ReiserFS Development List X-ASG-Orig-Subj: [patch 4/7] VFS: improve writeback_inodes_wb Subject: [patch 4/7] VFS: improve writeback_inodes_wb Date: Tue, 2 Feb 2010 02:56:18 +0100 User-Agent: KMail/1.12.3 (Linux/2.6.27.41-170.2.117.fc10.i686; KDE/4.3.3; i686; ; ) MIME-Version: 1.0 Message-Id: <201002020256.18841.edward.shishkin@gmail.com> Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265075799 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21413 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Do not pin/unpin superblock for every inode in writeback_inodes_wb(), pin it for the whole group of inodes which belong to the same superblock and call writeback_sb_inodes() handler for them. Signed-off-by: Edward Shishkin --- fs/fs-writeback.c | 133 +++++++++++++++++++++++++--------------------- include/linux/writeback.h | 3 + 2 files changed, 76 insertions(+), 60 deletions(-) Index: linux-2.6.33-rc5-mm1/fs/fs-writeback.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/fs-writeback.c +++ linux-2.6.33-rc5-mm1/fs/fs-writeback.c @@ -553,108 +553,85 @@ select_queue: return ret; } -static void unpin_sb_for_writeback(struct super_block **psb) +static void unpin_sb_for_writeback(struct super_block *sb) { - struct super_block *sb = *psb; - - if (sb) { - up_read(&sb->s_umount); - put_super(sb); - *psb = NULL; - } + up_read(&sb->s_umount); + put_super(sb); } +enum sb_pin_state { + SB_PINNED, + SB_NOT_PINNED, + SB_PIN_FAILED +}; + /* * For WB_SYNC_NONE writeback, the caller does not have the sb pinned * before calling writeback. So make sure that we do pin it, so it doesn't * go away while we are writing inodes from it. - * - * Returns 0 if the super was successfully pinned (or pinning wasn't needed), - * 1 if we failed. */ -static int pin_sb_for_writeback(struct writeback_control *wbc, - struct inode *inode, struct super_block **psb) +static enum sb_pin_state pin_sb_for_writeback(struct writeback_control *wbc, + struct super_block *sb) { - struct super_block *sb = inode->i_sb; - - /* - * If this sb is already pinned, nothing more to do. If not and - * *psb is non-NULL, unpin the old one first - */ - if (sb == *psb) - return 0; - else if (*psb) - unpin_sb_for_writeback(psb); - /* * Caller must already hold the ref for this */ if (wbc->sync_mode == WB_SYNC_ALL) { WARN_ON(!rwsem_is_locked(&sb->s_umount)); - return 0; + return SB_NOT_PINNED; } - spin_lock(&sb_lock); sb->s_count++; if (down_read_trylock(&sb->s_umount)) { if (sb->s_root) { spin_unlock(&sb_lock); - goto pinned; + return SB_PINNED; } /* * umounted, drop rwsem again and fall through to failure */ up_read(&sb->s_umount); } - sb->s_count--; spin_unlock(&sb_lock); - return 1; -pinned: - *psb = sb; - return 0; + return SB_PIN_FAILED; } -static void writeback_inodes_wb(struct bdi_writeback *wb, - struct writeback_control *wbc) +/* + * Write a portion of b_io inodes which belong to @sb. + * If @wbc->sb != NULL, then find and write all such + * inodes. Otherwise write only ones which go sequentially + * in reverse order. + * Return 1, if the caller writeback routine should be + * interrupted. Otherwise return 0. + */ +static int writeback_sb_inodes(struct super_block *sb, + struct bdi_writeback *wb, + struct writeback_control *wbc) { - struct super_block *sb = wbc->sb, *pin_sb = NULL; - const unsigned long start = jiffies; /* livelock avoidance */ - - spin_lock(&inode_lock); - - if (!wbc->for_kupdate || list_empty(&wb->b_io)) - queue_io(wb, wbc->older_than_this); - while (!list_empty(&wb->b_io)) { - struct inode *inode = list_entry(wb->b_io.prev, - struct inode, i_list); long pages_skipped; - - /* - * super block given and doesn't match, skip this inode - */ - if (sb && sb != inode->i_sb) { + struct inode *inode = list_entry(wb->b_io.prev, + struct inode, i_list); + if (wbc->sb && sb != inode->i_sb) { + /* super block given and doesn't + match, skip this inode */ redirty_tail(inode); continue; } - + if (sb != inode->i_sb) + /* finish with this superblock */ + return 0; if (inode->i_state & (I_NEW | I_WILL_FREE)) { requeue_io(inode); continue; } - /* * Was this inode dirtied after sync_sb_inodes was called? * This keeps sync from extra jobs and livelock. */ - if (inode_dirtied_after(inode, start)) - break; - - if (pin_sb_for_writeback(wbc, inode, &pin_sb)) { - requeue_io(inode); - continue; - } + if (inode_dirtied_after(inode, wbc->wb_start)) + return 1; BUG_ON(inode->i_state & (I_FREEING | I_CLEAR)); __iget(inode); @@ -673,14 +650,50 @@ static void writeback_inodes_wb(struct b spin_lock(&inode_lock); if (wbc->nr_to_write <= 0) { wbc->more_io = 1; - break; + return 1; } if (!list_empty(&wb->b_more_io)) wbc->more_io = 1; } + /* b_io is empty */ + return 1; +} + +static void writeback_inodes_wb(struct bdi_writeback *wb, + struct writeback_control *wbc) +{ + int ret = 0; - unpin_sb_for_writeback(&pin_sb); + wbc->wb_start = jiffies; /* livelock avoidance */ + spin_lock(&inode_lock); + if (!wbc->for_kupdate || list_empty(&wb->b_io)) + queue_io(wb, wbc->older_than_this); + while (!list_empty(&wb->b_io)) { + struct inode *inode = list_entry(wb->b_io.prev, + struct inode, i_list); + struct super_block *sb = inode->i_sb; + enum sb_pin_state state; + + if (wbc->sb && sb != wbc->sb) { + /* super block given and doesn't + match, skip this inode */ + redirty_tail(inode); + continue; + } + state = pin_sb_for_writeback(wbc, sb); + + if (state == SB_PIN_FAILED) { + requeue_io(inode); + continue; + } + ret = writeback_sb_inodes(sb, wb, wbc); + + if (state == SB_PINNED) + unpin_sb_for_writeback(sb); + if (ret) + break; + } spin_unlock(&inode_lock); /* Leave any unwritten inodes on b_io */ } Index: linux-2.6.33-rc5-mm1/include/linux/writeback.h =================================================================== --- linux-2.6.33-rc5-mm1.orig/include/linux/writeback.h +++ linux-2.6.33-rc5-mm1/include/linux/writeback.h @@ -34,6 +34,9 @@ struct writeback_control { enum writeback_sync_modes sync_mode; unsigned long *older_than_this; /* If !NULL, only write back inodes older than this */ + unsigned long wb_start; /* Time writeback_inodes_wb was + called. This is needed to avoid + extra jobs and livelock */ long nr_to_write; /* Write this many pages, and decrement this for each page written */ long pages_skipped; /* Pages which were not written */ From edward.shishkin@gmail.com Mon Feb 1 20:02:05 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.2 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1221i8u148270 for ; Mon, 1 Feb 2010 20:01:55 -0600 X-ASG-Debug-ID: 1265076159-2ee601320000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7AB711A3D7E for ; Mon, 1 Feb 2010 18:02:39 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id jHLs40rkYbdGyUr6 for ; Mon, 01 Feb 2010 18:02:39 -0800 (PST) Received: by ewy27 with SMTP id 27so918342ewy.18 for ; Mon, 01 Feb 2010 18:02:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:to:subject:date :user-agent:mime-version:message-id:cc:content-type :content-transfer-encoding; bh=BGDOYJkpSLs+yza6iJEfng+32tDSs5+CLnsMuKVhdg0=; b=NST20EMr1cUgP1Nk5uqYJqpFHlxH33G0i0P2s65tT6ZcNAr7++a6FxrsQROBsUuD9h UzddYzM+qXansiESvkvsPU/l4v92Nru4vcs8ZODyICewI7vRT6i03KGEXrZT/IL2LtSh ze54ilK+9aIAYn1uboDsQzLeNUatGCUHxaD0o= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:message-id:cc :content-type:content-transfer-encoding; b=ekVFOExD3nmZqCU3AnfwVcYlGAWNn/rNEiqTi045JgWk79f/mWLKBIAyFCMouUXafl IIkwg7WfzNNpN8+fNSHFj6rYbqmc4rUO6tGy2WASVRH63GchnT4N/L3H/wXS3QQPBbtb g6o9YXbDap82fc43/JWQUUI+lD+r7h+C/WGBU= Received: by 10.213.41.133 with SMTP id o5mr746809ebe.86.1265075772521; Mon, 01 Feb 2010 17:56:12 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 15sm4131032ewy.12.2010.02.01.17.56.12 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 17:56:12 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 15sm4072935ewy.8.2010.02.01.16.14.58 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 16:14:58 -0800 (PST) From: Edward Shishkin To: Andrew Morton , ReiserFS Development List X-ASG-Orig-Subj: [patch 5/7] VFS: add super operation writeback_inodes Subject: [patch 5/7] VFS: add super operation writeback_inodes Date: Tue, 2 Feb 2010 02:56:21 +0100 User-Agent: KMail/1.12.3 (Linux/2.6.27.41-170.2.117.fc10.i686; KDE/4.3.3; i686; ; ) MIME-Version: 1.0 Message-Id: <201002020256.21266.edward.shishkin@gmail.com> Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265076160 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21413 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean . add ->writeback_inodes() super operation. This patch adds new operation to struct super_operations - writeback_inodes, generic implementaion and changes fs-writeback.c:writeback_inodes_wb() to call filesystem's writeback_inodes if it is defined or generic implementaion otherwise. This new operation allows filesystem to decide itself what to flush. Reiser4 flushes dirty pages on basic of atoms, not of inodes. writeback_inodes_wb used to call address space flushing method (writepages) for every dirty inode. For reiser4 it caused having to commit atoms unnecessarily often. This turned into substantial slowdown. Having this method helped to fix that problem. . add vfs library function writeback_skip_sb_inodes() This function is for file systems which have their own means of periodical writeout of old data. Signed-off-by: Edward Shishkin --- fs/fs-writeback.c | 47 ++++++++++++++++++++++++++++++++++++++++++---- include/linux/fs.h | 10 +++++++++ include/linux/writeback.h | 6 +++++ 3 files changed, 59 insertions(+), 4 deletions(-) Index: linux-2.6.33-rc5-mm1/include/linux/fs.h =================================================================== --- linux-2.6.33-rc5-mm1.orig/include/linux/fs.h +++ linux-2.6.33-rc5-mm1/include/linux/fs.h @@ -514,6 +514,7 @@ enum positive_aop_returns { struct page; struct address_space; struct writeback_control; +struct bdi_writeback; struct iov_iter { const struct iovec *iov; @@ -1565,6 +1566,9 @@ struct super_operations { int (*remount_fs) (struct super_block *, int *, char *); void (*clear_inode) (struct inode *); void (*umount_begin) (struct super_block *); + int (*writeback_inodes)(struct super_block *sb, + struct bdi_writeback *wb, + struct writeback_control *wbc); int (*show_options)(struct seq_file *, struct vfsmount *); int (*show_stats)(struct seq_file *, struct vfsmount *); @@ -2073,6 +2077,12 @@ extern int invalidate_inode_pages2(struc extern int invalidate_inode_pages2_range(struct address_space *mapping, pgoff_t start, pgoff_t end); extern int write_inode_now(struct inode *, int); +extern void writeback_skip_sb_inodes(struct super_block *sb, + struct bdi_writeback *wb); +extern void writeback_inodes_wbc(struct writeback_control *wbc); +extern int generic_writeback_sb_inodes(struct super_block *sb, + struct bdi_writeback *wb, + struct writeback_control *wbc); extern int filemap_fdatawrite(struct address_space *); extern int filemap_flush(struct address_space *); extern int filemap_fdatawait(struct address_space *); Index: linux-2.6.33-rc5-mm1/include/linux/writeback.h =================================================================== --- linux-2.6.33-rc5-mm1.orig/include/linux/writeback.h +++ linux-2.6.33-rc5-mm1/include/linux/writeback.h @@ -13,6 +13,12 @@ extern spinlock_t inode_lock; extern struct list_head inode_in_use; extern struct list_head inode_unused; +static inline int is_flush_bd_task(struct task_struct *task) +{ + return task->flags & PF_FLUSHER; +} +#define current_is_flush_bd_task() is_flush_bd_task(current) + /* * fs/fs-writeback.c */ Index: linux-2.6.33-rc5-mm1/fs/fs-writeback.c =================================================================== --- linux-2.6.33-rc5-mm1.orig/fs/fs-writeback.c +++ linux-2.6.33-rc5-mm1/fs/fs-writeback.c @@ -605,9 +605,9 @@ static enum sb_pin_state pin_sb_for_writ * Return 1, if the caller writeback routine should be * interrupted. Otherwise return 0. */ -static int writeback_sb_inodes(struct super_block *sb, - struct bdi_writeback *wb, - struct writeback_control *wbc) +int generic_writeback_sb_inodes(struct super_block *sb, + struct bdi_writeback *wb, + struct writeback_control *wbc) { while (!list_empty(&wb->b_io)) { long pages_skipped; @@ -658,6 +658,32 @@ static int writeback_sb_inodes(struct su /* b_io is empty */ return 1; } +EXPORT_SYMBOL(generic_writeback_sb_inodes); + +/* + * This function is for file systems which have their + * own means of periodical write-out of old data. + * NOTE: inode_lock should be hold. + * + * Skip a portion of b_io inodes which belong to @sb + * and go sequentially in reverse order. + */ +void writeback_skip_sb_inodes(struct super_block *sb, + struct bdi_writeback *wb) +{ + while (1) { + struct inode *inode; + + if (list_empty(&wb->b_io)) + break; + inode = list_entry(wb->b_io.prev, struct inode, i_list); + if (sb != inode->i_sb) + break; + redirty_tail(inode); + } +} +EXPORT_SYMBOL(writeback_skip_sb_inodes); + static void writeback_inodes_wb(struct bdi_writeback *wb, struct writeback_control *wbc) @@ -687,7 +713,10 @@ static void writeback_inodes_wb(struct b requeue_io(inode); continue; } - ret = writeback_sb_inodes(sb, wb, wbc); + if (sb->s_op->writeback_inodes) + ret = sb->s_op->writeback_inodes(sb, wb, wbc); + else + ret = generic_writeback_sb_inodes(sb, wb, wbc); if (state == SB_PINNED) unpin_sb_for_writeback(sb); @@ -704,6 +733,7 @@ void writeback_inodes_wbc(struct writeba writeback_inodes_wb(&bdi->wb, wbc); } +EXPORT_SYMBOL(writeback_inodes_wbc); /* * The maximum number of pages to writeout in a single bdi flush/kupdate @@ -1289,3 +1319,12 @@ int sync_inode(struct inode *inode, stru return ret; } EXPORT_SYMBOL(sync_inode); +/* + * Local variables: + * c-indentation-style: "K&R" + * mode-name: "LC" + * c-basic-offset: 8 + * tab-width: 8 + * fill-column: 79 + * End: + */ From morfic@gmail.com Mon Feb 1 21:24:19 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: **** X-Spam-Status: No, score=4.6 required=5.0 tests=BAYES_00,DEAR_SOMETHING, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o123NxmH150879 for ; Mon, 1 Feb 2010 21:24:09 -0600 X-ASG-Debug-ID: 1265081074-254103700000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ww0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B458A1A3842 for ; Mon, 1 Feb 2010 19:24:34 -0800 (PST) Received: from mail-ww0-f53.google.com (mail-ww0-f53.google.com [74.125.82.53]) by cuda.sgi.com with ESMTP id 66fyaAi4vqPORNeu for ; Mon, 01 Feb 2010 19:24:34 -0800 (PST) Received: by wwc33 with SMTP id 33so173485wwc.26 for ; Mon, 01 Feb 2010 19:24:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=m/Y3XP0EIKPDgUU7IGuzDSnx6QRhYRVOHRpbS8iYuvI=; b=aWeeW12BxF6MZRYoypEL1j9Tho/9Qwd8US9C6rZ/L3yr+PuN541goXEzDvN5YTDt4z 7ImjhJSvbJcsUm5ACCTe7CTXXYix7QvuP1mlSYOy7NCRrEJ59Sfq1Y5NZAvW3PMyL7EQ 6egcslASq8roZOwWvevmu62Fl1mwpgZPmzkg8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=oVWPXyHiM25YZF2rrudolOmGO1ILwCth2HCCo+ggIKfOgHhx1NWjbN6e0nsvQ+xhe1 f7hJordwJI9h97o7gLaTlFvE+ZPPQdNh17i7XTw21F2Q98m7Fvn3uJJSzc7PP4+Zaymg HFWCwP7XR1zVqi5BDFRd2xPZ+RHSHDXuJzlFw= MIME-Version: 1.0 Received: by 10.216.87.204 with SMTP id y54mr1869075wee.164.1265081073100; Mon, 01 Feb 2010 19:24:33 -0800 (PST) Date: Mon, 1 Feb 2010 21:24:33 -0600 Message-ID: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> X-ASG-Orig-Subj: State of XFS on ARM Subject: State of XFS on ARM From: Daniel Goller To: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: mail-ww0-f53.google.com[74.125.82.53] X-Barracuda-Start-Time: 1265081098 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21419 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Dear Sir or Madam, I Would like to find out about the current state of XFS on ARM. I have been using XFS for a while successfully on my laptop, and was using XFS on a ARM device (little endian) since it would hold the same data i had on the laptop. It seems i have not been able to unmount it and then mounting it cleanly once, always required xfs_repair -L /dev/sdc3 I Could understand power issues or lockups causing this, but on clean umount and followed mount to see it fail is surprising. When mounting fails on the headless arm machine i move the drive to a x86_64 and run xfs_repair there when mounting there fails too (so log can't be replayed, making -L necessary). All of this leads me to ask: "Is XFS as well maintained on ARM as it is on x86/x86_64?" Thank you in advance for any info you can provide, Daniel Goller From edward.shishkin@gmail.com Mon Feb 1 21:33:01 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.1 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o123WfEm151097 for ; Mon, 1 Feb 2010 21:32:51 -0600 X-ASG-Debug-ID: 1265081599-620602d10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id ABF1A1C97ADE for ; Mon, 1 Feb 2010 19:33:20 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id mH24HTzrbIMnKiVN for ; Mon, 01 Feb 2010 19:33:20 -0800 (PST) Received: by ewy27 with SMTP id 27so955584ewy.18 for ; Mon, 01 Feb 2010 19:33:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:mime-version:message-id:cc:content-type :content-transfer-encoding; bh=WvTl84i6AssHSnJeG9IiC9Zb7FQlV2PJhaSSdLfavFg=; b=Um3YKDDQTTWMuGEmfb67eythFwLQWZ0KlSzhyZIrfQDeMU2uDKA72a6jP46I9NQEG6 IUnoDzyfybpFiUyG4eWwN+wD/PDpKgg45iXMQhauHwbPIQKd5Vah54oZx6t9fxj72rz1 GtJ6+4Q4voGb8q6FKbGM2YgCCPJR7T3Pc59y0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:message-id:cc :content-type:content-transfer-encoding; b=VxM+xISKAEtQZgcWfOOCu4cjm2RkcfHcGd5vSK9jQiQmMnrX3fTiAqhlNkVPcwItZ3 /qQFmSX72GX+jWK2hlrzkRZP2/4FXhokCbZmAs8Zw4kpu1Ir8js/dMVuAl7oPFSy/Ca7 m+tvWQpcjB1ytwx58z9Cp1DYSDhxkvXOYpBhw= Received: by 10.213.97.85 with SMTP id k21mr29869ebn.79.1265075739276; Mon, 01 Feb 2010 17:55:39 -0800 (PST) Received: from zeta.englab.brq.redhat.com (nat-pool-brq-t.redhat.com [209.132.186.34]) by mx.google.com with ESMTPS id 14sm4125601ewy.3.2010.02.01.17.55.38 (version=SSLv3 cipher=RC4-MD5); Mon, 01 Feb 2010 17:55:38 -0800 (PST) From: Edward Shishkin To: Andrew Morton , ReiserFS Development List X-ASG-Orig-Subj: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: [patch 0/7] per-bdi flushing model improvements. reiser4 Date: Tue, 2 Feb 2010 02:55:46 +0100 User-Agent: KMail/1.12.3 (Linux/2.6.27.41-170.2.117.fc10.i686; KDE/4.3.3; i686; ; ) MIME-Version: 1.0 Message-Id: <201002020255.46920.edward.shishkin@gmail.com> Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265081607 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21419 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello. Andrew Morton wrote: > reiser4 is currently disabled in -mm (via reiser4-disable.patch) > because recent changes to fs/fs/writeback.c wrecked the build. I fixed > it about ten times as the underlying code was churning, then gave up. It > would be nice if you take a look at that sometime please. > > I have taken a look at fs/fs-writeback.c and found that per-superblock flushing interface is eliminated. However migrating to per-bdi flushing model doesn't necessarily means that such interface doesn't exist or is not needed anymore. Flushing in accordance with the scheme "data-inode- data-inode-..." would be very suboptimal for reiser4. Also xfs people were unhappy with such flushing model: http://article.gmane.org/gmane.linux.file-systems/30153 Moreover, current stuff doesn't look fine. Why do we pin/unpin superblock for every inode? It would be more reasonable to pin it for the whole group of inodes and call a flushing handler for them. The patch 4 introduces such handler writeback_sb_inodes (which resembles dropped sync_sb_inodes, the difference is that the newer version doesn't flush necessarily all inodes of the superblock). Please, consider pushing this patch to mainline. The patch 5 adds a super operation .writeback_inodes (former .sync_inodes) which allows a file system to make optimizations. It can happen that reiser4 will flush a bit more inodes then generic implementation suggests. "a bit more" doesn't mean "all dirty inodes of the superblock" (see a comment about atoms in the header of patch 6). Finally, some file systems have its own means for periodical writeout of dirty data. Since b_io contains inodes of many superblocks we need to evict our inodes back to dirty list when flushing is going on with for_kupdate flag installed. The new library function writeback_skip_sb_inodes() provides such possibility. Please, apply. Thanks, Edward. From sandeen@sandeen.net Mon Feb 1 21:51:39 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: ** X-Spam-Status: No, score=2.7 required=5.0 tests=AWL,BAYES_00,DEAR_SOMETHING, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o123pc7R151877 for ; Mon, 1 Feb 2010 21:51:39 -0600 X-ASG-Debug-ID: 1265082764-5e4803400000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 62ED71C97B46 for ; Mon, 1 Feb 2010 19:52:44 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id kGrsNSW9m32gcf2o for ; Mon, 01 Feb 2010 19:52:44 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 5C2E2917EC6; Mon, 1 Feb 2010 21:52:44 -0600 (CST) Message-ID: <4B67A18C.8010009@sandeen.net> Date: Mon, 01 Feb 2010 21:52:44 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Daniel Goller CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> In-Reply-To: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265082765 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21421 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Daniel Goller wrote: > Dear Sir or Madam, > > I Would like to find out about the current state of XFS on ARM. > I have been using XFS for a while successfully on my laptop, and was > using XFS on a ARM device (little endian) since it would hold the same > data i had on the laptop. > It seems i have not been able to unmount it and then mounting it > cleanly once, always required xfs_repair -L /dev/sdc3 > I Could understand power issues or lockups causing this, but on clean > umount and followed mount to see it fail is surprising. well, actually neither power issues nor lockups should cause it either ;) > When mounting fails on the headless arm machine i move the drive to a > x86_64 and run xfs_repair there when mounting there fails too (so log > can't be replayed, making -L necessary). > All of this leads me to ask: "Is XFS as well maintained on ARM as it > is on x86/x86_64?" Short answer no, but effort is made. The last known issue, as far as I know, is a cache aliasing problem. This patch is a big-hammer approach, better things have been proposed but not yet upstream as far as I know. With "what doesn't work" helpfully commented out ;) Index: linux-2.6.22.18/fs/xfs/linux-2.6/xfs_buf.c =================================================================== --- linux-2.6.22.18.orig/fs/xfs/linux-2.6/xfs_buf.c +++ linux-2.6.22.18/fs/xfs/linux-2.6/xfs_buf.c @@ -1185,6 +1185,8 @@ _xfs_buf_ioapply( bio->bi_end_io = xfs_buf_bio_end_io; bio->bi_private = bp; + //flush_dcache_page(bp->b_pages[0]); + flush_cache_all(); bio_add_page(bio, bp->b_pages[0], PAGE_CACHE_SIZE, 0); size = 0; @@ -1211,6 +1213,8 @@ next_chunk: if (nbytes > size) nbytes = size; + //flush_dcache_page(bp->b_pages[map_i]); + flush_cache_all(); rbytes = bio_add_page(bio, bp->b_pages[map_i], nbytes, offset); if (rbytes < nbytes) break; > Thank you in advance for any info you can provide, > > Daniel Goller > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > From BATV+d426e2b78c7c125350ad+2354+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 2 02:16:23 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o128GMDc162512 for ; Tue, 2 Feb 2010 02:16:23 -0600 X-ASG-Debug-ID: 1265098650-636601e70000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B7941136D56C for ; Tue, 2 Feb 2010 00:17:30 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 1WKjkXNK3cG35nfn for ; Tue, 02 Feb 2010 00:17:30 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcDwu-0000dS-D7; Tue, 02 Feb 2010 08:17:28 +0000 Date: Tue, 2 Feb 2010 03:17:28 -0500 From: Christoph Hellwig To: Edward Shishkin Cc: Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Message-ID: <20100202081728.GA2384@infradead.org> References: <201002020255.46920.edward.shishkin@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201002020255.46920.edward.shishkin@gmail.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265098651 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean I got this introduction twice, but patches 1-3 didn't make it to any of the lists. From BATV+d426e2b78c7c125350ad+2354+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 2 05:21:56 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12BLrEm168669 for ; Tue, 2 Feb 2010 05:21:56 -0600 X-ASG-Debug-ID: 1265109782-0f36003b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 861F7136DC3A for ; Tue, 2 Feb 2010 03:23:02 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id Ro98E6BXUKfx6ANR for ; Tue, 02 Feb 2010 03:23:02 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcGqS-0006T9-B2; Tue, 02 Feb 2010 11:23:00 +0000 Date: Tue, 2 Feb 2010 06:23:00 -0500 From: Christoph Hellwig To: Daniel Goller , James.Bottomley@HansenPartnership.com Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM Message-ID: <20100202112300.GA23809@infradead.org> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265109782 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Feb 01, 2010 at 09:24:33PM -0600, Daniel Goller wrote: > It seems i have not been able to unmount it and then mounting it > cleanly once, always required xfs_repair -L /dev/sdc3 > I Could understand power issues or lockups causing this, but on clean > umount and followed mount to see it fail is surprising. > When mounting fails on the headless arm machine i move the drive to a > x86_64 and run xfs_repair there when mounting there fails too (so log > can't be replayed, making -L necessary). > All of this leads me to ask: "Is XFS as well maintained on ARM as it > is on x86/x86_64?" XFS itself is platform neutral. But it seems like you have an ARM platform with virtually indexed caches, which currently can't support the I/O XFS does. James has been working on fixing this for a while, but so far the architecture support patches for it unfortunately still haven't made it to mainline despite many people running into this issue. The git tree with the current versions of the patches to fix this is here: http://git.kernel.org/?p=linux/kernel/git/jejb/xfs-vipt/.git;a=summary From ebb9@byu.net Tue Feb 2 08:01:31 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12E1UaT174551 for ; Tue, 2 Feb 2010 08:01:31 -0600 X-ASG-Debug-ID: 1265119358-7e2402d60000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from qmta01.emeryville.ca.mail.comcast.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5826D1C98FB6 for ; Tue, 2 Feb 2010 06:02:38 -0800 (PST) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [76.96.30.16]) by cuda.sgi.com with ESMTP id sjQmJPVc7rTwjUwK for ; Tue, 02 Feb 2010 06:02:38 -0800 (PST) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta01.emeryville.ca.mail.comcast.net with comcast id d1K41d0090FhH24A122fhr; Tue, 02 Feb 2010 14:02:39 +0000 Received: from [192.168.0.5] ([24.10.247.83]) by omta08.emeryville.ca.mail.comcast.net with comcast id d22d1d0021ohlF48U22dQ3; Tue, 02 Feb 2010 14:02:39 +0000 Message-ID: <4B683082.3040803@byu.net> Date: Tue, 02 Feb 2010 07:02:42 -0700 From: Eric Blake User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.23) Gecko/20090812 Thunderbird/2.0.0.23 Mnenhy/0.7.6.666 MIME-Version: 1.0 To: Eric Sandeen CC: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] xfstests: add test for ctimes updates in futimens Subject: Re: [PATCH] xfstests: add test for ctimes updates in futimens References: <20100110215227.GA15354@infradead.org> <4B61FAA0.8020304@sandeen.net> In-Reply-To: <4B61FAA0.8020304@sandeen.net> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: qmta01.emeryville.ca.mail.comcast.net[76.96.30.16] X-Barracuda-Start-Time: 1265119359 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21459 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean According to Eric Sandeen on 1/28/2010 1:59 PM: > Late on review, but this won't build on older glibc's :( > > need some auto-fu somewhere? was going to put: > > +# > +# Check if we have a working futimens libc call > +# > +AC_DEFUN([AC_HAVE_FUTIMENS], > + [ AC_MSG_CHECKING([for futimens ]) > + AC_TRY_COMPILE([ > +#define _GNU_SOURCE > +#include > + ], [ > + struct timespec t[2] = { { 1000000000, 0 }, { 0, UTIME_OMIT } }; > + futimens(0, t); > + ], have_futimens=yes > + AC_MSG_RESULT(yes), > + AC_MSG_RESULT(no)) > + AC_SUBST(have_futimens) > + ]) You are missing some m4 quoting, and are using deprecated macros; also, caching the value speeds up subsequent configure runs. Furthermore, you are impinging on the autoconf macro namespace, which is discouraged (what if autoconf later introduces AC_HAVE_FUTIMENS with different semantics than your macro?). Then, since you are only checking for compilation, you can avoid messing with struct timespec. AC_SUBST is useful for makefiles, but AC_DEFINE is useful for preprocessor macros; I'm assuming you wanted the latter? So, assuming that you are using xfs_ as your package-specific namespace, I would write this as: AC_DEFUN([XFS_HAVE_FUTIMENS], [ AC_CACHE_CHECK([for futimens], [xfs_cv_have_futimens], [ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ #define _GNU_SOURCE #include ]], [[futimens (0, 0); ]])], [xfs_cv_have_futimens=yes], [xfs_cv_have_futimens=no]) ]) if test "x$xfs_cv_have_futimens" = xyes; then AC_DEFINE([HAVE_FUTIMENS], [1], [Define to 1 if futimens compiles]) fi ]) -- Don't work too hard, make some time for fun as well! Eric Blake ebb9@byu.net From edward.shishkin@gmail.com Tue Feb 2 09:25:04 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12FP31O177225 for ; Tue, 2 Feb 2010 09:25:04 -0600 X-ASG-Debug-ID: 1265124371-63e0006e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 15EC5136EEC5 for ; Tue, 2 Feb 2010 07:26:11 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id DhTjhL5sr6fYoHcq for ; Tue, 02 Feb 2010 07:26:11 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o12FPLou014330 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 2 Feb 2010 10:25:22 -0500 Received: from zeta.englab.brq.redhat.com (dhcp-lab-156.englab.brq.redhat.com [10.34.33.156]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o12FPJ8a030241; Tue, 2 Feb 2010 10:25:20 -0500 Message-ID: <4B6843E9.9070600@gmail.com> Date: Tue, 02 Feb 2010 16:25:29 +0100 From: Edward Shishkin User-Agent: Thunderbird 2.0.0.23 (X11/20090825) MIME-Version: 1.0 To: Christoph Hellwig CC: Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, jens.axboe@oracle.com X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 References: <201002020255.46920.edward.shishkin@gmail.com> <20100202081728.GA2384@infradead.org> In-Reply-To: <20100202081728.GA2384@infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265124372 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21464 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Christoph Hellwig wrote: > I got this introduction twice, but patches 1-3 didn't make it to any of > the lists. > > > done From realrichardsharpe@gmail.com Tue Feb 2 09:41:05 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12Ff5gZ177797 for ; Tue, 2 Feb 2010 09:41:05 -0600 X-ASG-Debug-ID: 1265125332-485001450000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-bw0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1E27A1A5DC5 for ; Tue, 2 Feb 2010 07:42:13 -0800 (PST) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by cuda.sgi.com with ESMTP id OOxtF6MYT45UoZbk for ; Tue, 02 Feb 2010 07:42:13 -0800 (PST) Received: by bwz27 with SMTP id 27so186411bwz.19 for ; Tue, 02 Feb 2010 07:42:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=zUbvrxwoetXw1avJyC8+qhoCZd6wfcjOX4O6PDDds24=; b=lx5vbisnVHQs6cRb6e/6yHkif2cAZyH4WI08H8yZGizQ3Y/IMle0EIXEM200v1iYrr 8adOolzLS3WXE8+vqSQ0WvbnkQD8BQL4PAxZeJU01acqUFif/DQaVQj62yomOQT3jpxJ sb8qVEqmd/vR1FZg/7D08WSE2Py6shI9berws= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=n72QJufJLausX446zEj5xpZJ03L2TUjOQzfUPhPBTGM66KIx+9CBiQs4AmLTEkKyrv QlAdUc2eET0ubZspIxEBo0ZOvk3scjhtYoOgdQAnG2Q/3pSXPKK8QHG2ORSboX8nEJzU +qFQ6nRQY64TmaH/w+Eij/UPV+QMj5qu7O3Xw= MIME-Version: 1.0 Received: by 10.204.137.215 with SMTP id x23mr4581952bkt.151.1265125331430; Tue, 02 Feb 2010 07:42:11 -0800 (PST) In-Reply-To: <20100202112300.GA23809@infradead.org> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> Date: Tue, 2 Feb 2010 07:42:11 -0800 Message-ID: <46b8a8851002020742w28891d18g24d01982631cdf2b@mail.gmail.com> X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM From: Richard Sharpe To: Christoph Hellwig Cc: Daniel Goller , James.Bottomley@hansenpartnership.com, xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-bw0-f227.google.com[209.85.218.227] X-Barracuda-Start-Time: 1265125334 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21465 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 2, 2010 at 3:23 AM, Christoph Hellwig wrote= : > On Mon, Feb 01, 2010 at 09:24:33PM -0600, Daniel Goller wrote: >> It seems i have not been able to unmount it and then mounting it >> cleanly once, always required xfs_repair -L =A0 /dev/sdc3 >> I Could understand power issues or lockups causing this, but on clean >> umount and followed mount to see it fail is surprising. >> When mounting fails on the headless arm machine i move the drive to a >> x86_64 and run xfs_repair there when mounting there fails too (so log >> can't be replayed, making -L necessary). >> All of this leads me to ask: =A0"Is XFS as well maintained on ARM as it >> is on x86/x86_64?" > > XFS itself is platform neutral. =A0But it seems like you have an ARM > platform with virtually indexed caches, which currently can't support > the I/O XFS does. =A0James has been working on fixing this for a while, > but so far the architecture support patches for it unfortunately still > haven't made it to mainline despite many people running into this issue. I would be interested in a precise explanation for this problem. We ran into an issue here with XFS on the Marvell 78200 (ARM) which has a VIVT cache. We eventually solved the problem by using flush_icache_range in a strategic place (after also discovering the need for flush_dcache_page, but that has to do with doing IO between the cores on the 78200), but we really did not understand why that fixed the problem. However, it stopped the corruption we were experiencing when using a file system exerciser. (We have subsequently stopped using XFS, but not because of any corruption issues.) --=20 Regards, Richard Sharpe From James.Bottomley@HansenPartnership.com Tue Feb 2 10:11:15 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12GBE55178783 for ; Tue, 2 Feb 2010 10:11:14 -0600 X-ASG-Debug-ID: 1265127142-5a8502880000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bedivere.hansenpartnership.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EFD591DB8174 for ; Tue, 2 Feb 2010 08:12:22 -0800 (PST) Received: from bedivere.hansenpartnership.com (bedivere.hansenpartnership.com [66.63.167.143]) by cuda.sgi.com with ESMTP id yyTlTwX952cbwvql for ; Tue, 02 Feb 2010 08:12:22 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by bedivere.hansenpartnership.com (Postfix) with ESMTP id 1F00C8EE0F2; Tue, 2 Feb 2010 08:12:22 -0800 (PST) Received: from bedivere.hansenpartnership.com ([127.0.0.1]) by localhost (bedivere.hansenpartnership.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id inFNtFy4T8KV; Tue, 2 Feb 2010 08:12:22 -0800 (PST) Received: from [192.168.10.224] (newmulgrave.ext.hansenpartnership.com [192.168.10.224]) by bedivere.hansenpartnership.com (Postfix) with ESMTP id 24F608EE0D2; Tue, 2 Feb 2010 08:12:19 -0800 (PST) X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM From: James Bottomley To: Richard Sharpe Cc: Christoph Hellwig , Daniel Goller , xfs@oss.sgi.com In-Reply-To: <46b8a8851002020742w28891d18g24d01982631cdf2b@mail.gmail.com> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> <46b8a8851002020742w28891d18g24d01982631cdf2b@mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Date: Tue, 02 Feb 2010 10:12:17 -0600 Message-Id: <1265127137.2800.67.camel@mulgrave.site> Mime-Version: 1.0 X-Mailer: Evolution 2.28.0 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: bedivere.hansenpartnership.com[66.63.167.143] X-Barracuda-Start-Time: 1265127142 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21467 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, 2010-02-02 at 07:42 -0800, Richard Sharpe wrote: > On Tue, Feb 2, 2010 at 3:23 AM, Christoph Hellwig wrote: > > On Mon, Feb 01, 2010 at 09:24:33PM -0600, Daniel Goller wrote: > >> It seems i have not been able to unmount it and then mounting it > >> cleanly once, always required xfs_repair -L /dev/sdc3 > >> I Could understand power issues or lockups causing this, but on clean > >> umount and followed mount to see it fail is surprising. > >> When mounting fails on the headless arm machine i move the drive to a > >> x86_64 and run xfs_repair there when mounting there fails too (so log > >> can't be replayed, making -L necessary). > >> All of this leads me to ask: "Is XFS as well maintained on ARM as it > >> is on x86/x86_64?" > > > > XFS itself is platform neutral. But it seems like you have an ARM > > platform with virtually indexed caches, which currently can't support > > the I/O XFS does. James has been working on fixing this for a while, > > but so far the architecture support patches for it unfortunately still > > haven't made it to mainline despite many people running into this issue. > > I would be interested in a precise explanation for this problem. We > ran into an issue here with XFS on the Marvell 78200 (ARM) which has a > VIVT cache. We eventually solved the problem by using > flush_icache_range in a strategic place (after also discovering the > need for flush_dcache_page, but that has to do with doing IO between > the cores on the 78200), but we really did not understand why that > fixed the problem. However, it stopped the corruption we were > experiencing when using a file system exerciser. XFS uses large buffers for log operations. Because large physically contiguous pieces of memory are hard to come by, it makes its logs from virtually contiguous pages. The problem is that doing this sets up aliasing within the kernel: each page in the range now has two virtual addresses: one for the standard kernel offset map and the other for the xfs created virtual area. When I/O is done on a kernel page, the kernel always flushes the offset map to ensure that the data is in main memory. However, when the page has another alias in the xfs created virtual area, the kernel I/O subsystem has no idea about this. Since xfs is reading and writing to this other alias, that's what needs to be flushed to make sure the data is safely in main memory before I/O takes place. James From axboe@kernel.dk Tue Feb 2 13:41:02 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12Jf1qU188198 for ; Tue, 2 Feb 2010 13:41:02 -0600 X-ASG-Debug-ID: 1265139729-1c9800df0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from kernel.dk (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9138D1A59D4 for ; Tue, 2 Feb 2010 11:42:09 -0800 (PST) Received: from kernel.dk (0122700014.0.fullrate.dk [95.166.99.235]) by cuda.sgi.com with ESMTP id E0qgPPya4BEYcUjs for ; Tue, 02 Feb 2010 11:42:09 -0800 (PST) Received: by kernel.dk (Postfix, from userid 1000) id 3991E37A0B3; Tue, 2 Feb 2010 20:42:08 +0100 (CET) Date: Tue, 2 Feb 2010 20:42:08 +0100 From: Jens Axboe To: Edward Shishkin Cc: Christoph Hellwig , Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Message-ID: <20100202194208.GD5733@kernel.dk> References: <201002020255.46920.edward.shishkin@gmail.com> <20100202081728.GA2384@infradead.org> <4B6843E9.9070600@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B6843E9.9070600@gmail.com> X-Barracuda-Connect: 0122700014.0.fullrate.dk[95.166.99.235] X-Barracuda-Start-Time: 1265139730 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21480 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 02 2010, Edward Shishkin wrote: > Christoph Hellwig wrote: >> I got this introduction twice, but patches 1-3 didn't make it to any of >> the lists. >> >> >> > done Where? -- Jens Axboe From theronni@gmail.com Tue Feb 2 15:44:16 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.7 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX, HTML_MESSAGE,J_CHICKENPOX_45 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12LiGZo193832 for ; Tue, 2 Feb 2010 15:44:16 -0600 X-ASG-Debug-ID: 1265147123-42f201070000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f211.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D0BC11C9ADFB for ; Tue, 2 Feb 2010 13:45:23 -0800 (PST) Received: from mail-fx0-f211.google.com (mail-fx0-f211.google.com [209.85.220.211]) by cuda.sgi.com with ESMTP id jXaVeH2vAm5xDHr3 for ; Tue, 02 Feb 2010 13:45:23 -0800 (PST) Received: by fxm3 with SMTP id 3so612129fxm.19 for ; Tue, 02 Feb 2010 13:45:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=Vf3bPjAm7kpKCih6ILFvGpzDAEQqbMVbO+WOCgWnU1g=; b=VgSz7lwwS96MwTwxQ4rWgUi46W4f3uEp7YYen/9gFziCLz+IkJ+bsA3xOvbORkomJ1 hGnWrGmsENeXSRZyYemKlJuwb7H479EBYPsU78flDIAGuLf0KtQ4NjYBe4W8S1vnmnKo fUTJkCY6bglkSp2Dsu+PlqDigDF95EQaeblLw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=XBbHtHaSR4fHDnQyBgT++kZ0TLuzV1b4dzQN410R2LsaUfzH004nuyW/UOsDjCL8PW ENZt4K7Mor88L/3eZYq4sb+XLs8tAHEJ25G1AwPu6W6R7HcrI22Mx9XdgdkLlMfYkM+E ZnWr3lO68dEORwIz96FVwSH1wt9squORIoZ68= MIME-Version: 1.0 Received: by 10.223.2.149 with SMTP id 21mr5077891faj.49.1265147122448; Tue, 02 Feb 2010 13:45:22 -0800 (PST) In-Reply-To: <20100202194208.GD5733@kernel.dk> References: <201002020255.46920.edward.shishkin@gmail.com> <20100202081728.GA2384@infradead.org> <4B6843E9.9070600@gmail.com> <20100202194208.GD5733@kernel.dk> Date: Wed, 3 Feb 2010 01:45:22 +0400 Message-ID: <5e2ec5ac1002021345g569c743dtc5ccf77cde6738ae@mail.gmail.com> X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 From: Ronni Holm-Nielsen To: Jens Axboe Cc: Edward Shishkin , Christoph Hellwig , Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com Content-Type: multipart/alternative; boundary=000e0ce0f19ad27bf7047ea504db X-Barracuda-Connect: mail-fx0-f211.google.com[209.85.220.211] X-Barracuda-Start-Time: 1265147124 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21488 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.00 HTML_MESSAGE BODY: HTML included in message X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean --000e0ce0f19ad27bf7047ea504db Content-Type: text/plain; charset=UTF-8 To clarify (being a ReiserFS subscriber): patch 0, 4-6 sent to Andrew, ReiserFS, linux-fsdevel, linux-kernel, xfs, jens.axboe patch 1-3, 7 sent to Andrew, ReiserFS - Ronni On Tue, Feb 2, 2010 at 11:42 PM, Jens Axboe wrote: > On Tue, Feb 02 2010, Edward Shishkin wrote: > > Christoph Hellwig wrote: > >> I got this introduction twice, but patches 1-3 didn't make it to any of > >> the lists. > >> > >> > >> > > done > > Where? > > -- > Jens Axboe > > -- > To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" > in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Venlig hilsen Ronni --000e0ce0f19ad27bf7047ea504db Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
To clarify (being a ReiserFS subscriber):

patch 0,= 4-6 sent to=C2=A0Andrew, ReiserFS, linux-fsdevel, linux-kernel, xfs, jens.= axboe
patch 1-3, 7 sent to=C2=A0Andrew, ReiserFS

- Ronni

On Tue, Feb 2, 2010 at 11:42 PM, Jens Axboe = <jens.axboe@o= racle.com> wrote:
On Tue, Feb 02 2010, Edward Shishkin wrote:
> Christoph Hellwig wrote:
>> I got this introduction twice, but patches 1-3 didn't make it = to any of
>> the lists.
>>
>>
>>
> done

Where?

--
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe reiserfs-dev= el" in
the body of a message to major= domo@vger.kernel.org
More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.html



--
Venlig hils= en
Ronni
--000e0ce0f19ad27bf7047ea504db-- From theronni@gmail.com Tue Feb 2 15:46:43 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_45 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12LkhDV194004 for ; Tue, 2 Feb 2010 15:46:43 -0600 X-ASG-Debug-ID: 1265147270-75cf01030000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-fx0-f211.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9A25E1A639A for ; Tue, 2 Feb 2010 13:47:51 -0800 (PST) Received: from mail-fx0-f211.google.com (mail-fx0-f211.google.com [209.85.220.211]) by cuda.sgi.com with ESMTP id rvdtzLx5K74T8gZC for ; Tue, 02 Feb 2010 13:47:51 -0800 (PST) Received: by fxm3 with SMTP id 3so614929fxm.19 for ; Tue, 02 Feb 2010 13:47:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=T/6pg1oYMwT8yInpvoYc6mW03stgNfH5H2hcbFRU1cQ=; b=YDITFTYipYYx3fo/Mm7SdcFF6tPD7RprRBfalk5ISBYlk5xz2xWcyk+3PdF9MRp5O9 5IaQVoFmeS0XDbmzrM4xaKXAIauhqzsL1tEsuWY91ThFWhDXd8XH+KyqNl+Fu6hRSoAw Icr8sRLv1UZwhGszpKZLkzNxO42dRkhkVHSjc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=kFUvz/1tfff6nIT+34QLrdFrJGUAASOV62EpgCzpomRVtCa+c41yHommpdbc1FNDjc xZ/CWaRYDqtNNnLDm5+uzXxBnmXGFAN37PH0KMjzjLOUYBBnK8iV5pRSfVeVg0JdgRih vYIekemFAmyWPvVqEtV1C0+4h1OPM/SFxGayU= MIME-Version: 1.0 Received: by 10.223.95.74 with SMTP id c10mr6748278fan.82.1265147270572; Tue, 02 Feb 2010 13:47:50 -0800 (PST) In-Reply-To: <20100202194208.GD5733@kernel.dk> References: <201002020255.46920.edward.shishkin@gmail.com> <20100202081728.GA2384@infradead.org> <4B6843E9.9070600@gmail.com> <20100202194208.GD5733@kernel.dk> Date: Wed, 3 Feb 2010 01:47:50 +0400 Message-ID: <5e2ec5ac1002021347k3a8cd8adta5ff76efe1eeaadb@mail.gmail.com> X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 From: Ronni Holm-Nielsen To: Jens Axboe Cc: Edward Shishkin , Christoph Hellwig , Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-fx0-f211.google.com[209.85.220.211] X-Barracuda-Start-Time: 1265147272 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21488 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 2, 2010 at 11:42 PM, Jens Axboe wrote: > > On Tue, Feb 02 2010, Edward Shishkin wrote: > > Christoph Hellwig wrote: > >> I got this introduction twice, but patches 1-3 didn't make it to any o= f > >> the lists. > Where? To clarify (being a ReiserFS subscriber): patch 0, 4-6 sent to=C2=A0Andrew, ReiserFS, linux-fsdevel, linux-kernel, xfs, jens.axboe patch 1-3, 7 sent to=C2=A0Andrew, ReiserFS - Ronni From edward.shishkin@gmail.com Tue Feb 2 16:25:56 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=1.0 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_45 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12MPt7P195633 for ; Tue, 2 Feb 2010 16:25:56 -0600 X-ASG-Debug-ID: 1265149624-6aa100c60000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B54B51C9AF24 for ; Tue, 2 Feb 2010 14:27:04 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 1XUdq4pWXOeklnUm for ; Tue, 02 Feb 2010 14:27:04 -0800 (PST) Received: from int-mx04.intmail.prod.int.phx2.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.17]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o12MQDJX031690 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 2 Feb 2010 17:26:14 -0500 Received: from zeta.englab.brq.redhat.com (dhcp-lab-156.englab.brq.redhat.com [10.34.33.156]) by int-mx04.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o12MQArR026159; Tue, 2 Feb 2010 17:26:11 -0500 Message-ID: <4B68A68C.6030001@gmail.com> Date: Tue, 02 Feb 2010 23:26:20 +0100 From: Edward Shishkin User-Agent: Thunderbird 2.0.0.23 (X11/20090825) MIME-Version: 1.0 To: Ronni Holm-Nielsen CC: Jens Axboe , Christoph Hellwig , Andrew Morton , ReiserFS Development List , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com, Johannes Buchner , Artem Bityutskiy X-ASG-Orig-Subj: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 Subject: Re: [patch 0/7] per-bdi flushing model improvements. reiser4 References: <201002020255.46920.edward.shishkin@gmail.com> <20100202081728.GA2384@infradead.org> <4B6843E9.9070600@gmail.com> <20100202194208.GD5733@kernel.dk> <5e2ec5ac1002021345g569c743dtc5ccf77cde6738ae@mail.gmail.com> In-Reply-To: <5e2ec5ac1002021345g569c743dtc5ccf77cde6738ae@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.17 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265149624 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.01 X-Barracuda-Spam-Status: No, SCORE=-2.01 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_SC0_SA_TO_FROM_DOMAIN_MATCH X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21492 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient Domain X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello everyone. The patches 1-3 are reverses for the following -mm stuff: http://userweb.kernel.org/~akpm/mmotm/broken-out/reiser4-fixed-null-pointer-dereference.patch http://userweb.kernel.org/~akpm/mmotm/broken-out/reiser4-generic_sync_sb_inodes-doesnt-exist-anymore.patch http://userweb.kernel.org/~akpm/mmotm/broken-out/reiser4-vfs-add-super_operationssync_inodes-2.patch This is incorrect attempts to adjust reiser4 to the new per-bdi flushing model. The authors are cc-ed, any comments, suggestions are welcome. Thanks, Edward. Ronni Holm-Nielsen wrote: > To clarify (being a ReiserFS subscriber): > > patch 0, 4-6 sent to Andrew, ReiserFS, linux-fsdevel, linux-kernel, xfs, > jens.axboe > patch 1-3, 7 sent to Andrew, ReiserFS > > - Ronni > > On Tue, Feb 2, 2010 at 11:42 PM, Jens Axboe wrote: > > >> On Tue, Feb 02 2010, Edward Shishkin wrote: >> >>> Christoph Hellwig wrote: >>> >>>> I got this introduction twice, but patches 1-3 didn't make it to any of >>>> the lists. >>>> >>>> >>>> >>>> >>> done >>> >> Where? >> >> -- >> Jens Axboe >> >> -- >> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" >> in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> > > > > From sandeen@sandeen.net Tue Feb 2 17:05:40 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12N5dL6197575 for ; Tue, 2 Feb 2010 17:05:40 -0600 X-ASG-Debug-ID: 1265152007-2df503740000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 96A5913706F4 for ; Tue, 2 Feb 2010 15:06:48 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id OLdKQtKIpdWIJOx5 for ; Tue, 02 Feb 2010 15:06:48 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 543EB109E858 for ; Tue, 2 Feb 2010 17:06:47 -0600 (CST) Message-ID: <4B68B007.1020705@sandeen.net> Date: Tue, 02 Feb 2010 17:06:47 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss X-ASG-Orig-Subj: [PATCH V3] xfstests: filter selinux output in _acl_ls etc Subject: [PATCH V3] xfstests: filter selinux output in _acl_ls etc References: <4B61C1BC.4050800@sandeen.net> <4B626697.3080006@sandeen.net> In-Reply-To: <4B626697.3080006@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265152008 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21495 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When selinux is on, ls -l gives us a "." to indicate selinux attrs, which breaks some tests: === Test minimal ACE === Setup file --rwxrw-r-- id1 id2 file1 +-rwxrw-r--. id1 id2 file1 so make an _ls_l helper to filter that out. Signed-off-by: Eric Sandeen --- V3: after talking to christoph, maybe a sed that looks just for the ls -l chars might be better. diff --git a/105 b/105 index e3163fd..9544c66 100755 --- a/105 +++ b/105 @@ -76,7 +76,7 @@ chown $acl1 subdir # put a file in the directory echo data > subdir/file -ls -l subdir/file | awk '{ print $1, $3 }' +_ls_l subdir/file | awk '{ print $1, $3 }' # add an ACL with a user ACE which has no exec permission if [ "$HOSTOS" == "Linux" ]; then @@ -91,7 +91,7 @@ fi # With the bug this gives: `ls: subdir/file: Permission denied' # because one needs at least an exec perm somewhere in acl # However, this should not hold true for directories. -ls -l subdir/file | awk '{ print $1, $3 }' +_ls_l subdir/file | awk '{ print $1, $3 }' # With the bug this gives: `subdir/file2: Permission denied'. echo data2 > subdir/file2 diff --git a/common.attr b/common.attr index a6b9b3b..d12cc02 100644 --- a/common.attr +++ b/common.attr @@ -58,7 +58,7 @@ _acl_filter_id() # _acl_ls() { - ls -ln $* | awk '{ print $1, $3, $4, $NF }' | _acl_filter_id + _ls_l -n $* | awk '{ print $1, $3, $4, $NF }' | _acl_filter_id } # diff --git a/common.rc b/common.rc index 6424871..4425007 100644 --- a/common.rc +++ b/common.rc @@ -37,6 +37,14 @@ dd() fi } +# ls -l w/ selinux sometimes puts a dot at the end: +# -rwxrw-r--. id1 id2 file1 + +_ls_l() +{ + ls -l $* | sed "s/\(^[-rwxdlbcpsStT]*\)\. /\1 /" +} + _mount_opts() { case $FSTYP in From sandeen@sandeen.net Tue Feb 2 17:27:44 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_43,J_CHICKENPOX_48,J_CHICKENPOX_52 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o12NRh4l198220 for ; Tue, 2 Feb 2010 17:27:43 -0600 X-ASG-Debug-ID: 1265153331-75cb029c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1AD3C1A706E for ; Tue, 2 Feb 2010 15:28:51 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id 8yj7rB0LND90umfq for ; Tue, 02 Feb 2010 15:28:51 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id E65D5109E84B; Tue, 2 Feb 2010 17:28:50 -0600 (CST) Message-ID: <4B68B532.9000103@sandeen.net> Date: Tue, 02 Feb 2010 17:28:50 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: ext4 development , xfs-oss X-ASG-Orig-Subj: [PATCH V2] xfstests: 223 - test file alignment on stripe geometry Subject: [PATCH V2] xfstests: 223 - test file alignment on stripe geometry References: <4B621529.20708@sandeen.net> <20100130105236.GA18286@infradead.org> In-Reply-To: <20100130105236.GA18286@infradead.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265153332 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21496 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean A first-cut test to ensure that files are well-aligned on filesystems with stripe geometry. Several sizes of stripe units are mkfs'd, and then files are written and fallocated in various multiples of those stripe sizes. Each file is checked to ensure that the first block is stripe-aligned. (Ideally, for any fragmented files, we should ensure that each fragment start is well-aligned, but this does not do that yet) (slightly unrelated: don't send scratch mkfs output to /dev/null, we'd like to see mkfs output and direct it to $seq.full - this more or less matches _scratch_mkfs_xfs behavior and doesn't break any tests that I can see) Signed-off-by: Eric Sandeen --- V2: Address Christoph's review comments diff --git a/223 b/223 new file mode 100755 index 0000000..45057a3 --- /dev/null +++ b/223 @@ -0,0 +1,96 @@ +#! /bin/bash +# FS QA Test No. 223 +# +# File alignment tests +# +#----------------------------------------------------------------------- +# Copyright (c) 2010 Eric Sandeen. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=sandeen@sandeen.net + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! + +_cleanup() +{ + rm -f $tmp.* +} + +trap "_cleanup ; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs generic +_supported_os Linux + +_require_scratch +_require_xfs_io_falloc + +rm -f $seq.full + +_filter_scratch() +{ + sed -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" +} + +BLOCKSIZE=4096 + +for SUNIT_K in 8 16 32 64 128; do + let SUNIT_BYTES=$SUNIT_K*1024 + let SUNIT_BLOCKS=$SUNIT_BYTES/$BLOCKSIZE + + echo "=== mkfs with su $SUNIT_BLOCKS blocks x 4 ===" + scratch_mkfs_geom $BLOCKSIZE $SUNIT_BYTES 4 + _scratch_mount + + for SIZE_MULT in 1 2 8 64 256; do + let SIZE=$SIZE_MULT*$SUNIT_BYTES + + echo "=== Testing size ${SIZE_MULT}*${SUNIT_K}k on ${SUNIT_K}k stripe ===" + for FILE in 1 2 3 4; do + xfs_io -F -f -c "falloc 0 $SIZE" $SCRATCH_MNT/file-$FILE-$SIZE-falloc &>> $seq.full + xfs_io -F -f -c "pwrite 0 $SIZE" $SCRATCH_MNT/file-$FILE-$SIZE-write &>> $seq.full + src/fibmap $SCRATCH_MNT/file-$FILE-$SIZE-falloc $SUNIT_BLOCKS | \ + _filter_scratch + src/fibmap $SCRATCH_MNT/file-$FILE-$SIZE-write $SUNIT_BLOCKS | \ + _filter_scratch + done + done + + echo "=== Testing size 1g falloc on ${SUNIT_K}k stripe ===" + xfs_io -F -f -c "falloc 0 1g" $SCRATCH_MNT/file-1g-falloc &>> $seq.full + src/fibmap $SCRATCH_MNT/file-1g-falloc $SUNIT_BLOCKS + + rm -f $SCRATCH_MNT/file-1g-falloc | _filter_scratch + + echo "=== Testing size 1073745920 falloc on ${SUNIT_K}k stripe ===" + xfs_io -F -f -c "falloc 0 1073745920" $SCRATCH_MNT/file-1073745920-falloc &>> $seq.full + src/fibmap $SCRATCH_MNT/file-1073745920-falloc $SUNIT_BLOCKS | _filter_scratch + + _scratch_unmount +done + +status=0 ; exit diff --git a/223.out b/223.out new file mode 100644 index 0000000..c9588ef --- /dev/null +++ b/223.out @@ -0,0 +1,251 @@ +QA output created by 223 +=== mkfs with su 2 blocks x 4 === +=== Testing size 1*8k on 8k stripe === +SCRATCH_MNT/file-1-8192-falloc: well-aligned +SCRATCH_MNT/file-1-8192-write: well-aligned +SCRATCH_MNT/file-2-8192-falloc: well-aligned +SCRATCH_MNT/file-2-8192-write: well-aligned +SCRATCH_MNT/file-3-8192-falloc: well-aligned +SCRATCH_MNT/file-3-8192-write: well-aligned +SCRATCH_MNT/file-4-8192-falloc: well-aligned +SCRATCH_MNT/file-4-8192-write: well-aligned +=== Testing size 2*8k on 8k stripe === +SCRATCH_MNT/file-1-16384-falloc: well-aligned +SCRATCH_MNT/file-1-16384-write: well-aligned +SCRATCH_MNT/file-2-16384-falloc: well-aligned +SCRATCH_MNT/file-2-16384-write: well-aligned +SCRATCH_MNT/file-3-16384-falloc: well-aligned +SCRATCH_MNT/file-3-16384-write: well-aligned +SCRATCH_MNT/file-4-16384-falloc: well-aligned +SCRATCH_MNT/file-4-16384-write: well-aligned +=== Testing size 8*8k on 8k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 64*8k on 8k stripe === +SCRATCH_MNT/file-1-524288-falloc: well-aligned +SCRATCH_MNT/file-1-524288-write: well-aligned +SCRATCH_MNT/file-2-524288-falloc: well-aligned +SCRATCH_MNT/file-2-524288-write: well-aligned +SCRATCH_MNT/file-3-524288-falloc: well-aligned +SCRATCH_MNT/file-3-524288-write: well-aligned +SCRATCH_MNT/file-4-524288-falloc: well-aligned +SCRATCH_MNT/file-4-524288-write: well-aligned +=== Testing size 256*8k on 8k stripe === +SCRATCH_MNT/file-1-2097152-falloc: well-aligned +SCRATCH_MNT/file-1-2097152-write: well-aligned +SCRATCH_MNT/file-2-2097152-falloc: well-aligned +SCRATCH_MNT/file-2-2097152-write: well-aligned +SCRATCH_MNT/file-3-2097152-falloc: well-aligned +SCRATCH_MNT/file-3-2097152-write: well-aligned +SCRATCH_MNT/file-4-2097152-falloc: well-aligned +SCRATCH_MNT/file-4-2097152-write: well-aligned +=== Testing size 1g falloc on 8k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 8k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 4 blocks x 4 === +=== Testing size 1*16k on 16k stripe === +SCRATCH_MNT/file-1-16384-falloc: well-aligned +SCRATCH_MNT/file-1-16384-write: well-aligned +SCRATCH_MNT/file-2-16384-falloc: well-aligned +SCRATCH_MNT/file-2-16384-write: well-aligned +SCRATCH_MNT/file-3-16384-falloc: well-aligned +SCRATCH_MNT/file-3-16384-write: well-aligned +SCRATCH_MNT/file-4-16384-falloc: well-aligned +SCRATCH_MNT/file-4-16384-write: well-aligned +=== Testing size 2*16k on 16k stripe === +SCRATCH_MNT/file-1-32768-falloc: well-aligned +SCRATCH_MNT/file-1-32768-write: well-aligned +SCRATCH_MNT/file-2-32768-falloc: well-aligned +SCRATCH_MNT/file-2-32768-write: well-aligned +SCRATCH_MNT/file-3-32768-falloc: well-aligned +SCRATCH_MNT/file-3-32768-write: well-aligned +SCRATCH_MNT/file-4-32768-falloc: well-aligned +SCRATCH_MNT/file-4-32768-write: well-aligned +=== Testing size 8*16k on 16k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 64*16k on 16k stripe === +SCRATCH_MNT/file-1-1048576-falloc: well-aligned +SCRATCH_MNT/file-1-1048576-write: well-aligned +SCRATCH_MNT/file-2-1048576-falloc: well-aligned +SCRATCH_MNT/file-2-1048576-write: well-aligned +SCRATCH_MNT/file-3-1048576-falloc: well-aligned +SCRATCH_MNT/file-3-1048576-write: well-aligned +SCRATCH_MNT/file-4-1048576-falloc: well-aligned +SCRATCH_MNT/file-4-1048576-write: well-aligned +=== Testing size 256*16k on 16k stripe === +SCRATCH_MNT/file-1-4194304-falloc: well-aligned +SCRATCH_MNT/file-1-4194304-write: well-aligned +SCRATCH_MNT/file-2-4194304-falloc: well-aligned +SCRATCH_MNT/file-2-4194304-write: well-aligned +SCRATCH_MNT/file-3-4194304-falloc: well-aligned +SCRATCH_MNT/file-3-4194304-write: well-aligned +SCRATCH_MNT/file-4-4194304-falloc: well-aligned +SCRATCH_MNT/file-4-4194304-write: well-aligned +=== Testing size 1g falloc on 16k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 16k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 8 blocks x 4 === +=== Testing size 1*32k on 32k stripe === +SCRATCH_MNT/file-1-32768-falloc: well-aligned +SCRATCH_MNT/file-1-32768-write: well-aligned +SCRATCH_MNT/file-2-32768-falloc: well-aligned +SCRATCH_MNT/file-2-32768-write: well-aligned +SCRATCH_MNT/file-3-32768-falloc: well-aligned +SCRATCH_MNT/file-3-32768-write: well-aligned +SCRATCH_MNT/file-4-32768-falloc: well-aligned +SCRATCH_MNT/file-4-32768-write: well-aligned +=== Testing size 2*32k on 32k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 8*32k on 32k stripe === +SCRATCH_MNT/file-1-262144-falloc: well-aligned +SCRATCH_MNT/file-1-262144-write: well-aligned +SCRATCH_MNT/file-2-262144-falloc: well-aligned +SCRATCH_MNT/file-2-262144-write: well-aligned +SCRATCH_MNT/file-3-262144-falloc: well-aligned +SCRATCH_MNT/file-3-262144-write: well-aligned +SCRATCH_MNT/file-4-262144-falloc: well-aligned +SCRATCH_MNT/file-4-262144-write: well-aligned +=== Testing size 64*32k on 32k stripe === +SCRATCH_MNT/file-1-2097152-falloc: well-aligned +SCRATCH_MNT/file-1-2097152-write: well-aligned +SCRATCH_MNT/file-2-2097152-falloc: well-aligned +SCRATCH_MNT/file-2-2097152-write: well-aligned +SCRATCH_MNT/file-3-2097152-falloc: well-aligned +SCRATCH_MNT/file-3-2097152-write: well-aligned +SCRATCH_MNT/file-4-2097152-falloc: well-aligned +SCRATCH_MNT/file-4-2097152-write: well-aligned +=== Testing size 256*32k on 32k stripe === +SCRATCH_MNT/file-1-8388608-falloc: well-aligned +SCRATCH_MNT/file-1-8388608-write: well-aligned +SCRATCH_MNT/file-2-8388608-falloc: well-aligned +SCRATCH_MNT/file-2-8388608-write: well-aligned +SCRATCH_MNT/file-3-8388608-falloc: well-aligned +SCRATCH_MNT/file-3-8388608-write: well-aligned +SCRATCH_MNT/file-4-8388608-falloc: well-aligned +SCRATCH_MNT/file-4-8388608-write: well-aligned +=== Testing size 1g falloc on 32k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 32k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 16 blocks x 4 === +=== Testing size 1*64k on 64k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 2*64k on 64k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 8*64k on 64k stripe === +SCRATCH_MNT/file-1-524288-falloc: well-aligned +SCRATCH_MNT/file-1-524288-write: well-aligned +SCRATCH_MNT/file-2-524288-falloc: well-aligned +SCRATCH_MNT/file-2-524288-write: well-aligned +SCRATCH_MNT/file-3-524288-falloc: well-aligned +SCRATCH_MNT/file-3-524288-write: well-aligned +SCRATCH_MNT/file-4-524288-falloc: well-aligned +SCRATCH_MNT/file-4-524288-write: well-aligned +=== Testing size 64*64k on 64k stripe === +SCRATCH_MNT/file-1-4194304-falloc: well-aligned +SCRATCH_MNT/file-1-4194304-write: well-aligned +SCRATCH_MNT/file-2-4194304-falloc: well-aligned +SCRATCH_MNT/file-2-4194304-write: well-aligned +SCRATCH_MNT/file-3-4194304-falloc: well-aligned +SCRATCH_MNT/file-3-4194304-write: well-aligned +SCRATCH_MNT/file-4-4194304-falloc: well-aligned +SCRATCH_MNT/file-4-4194304-write: well-aligned +=== Testing size 256*64k on 64k stripe === +SCRATCH_MNT/file-1-16777216-falloc: well-aligned +SCRATCH_MNT/file-1-16777216-write: well-aligned +SCRATCH_MNT/file-2-16777216-falloc: well-aligned +SCRATCH_MNT/file-2-16777216-write: well-aligned +SCRATCH_MNT/file-3-16777216-falloc: well-aligned +SCRATCH_MNT/file-3-16777216-write: well-aligned +SCRATCH_MNT/file-4-16777216-falloc: well-aligned +SCRATCH_MNT/file-4-16777216-write: well-aligned +=== Testing size 1g falloc on 64k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 64k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 32 blocks x 4 === +=== Testing size 1*128k on 128k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 2*128k on 128k stripe === +SCRATCH_MNT/file-1-262144-falloc: well-aligned +SCRATCH_MNT/file-1-262144-write: well-aligned +SCRATCH_MNT/file-2-262144-falloc: well-aligned +SCRATCH_MNT/file-2-262144-write: well-aligned +SCRATCH_MNT/file-3-262144-falloc: well-aligned +SCRATCH_MNT/file-3-262144-write: well-aligned +SCRATCH_MNT/file-4-262144-falloc: well-aligned +SCRATCH_MNT/file-4-262144-write: well-aligned +=== Testing size 8*128k on 128k stripe === +SCRATCH_MNT/file-1-1048576-falloc: well-aligned +SCRATCH_MNT/file-1-1048576-write: well-aligned +SCRATCH_MNT/file-2-1048576-falloc: well-aligned +SCRATCH_MNT/file-2-1048576-write: well-aligned +SCRATCH_MNT/file-3-1048576-falloc: well-aligned +SCRATCH_MNT/file-3-1048576-write: well-aligned +SCRATCH_MNT/file-4-1048576-falloc: well-aligned +SCRATCH_MNT/file-4-1048576-write: well-aligned +=== Testing size 64*128k on 128k stripe === +SCRATCH_MNT/file-1-8388608-falloc: well-aligned +SCRATCH_MNT/file-1-8388608-write: well-aligned +SCRATCH_MNT/file-2-8388608-falloc: well-aligned +SCRATCH_MNT/file-2-8388608-write: well-aligned +SCRATCH_MNT/file-3-8388608-falloc: well-aligned +SCRATCH_MNT/file-3-8388608-write: well-aligned +SCRATCH_MNT/file-4-8388608-falloc: well-aligned +SCRATCH_MNT/file-4-8388608-write: well-aligned +=== Testing size 256*128k on 128k stripe === +SCRATCH_MNT/file-1-33554432-falloc: well-aligned +SCRATCH_MNT/file-1-33554432-write: well-aligned +SCRATCH_MNT/file-2-33554432-falloc: well-aligned +SCRATCH_MNT/file-2-33554432-write: well-aligned +SCRATCH_MNT/file-3-33554432-falloc: well-aligned +SCRATCH_MNT/file-3-33554432-write: well-aligned +SCRATCH_MNT/file-4-33554432-falloc: well-aligned +SCRATCH_MNT/file-4-33554432-write: well-aligned +=== Testing size 1g falloc on 128k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 128k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned diff --git a/common.rc b/common.rc index 4425007..6cd11ff 100644 --- a/common.rc +++ b/common.rc @@ -300,11 +300,35 @@ _scratch_mkfs() $MKFS_UDF_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null ;; *) - /sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null + /sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV ;; esac } +scratch_mkfs_geom() +{ + blocksize=$1 + sunit_bytes=$2 + swidth_mult=$3 + + let sunit_blocks=$sunit_bytes/$blocksize + let swidth_blocks=$sunit_blocks*$swidth_mult + + case $FSTYP in + xfs) + MKFS_OPTIONS="-b size=$blocksize, -d su=$sunit_bytes,sw=$swidth_mult" + ;; + ext4) + MKFS_OPTIONS="-b $blocksize -E stride=$sunit_blocks,stripe_width=$swidth_blocks" + ;; + *) + _notrun "can't mkfs $FSTYP with geometry" + ;; + esac + _scratch_mkfs &>> $seq.full +} + +# Emulate a 4-data-disk stripe w/ various stripe units _scratch_xfs_db_options() { SCRATCH_OPTIONS="" diff --git a/group b/group index 342ac89..6b8528f 100644 --- a/group +++ b/group @@ -336,3 +336,4 @@ deprecated 220 auto quota quick 221 auto metadata quick 222 auto fsr ioctl quick +223 auto quick diff --git a/src/Makefile b/src/Makefile index baa2e94..d86d50a 100644 --- a/src/Makefile +++ b/src/Makefile @@ -14,7 +14,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \ - locktest unwritten_mmap bulkstat_unlink_test \ + locktest unwritten_mmap bulkstat_unlink_test t_stripealign \ bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable SUBDIRS = diff --git a/src/t_stripealign.c b/src/t_stripealign.c new file mode 100644 index 0000000..b2ef5c2 --- /dev/null +++ b/src/t_stripealign.c @@ -0,0 +1,72 @@ +/* + * t_stripealign.c + * + * Pint whether the file start block is stripe-aligned. + * + * Copyright (c) 2010 Eric Sandeen + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2, or (at your option) + * any later version. + */ +#include +#include +#include +#include +#include +#include +#include + +#define FIBMAP _IO(0x00, 1) /* bmap access */ + +/* + * If only filename given, print first block. + * + * If filename & sunit (in blocks) given, print whether we are well-aligned + */ + +int main(int argc, char ** argv) +{ + int fd; + int ret; + int sunit = 0; /* in blocks */ + char *filename; + unsigned int block = 0; + + if (argc < 3) { + printf("Usage: %s \n", argv[0]); + return 1; + } + + filename = argv[1]; + sunit = atoi(argv[2]); + + fd = open(filename, O_RDONLY); + if (fd < 0) { + perror("can't open file\n"); + return 1; + } + + ret = ioctl(fd, FIBMAP, &block); + if (ret < 0) { + close(fd); + perror("fibmap"); + return 1; + } + + close(fd); + + if (block % sunit) { + printf("%s: Start block %u not multiple of sunit %u\n", + filename, block, sunit); + return 1; + } else + printf("%s: well-aligned\n", filename); + + return 0; +} + + + + From sandeen@sandeen.net Tue Feb 2 19:47:09 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.9 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_43,J_CHICKENPOX_48,J_CHICKENPOX_52,J_CHICKENPOX_62 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o131l8Po203348 for ; Tue, 2 Feb 2010 19:47:08 -0600 X-ASG-Debug-ID: 1265161695-625403a80000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7D22A1371DC7 for ; Tue, 2 Feb 2010 17:48:15 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id cDgahNvP9qqvwxcC for ; Tue, 02 Feb 2010 17:48:15 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 8E0FD1120416; Tue, 2 Feb 2010 19:48:14 -0600 (CST) Message-ID: <4B68D5DF.20808@sandeen.net> Date: Tue, 02 Feb 2010 19:48:15 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: xfs-oss X-ASG-Orig-Subj: [PATCH V3] xfstests: 223 - test file alignment on stripe geometry Subject: [PATCH V3] xfstests: 223 - test file alignment on stripe geometry References: <4B621529.20708@sandeen.net> <20100130105236.GA18286@infradead.org> <4B68B532.9000103@sandeen.net> In-Reply-To: <4B68B532.9000103@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265161696 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21505 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean A first-cut test to ensure that files are well-aligned on filesystems with stripe geometry. Several sizes of stripe units are mkfs'd, and then files are written and fallocated in various multiples of those stripe sizes. Each file is checked to ensure that the first block is stripe-aligned. (Ideally, for any fragmented files, we should ensure that each fragment start is well-aligned, but this does not do that yet) (slightly unrelated: don't send scratch mkfs output to /dev/null, we'd like to see mkfs output and direct it to $seq.full - this more or less matches _scratch_mkfs_xfs behavior and doesn't break any tests that I can see) Signed-off-by: Eric Sandeen --- V2: Address Christoph's review comments V3: add missing _ to the helper function, minor cleanups to the common.rc file (sorry for all the versions) :) diff --git a/223 b/223 new file mode 100755 index 0000000..a605b6d --- /dev/null +++ b/223 @@ -0,0 +1,96 @@ +#! /bin/bash +# FS QA Test No. 223 +# +# File alignment tests +# +#----------------------------------------------------------------------- +# Copyright (c) 2010 Eric Sandeen. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +# +#----------------------------------------------------------------------- +# +# creator +owner=sandeen@sandeen.net + +seq=`basename $0` +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! + +_cleanup() +{ + rm -f $tmp.* +} + +trap "_cleanup ; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter + +# real QA test starts here +_supported_fs generic +_supported_os Linux + +_require_scratch +_require_xfs_io_falloc + +rm -f $seq.full + +_filter_scratch() +{ + sed -e "s,$SCRATCH_MNT,SCRATCH_MNT,g" +} + +BLOCKSIZE=4096 + +for SUNIT_K in 8 16 32 64 128; do + let SUNIT_BYTES=$SUNIT_K*1024 + let SUNIT_BLOCKS=$SUNIT_BYTES/$BLOCKSIZE + + echo "=== mkfs with su $SUNIT_BLOCKS blocks x 4 ===" + _scratch_mkfs_geom $BLOCKSIZE $SUNIT_BYTES 4 &>> $seq.full + _scratch_mount + + for SIZE_MULT in 1 2 8 64 256; do + let SIZE=$SIZE_MULT*$SUNIT_BYTES + + echo "=== Testing size ${SIZE_MULT}*${SUNIT_K}k on ${SUNIT_K}k stripe ===" + for FILE in 1 2 3 4; do + xfs_io -F -f -c "falloc 0 $SIZE" $SCRATCH_MNT/file-$FILE-$SIZE-falloc &>> $seq.full + xfs_io -F -f -c "pwrite 0 $SIZE" $SCRATCH_MNT/file-$FILE-$SIZE-write &>> $seq.full + src/fibmap $SCRATCH_MNT/file-$FILE-$SIZE-falloc $SUNIT_BLOCKS | \ + _filter_scratch + src/fibmap $SCRATCH_MNT/file-$FILE-$SIZE-write $SUNIT_BLOCKS | \ + _filter_scratch + done + done + + echo "=== Testing size 1g falloc on ${SUNIT_K}k stripe ===" + xfs_io -F -f -c "falloc 0 1g" $SCRATCH_MNT/file-1g-falloc &>> $seq.full + src/fibmap $SCRATCH_MNT/file-1g-falloc $SUNIT_BLOCKS + + rm -f $SCRATCH_MNT/file-1g-falloc | _filter_scratch + + echo "=== Testing size 1073745920 falloc on ${SUNIT_K}k stripe ===" + xfs_io -F -f -c "falloc 0 1073745920" $SCRATCH_MNT/file-1073745920-falloc &>> $seq.full + src/fibmap $SCRATCH_MNT/file-1073745920-falloc $SUNIT_BLOCKS | _filter_scratch + + _scratch_unmount +done + +status=0 ; exit diff --git a/223.out b/223.out new file mode 100644 index 0000000..c9588ef --- /dev/null +++ b/223.out @@ -0,0 +1,251 @@ +QA output created by 223 +=== mkfs with su 2 blocks x 4 === +=== Testing size 1*8k on 8k stripe === +SCRATCH_MNT/file-1-8192-falloc: well-aligned +SCRATCH_MNT/file-1-8192-write: well-aligned +SCRATCH_MNT/file-2-8192-falloc: well-aligned +SCRATCH_MNT/file-2-8192-write: well-aligned +SCRATCH_MNT/file-3-8192-falloc: well-aligned +SCRATCH_MNT/file-3-8192-write: well-aligned +SCRATCH_MNT/file-4-8192-falloc: well-aligned +SCRATCH_MNT/file-4-8192-write: well-aligned +=== Testing size 2*8k on 8k stripe === +SCRATCH_MNT/file-1-16384-falloc: well-aligned +SCRATCH_MNT/file-1-16384-write: well-aligned +SCRATCH_MNT/file-2-16384-falloc: well-aligned +SCRATCH_MNT/file-2-16384-write: well-aligned +SCRATCH_MNT/file-3-16384-falloc: well-aligned +SCRATCH_MNT/file-3-16384-write: well-aligned +SCRATCH_MNT/file-4-16384-falloc: well-aligned +SCRATCH_MNT/file-4-16384-write: well-aligned +=== Testing size 8*8k on 8k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 64*8k on 8k stripe === +SCRATCH_MNT/file-1-524288-falloc: well-aligned +SCRATCH_MNT/file-1-524288-write: well-aligned +SCRATCH_MNT/file-2-524288-falloc: well-aligned +SCRATCH_MNT/file-2-524288-write: well-aligned +SCRATCH_MNT/file-3-524288-falloc: well-aligned +SCRATCH_MNT/file-3-524288-write: well-aligned +SCRATCH_MNT/file-4-524288-falloc: well-aligned +SCRATCH_MNT/file-4-524288-write: well-aligned +=== Testing size 256*8k on 8k stripe === +SCRATCH_MNT/file-1-2097152-falloc: well-aligned +SCRATCH_MNT/file-1-2097152-write: well-aligned +SCRATCH_MNT/file-2-2097152-falloc: well-aligned +SCRATCH_MNT/file-2-2097152-write: well-aligned +SCRATCH_MNT/file-3-2097152-falloc: well-aligned +SCRATCH_MNT/file-3-2097152-write: well-aligned +SCRATCH_MNT/file-4-2097152-falloc: well-aligned +SCRATCH_MNT/file-4-2097152-write: well-aligned +=== Testing size 1g falloc on 8k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 8k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 4 blocks x 4 === +=== Testing size 1*16k on 16k stripe === +SCRATCH_MNT/file-1-16384-falloc: well-aligned +SCRATCH_MNT/file-1-16384-write: well-aligned +SCRATCH_MNT/file-2-16384-falloc: well-aligned +SCRATCH_MNT/file-2-16384-write: well-aligned +SCRATCH_MNT/file-3-16384-falloc: well-aligned +SCRATCH_MNT/file-3-16384-write: well-aligned +SCRATCH_MNT/file-4-16384-falloc: well-aligned +SCRATCH_MNT/file-4-16384-write: well-aligned +=== Testing size 2*16k on 16k stripe === +SCRATCH_MNT/file-1-32768-falloc: well-aligned +SCRATCH_MNT/file-1-32768-write: well-aligned +SCRATCH_MNT/file-2-32768-falloc: well-aligned +SCRATCH_MNT/file-2-32768-write: well-aligned +SCRATCH_MNT/file-3-32768-falloc: well-aligned +SCRATCH_MNT/file-3-32768-write: well-aligned +SCRATCH_MNT/file-4-32768-falloc: well-aligned +SCRATCH_MNT/file-4-32768-write: well-aligned +=== Testing size 8*16k on 16k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 64*16k on 16k stripe === +SCRATCH_MNT/file-1-1048576-falloc: well-aligned +SCRATCH_MNT/file-1-1048576-write: well-aligned +SCRATCH_MNT/file-2-1048576-falloc: well-aligned +SCRATCH_MNT/file-2-1048576-write: well-aligned +SCRATCH_MNT/file-3-1048576-falloc: well-aligned +SCRATCH_MNT/file-3-1048576-write: well-aligned +SCRATCH_MNT/file-4-1048576-falloc: well-aligned +SCRATCH_MNT/file-4-1048576-write: well-aligned +=== Testing size 256*16k on 16k stripe === +SCRATCH_MNT/file-1-4194304-falloc: well-aligned +SCRATCH_MNT/file-1-4194304-write: well-aligned +SCRATCH_MNT/file-2-4194304-falloc: well-aligned +SCRATCH_MNT/file-2-4194304-write: well-aligned +SCRATCH_MNT/file-3-4194304-falloc: well-aligned +SCRATCH_MNT/file-3-4194304-write: well-aligned +SCRATCH_MNT/file-4-4194304-falloc: well-aligned +SCRATCH_MNT/file-4-4194304-write: well-aligned +=== Testing size 1g falloc on 16k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 16k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 8 blocks x 4 === +=== Testing size 1*32k on 32k stripe === +SCRATCH_MNT/file-1-32768-falloc: well-aligned +SCRATCH_MNT/file-1-32768-write: well-aligned +SCRATCH_MNT/file-2-32768-falloc: well-aligned +SCRATCH_MNT/file-2-32768-write: well-aligned +SCRATCH_MNT/file-3-32768-falloc: well-aligned +SCRATCH_MNT/file-3-32768-write: well-aligned +SCRATCH_MNT/file-4-32768-falloc: well-aligned +SCRATCH_MNT/file-4-32768-write: well-aligned +=== Testing size 2*32k on 32k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 8*32k on 32k stripe === +SCRATCH_MNT/file-1-262144-falloc: well-aligned +SCRATCH_MNT/file-1-262144-write: well-aligned +SCRATCH_MNT/file-2-262144-falloc: well-aligned +SCRATCH_MNT/file-2-262144-write: well-aligned +SCRATCH_MNT/file-3-262144-falloc: well-aligned +SCRATCH_MNT/file-3-262144-write: well-aligned +SCRATCH_MNT/file-4-262144-falloc: well-aligned +SCRATCH_MNT/file-4-262144-write: well-aligned +=== Testing size 64*32k on 32k stripe === +SCRATCH_MNT/file-1-2097152-falloc: well-aligned +SCRATCH_MNT/file-1-2097152-write: well-aligned +SCRATCH_MNT/file-2-2097152-falloc: well-aligned +SCRATCH_MNT/file-2-2097152-write: well-aligned +SCRATCH_MNT/file-3-2097152-falloc: well-aligned +SCRATCH_MNT/file-3-2097152-write: well-aligned +SCRATCH_MNT/file-4-2097152-falloc: well-aligned +SCRATCH_MNT/file-4-2097152-write: well-aligned +=== Testing size 256*32k on 32k stripe === +SCRATCH_MNT/file-1-8388608-falloc: well-aligned +SCRATCH_MNT/file-1-8388608-write: well-aligned +SCRATCH_MNT/file-2-8388608-falloc: well-aligned +SCRATCH_MNT/file-2-8388608-write: well-aligned +SCRATCH_MNT/file-3-8388608-falloc: well-aligned +SCRATCH_MNT/file-3-8388608-write: well-aligned +SCRATCH_MNT/file-4-8388608-falloc: well-aligned +SCRATCH_MNT/file-4-8388608-write: well-aligned +=== Testing size 1g falloc on 32k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 32k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 16 blocks x 4 === +=== Testing size 1*64k on 64k stripe === +SCRATCH_MNT/file-1-65536-falloc: well-aligned +SCRATCH_MNT/file-1-65536-write: well-aligned +SCRATCH_MNT/file-2-65536-falloc: well-aligned +SCRATCH_MNT/file-2-65536-write: well-aligned +SCRATCH_MNT/file-3-65536-falloc: well-aligned +SCRATCH_MNT/file-3-65536-write: well-aligned +SCRATCH_MNT/file-4-65536-falloc: well-aligned +SCRATCH_MNT/file-4-65536-write: well-aligned +=== Testing size 2*64k on 64k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 8*64k on 64k stripe === +SCRATCH_MNT/file-1-524288-falloc: well-aligned +SCRATCH_MNT/file-1-524288-write: well-aligned +SCRATCH_MNT/file-2-524288-falloc: well-aligned +SCRATCH_MNT/file-2-524288-write: well-aligned +SCRATCH_MNT/file-3-524288-falloc: well-aligned +SCRATCH_MNT/file-3-524288-write: well-aligned +SCRATCH_MNT/file-4-524288-falloc: well-aligned +SCRATCH_MNT/file-4-524288-write: well-aligned +=== Testing size 64*64k on 64k stripe === +SCRATCH_MNT/file-1-4194304-falloc: well-aligned +SCRATCH_MNT/file-1-4194304-write: well-aligned +SCRATCH_MNT/file-2-4194304-falloc: well-aligned +SCRATCH_MNT/file-2-4194304-write: well-aligned +SCRATCH_MNT/file-3-4194304-falloc: well-aligned +SCRATCH_MNT/file-3-4194304-write: well-aligned +SCRATCH_MNT/file-4-4194304-falloc: well-aligned +SCRATCH_MNT/file-4-4194304-write: well-aligned +=== Testing size 256*64k on 64k stripe === +SCRATCH_MNT/file-1-16777216-falloc: well-aligned +SCRATCH_MNT/file-1-16777216-write: well-aligned +SCRATCH_MNT/file-2-16777216-falloc: well-aligned +SCRATCH_MNT/file-2-16777216-write: well-aligned +SCRATCH_MNT/file-3-16777216-falloc: well-aligned +SCRATCH_MNT/file-3-16777216-write: well-aligned +SCRATCH_MNT/file-4-16777216-falloc: well-aligned +SCRATCH_MNT/file-4-16777216-write: well-aligned +=== Testing size 1g falloc on 64k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 64k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned +=== mkfs with su 32 blocks x 4 === +=== Testing size 1*128k on 128k stripe === +SCRATCH_MNT/file-1-131072-falloc: well-aligned +SCRATCH_MNT/file-1-131072-write: well-aligned +SCRATCH_MNT/file-2-131072-falloc: well-aligned +SCRATCH_MNT/file-2-131072-write: well-aligned +SCRATCH_MNT/file-3-131072-falloc: well-aligned +SCRATCH_MNT/file-3-131072-write: well-aligned +SCRATCH_MNT/file-4-131072-falloc: well-aligned +SCRATCH_MNT/file-4-131072-write: well-aligned +=== Testing size 2*128k on 128k stripe === +SCRATCH_MNT/file-1-262144-falloc: well-aligned +SCRATCH_MNT/file-1-262144-write: well-aligned +SCRATCH_MNT/file-2-262144-falloc: well-aligned +SCRATCH_MNT/file-2-262144-write: well-aligned +SCRATCH_MNT/file-3-262144-falloc: well-aligned +SCRATCH_MNT/file-3-262144-write: well-aligned +SCRATCH_MNT/file-4-262144-falloc: well-aligned +SCRATCH_MNT/file-4-262144-write: well-aligned +=== Testing size 8*128k on 128k stripe === +SCRATCH_MNT/file-1-1048576-falloc: well-aligned +SCRATCH_MNT/file-1-1048576-write: well-aligned +SCRATCH_MNT/file-2-1048576-falloc: well-aligned +SCRATCH_MNT/file-2-1048576-write: well-aligned +SCRATCH_MNT/file-3-1048576-falloc: well-aligned +SCRATCH_MNT/file-3-1048576-write: well-aligned +SCRATCH_MNT/file-4-1048576-falloc: well-aligned +SCRATCH_MNT/file-4-1048576-write: well-aligned +=== Testing size 64*128k on 128k stripe === +SCRATCH_MNT/file-1-8388608-falloc: well-aligned +SCRATCH_MNT/file-1-8388608-write: well-aligned +SCRATCH_MNT/file-2-8388608-falloc: well-aligned +SCRATCH_MNT/file-2-8388608-write: well-aligned +SCRATCH_MNT/file-3-8388608-falloc: well-aligned +SCRATCH_MNT/file-3-8388608-write: well-aligned +SCRATCH_MNT/file-4-8388608-falloc: well-aligned +SCRATCH_MNT/file-4-8388608-write: well-aligned +=== Testing size 256*128k on 128k stripe === +SCRATCH_MNT/file-1-33554432-falloc: well-aligned +SCRATCH_MNT/file-1-33554432-write: well-aligned +SCRATCH_MNT/file-2-33554432-falloc: well-aligned +SCRATCH_MNT/file-2-33554432-write: well-aligned +SCRATCH_MNT/file-3-33554432-falloc: well-aligned +SCRATCH_MNT/file-3-33554432-write: well-aligned +SCRATCH_MNT/file-4-33554432-falloc: well-aligned +SCRATCH_MNT/file-4-33554432-write: well-aligned +=== Testing size 1g falloc on 128k stripe === +/mnt/scratch/file-1g-falloc: well-aligned +=== Testing size 1073745920 falloc on 128k stripe === +SCRATCH_MNT/file-1073745920-falloc: well-aligned diff --git a/common.rc b/common.rc index 1685514..9c375dc 100644 --- a/common.rc +++ b/common.rc @@ -292,11 +292,37 @@ _scratch_mkfs() $MKFS_UDF_PROG $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null ;; *) - /sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV > /dev/null + /sbin/mkfs -t $FSTYP -- $MKFS_OPTIONS $* $SCRATCH_DEV ;; esac } +# Emulate an N-data-disk stripe w/ various stripe units +# _scratch_mkfs_geom [optional blocksize] +_scratch_mkfs_geom() +{ + sunit_bytes=$1 + swidth_mult=$2 + blocksize=$3 + [ -z "$blocksize" ] && blocksize=4096 + + let sunit_blocks=$sunit_bytes/$blocksize + let swidth_blocks=$sunit_blocks*$swidth_mult + + case $FSTYP in + xfs) + MKFS_OPTIONS="-b size=$blocksize, -d su=$sunit_bytes,sw=$swidth_mult" + ;; + ext4) + MKFS_OPTIONS="-b $blocksize -E stride=$sunit_blocks,stripe_width=$swidth_blocks" + ;; + *) + _notrun "can't mkfs $FSTYP with geometry" + ;; + esac + _scratch_mkfs +} + _scratch_xfs_db_options() { SCRATCH_OPTIONS="" diff --git a/group b/group index 342ac89..6b8528f 100644 --- a/group +++ b/group @@ -336,3 +336,4 @@ deprecated 220 auto quota quick 221 auto metadata quick 222 auto fsr ioctl quick +223 auto quick diff --git a/src/Makefile b/src/Makefile index baa2e94..d86d50a 100644 --- a/src/Makefile +++ b/src/Makefile @@ -14,7 +14,7 @@ TARGETS = dirstress fill fill2 getpagesize holes lstat64 \ LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \ preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest \ - locktest unwritten_mmap bulkstat_unlink_test \ + locktest unwritten_mmap bulkstat_unlink_test t_stripealign \ bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable SUBDIRS = diff --git a/src/t_stripealign.c b/src/t_stripealign.c new file mode 100644 index 0000000..e6b0925 --- /dev/null +++ b/src/t_stripealign.c @@ -0,0 +1,68 @@ +/* + * t_stripealign.c + * + * Print whether the file start block is stripe-aligned. + * + * Copyright (c) 2010 Eric Sandeen + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2, or (at your option) + * any later version. + */ +#include +#include +#include +#include +#include +#include +#include + +#define FIBMAP _IO(0x00, 1) /* bmap access */ + +/* + * If only filename given, print first block. + * + * If filename & sunit (in blocks) given, print whether we are well-aligned + */ + +int main(int argc, char ** argv) +{ + int fd; + int ret; + int sunit = 0; /* in blocks */ + char *filename; + unsigned int block = 0; + + if (argc < 3) { + printf("Usage: %s \n", argv[0]); + return 1; + } + + filename = argv[1]; + sunit = atoi(argv[2]); + + fd = open(filename, O_RDONLY); + if (fd < 0) { + perror("can't open file\n"); + return 1; + } + + ret = ioctl(fd, FIBMAP, &block); + if (ret < 0) { + close(fd); + perror("fibmap"); + return 1; + } + + close(fd); + + if (block % sunit) { + printf("%s: Start block %u not multiple of sunit %u\n", + filename, block, sunit); + return 1; + } else + printf("%s: well-aligned\n", filename); + + return 0; +} From sandeen@sandeen.net Tue Feb 2 19:53:27 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o131rQpO203516 for ; Tue, 2 Feb 2010 19:53:27 -0600 X-ASG-Debug-ID: 1265162074-6fc602e30000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B92C91C9B785 for ; Tue, 2 Feb 2010 17:54:34 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id MhynKT2gwYGx9JMG for ; Tue, 02 Feb 2010 17:54:34 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id EDFE311203F1 for ; Tue, 2 Feb 2010 19:54:33 -0600 (CST) Message-ID: <4B68D75A.9020105@sandeen.net> Date: Tue, 02 Feb 2010 19:54:34 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss X-ASG-Orig-Subj: [PATCH V2] xfstests: routine to create scratch of certain size Subject: [PATCH V2] xfstests: routine to create scratch of certain size References: <4B621D89.8070800@sandeen.net> In-Reply-To: <4B621D89.8070800@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265162075 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is needed for later enospc tests to be generic Signed-off-by: Eric Sandeen --- V2: Address Christoph's comment about use of MKFS_OPTIONS; just do it like the other geom helper and explicitly -set- MKFS_OPTIONS for scratch, to set the size, then just call _scratch_mkfs (this would override any global MKFS_OPTIONS setting, but I think that's ok for now, and not sure how these options might interact w/ previously set MKFS_OPTIONS anyway) diff --git a/common.rc b/common.rc index 9c375dc..a6526a3 100644 --- a/common.rc +++ b/common.rc @@ -297,6 +297,29 @@ _scratch_mkfs() esac } +# Create fs of certain size on scratch device +# _scratch_mkfs_sized [optional blocksize] +_scratch_mkfs_sized() +{ + fssize=$1 + blocksize=$2 + [ -z "$blocksize" ] && blocksize=4096 + let blocks=$fssize/$blocksize + + case $FSTYP in + xfs) + MKFS_OPTIONS="-d size=$fssize -b size=$blocksize" + ;; + ext2|ext3|ext4) + MKFS_OPTIONS="-b $blocksize $SCRATCH_DEV $blocks" + ;; + *) + _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized" + ;; + esac + _scratch_mkfs +} + # Emulate an N-data-disk stripe w/ various stripe units # _scratch_mkfs_geom [optional blocksize] _scratch_mkfs_geom() From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:28 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329SLg204047 for ; Tue, 2 Feb 2010 20:09:28 -0600 X-ASG-Debug-ID: 1265163035-525703e10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8131D1A72E3 for ; Tue, 2 Feb 2010 18:10:36 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id 1nfxv8mqiJXH0weA for ; Tue, 02 Feb 2010 18:10:36 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207811-1927428 for ; Wed, 03 Feb 2010 12:40:34 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7h-0003nl-Du for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:33 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006j0-Cv for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 06/10] xfs: kill the unused XFS_QMOPT_* flush flags V2 Subject: [PATCH 06/10] xfs: kill the unused XFS_QMOPT_* flush flags V2 Date: Wed, 3 Feb 2010 10:25:00 +1100 Message-Id: <1265153104-29680-7-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163037 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean dquots are never flushed asynchronously. Remove the flag and the async write support from the flush function. Make the default flush a delwri flush to make the inode flush code, which leaves the XFS_QMOPT_SYNC the only flag remaining. Convert that to use SYNC_WAIT instead, just like the inode flush code. V2: - just pass flush flags straight through Signed-off-by: Dave Chinner --- fs/xfs/quota/xfs_dquot.c | 13 ++++++------- fs/xfs/quota/xfs_dquot_item.c | 2 +- fs/xfs/quota/xfs_qm.c | 14 ++++++-------- fs/xfs/xfs_quota.h | 8 +------- 4 files changed, 14 insertions(+), 23 deletions(-) diff --git a/fs/xfs/quota/xfs_dquot.c b/fs/xfs/quota/xfs_dquot.c index 1620a56..5f79dd7 100644 --- a/fs/xfs/quota/xfs_dquot.c +++ b/fs/xfs/quota/xfs_dquot.c @@ -1187,7 +1187,7 @@ xfs_qm_dqflush( * block, nada. */ if (!XFS_DQ_IS_DIRTY(dqp) || - (!(flags & XFS_QMOPT_SYNC) && atomic_read(&dqp->q_pincount) > 0)) { + (!(flags & SYNC_WAIT) && atomic_read(&dqp->q_pincount) > 0)) { xfs_dqfunlock(dqp); return 0; } @@ -1251,18 +1251,17 @@ xfs_qm_dqflush( xfs_log_force(mp, 0); } - if (flags & XFS_QMOPT_DELWRI) { - xfs_bdwrite(mp, bp); - } else { + if (flags & SYNC_WAIT) error = xfs_bwrite(mp, bp); - } + else + xfs_bdwrite(mp, bp); trace_xfs_dqflush_done(dqp); /* * dqp is still locked, but caller is free to unlock it now. */ - return (error); + return error; } @@ -1443,7 +1442,7 @@ xfs_qm_dqpurge( * We don't care about getting disk errors here. We need * to purge this dquot anyway, so we go ahead regardless. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_SYNC); + error = xfs_qm_dqflush(dqp, SYNC_WAIT); if (error) xfs_fs_cmn_err(CE_WARN, mp, "xfs_qm_dqpurge: dquot %p flush failed", dqp); diff --git a/fs/xfs/quota/xfs_dquot_item.c b/fs/xfs/quota/xfs_dquot_item.c index dda0fb0..4e4ee9a 100644 --- a/fs/xfs/quota/xfs_dquot_item.c +++ b/fs/xfs/quota/xfs_dquot_item.c @@ -153,7 +153,7 @@ xfs_qm_dquot_logitem_push( * lock without sleeping, then there must not have been * anyone in the process of flushing the dquot. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dquot_logitem_push: push error %d on dqp %p", diff --git a/fs/xfs/quota/xfs_qm.c b/fs/xfs/quota/xfs_qm.c index 11cfd82..8699e51 100644 --- a/fs/xfs/quota/xfs_qm.c +++ b/fs/xfs/quota/xfs_qm.c @@ -450,7 +450,7 @@ xfs_qm_unmount_quotas( STATIC int xfs_qm_dqflush_all( xfs_mount_t *mp, - int flags) + int sync_mode) { int recl; xfs_dquot_t *dqp; @@ -486,7 +486,7 @@ again: * across a disk write. */ xfs_qm_mplist_unlock(mp); - error = xfs_qm_dqflush(dqp, flags); + error = xfs_qm_dqflush(dqp, sync_mode); xfs_dqunlock(dqp); if (error) return error; @@ -926,13 +926,11 @@ xfs_qm_sync( { int recl, restarts; xfs_dquot_t *dqp; - uint flush_flags; int error; if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp)) return 0; - flush_flags = (flags & SYNC_WAIT) ? XFS_QMOPT_SYNC : XFS_QMOPT_DELWRI; restarts = 0; again: @@ -992,7 +990,7 @@ xfs_qm_sync( * across a disk write */ xfs_qm_mplist_unlock(mp); - error = xfs_qm_dqflush(dqp, flush_flags); + error = xfs_qm_dqflush(dqp, flags); xfs_dqunlock(dqp); if (error && XFS_FORCED_SHUTDOWN(mp)) return 0; /* Need to prevent umount failure */ @@ -1796,7 +1794,7 @@ xfs_qm_quotacheck( * successfully. */ if (!error) - error = xfs_qm_dqflush_all(mp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush_all(mp, 0); /* * We can get this error if we couldn't do a dquot allocation inside @@ -2018,7 +2016,7 @@ xfs_qm_shake_freelist( * We flush it delayed write, so don't bother * releasing the mplock. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) { xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dqflush_all: dquot %p flush failed", dqp); @@ -2201,7 +2199,7 @@ xfs_qm_dqreclaim_one(void) * We flush it delayed write, so don't bother * releasing the freelist lock. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) { xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dqreclaim: dquot %p flush failed", dqp); diff --git a/fs/xfs/xfs_quota.h b/fs/xfs/xfs_quota.h index 21d11d9..fdcab3f 100644 --- a/fs/xfs/xfs_quota.h +++ b/fs/xfs/xfs_quota.h @@ -223,15 +223,9 @@ typedef struct xfs_qoff_logformat { #define XFS_QMOPT_RES_INOS 0x0800000 /* - * flags for dqflush and dqflush_all. - */ -#define XFS_QMOPT_SYNC 0x1000000 -#define XFS_QMOPT_DELWRI 0x4000000 - -/* * flags for dqalloc. */ -#define XFS_QMOPT_INHERIT 0x8000000 +#define XFS_QMOPT_INHERIT 0x1000000 /* * flags to xfs_trans_mod_dquot. -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:29 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329SJo204045 for ; Tue, 2 Feb 2010 20:09:28 -0600 X-ASG-Debug-ID: 1265163035-1da700e90000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BEB811371D8C for ; Tue, 2 Feb 2010 18:10:35 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id ONJ5AcT2DhvfFPtz for ; Tue, 02 Feb 2010 18:10:35 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207801-1927428 for ; Wed, 03 Feb 2010 12:40:33 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7h-0003q0-LG for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:33 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7g-0006jB-Iq for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:32 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 10/10] xfs: kill xfs_bawrite Subject: [PATCH 10/10] xfs: kill xfs_bawrite Date: Wed, 3 Feb 2010 10:25:04 +1100 Message-Id: <1265153104-29680-11-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163036 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean There are no more users of this function left in the XFS code now that we've switched everything to delayed write flushing. Remove it. Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_buf.c | 19 ------------------- fs/xfs/linux-2.6/xfs_buf.h | 1 - 2 files changed, 0 insertions(+), 20 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index 4556a4c..d50df3a 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -1078,25 +1078,6 @@ xfs_bwrite( return error; } -int -xfs_bawrite( - void *mp, - struct xfs_buf *bp) -{ - trace_xfs_buf_bawrite(bp, _RET_IP_); - - ASSERT(bp->b_bn != XFS_BUF_DADDR_NULL); - - xfs_buf_delwri_dequeue(bp); - - bp->b_flags &= ~(XBF_READ | XBF_DELWRI | XBF_READ_AHEAD); - bp->b_flags |= (XBF_WRITE | XBF_ASYNC | _XBF_RUN_QUEUES); - - bp->b_mount = mp; - bp->b_strat = xfs_bdstrat_cb; - return xfs_bdstrat_cb(bp); -} - void xfs_bdwrite( void *mp, diff --git a/fs/xfs/linux-2.6/xfs_buf.h b/fs/xfs/linux-2.6/xfs_buf.h index be45e8c..386e736 100644 --- a/fs/xfs/linux-2.6/xfs_buf.h +++ b/fs/xfs/linux-2.6/xfs_buf.h @@ -233,7 +233,6 @@ extern void xfs_buf_unlock(xfs_buf_t *); /* Buffer Read and Write Routines */ extern int xfs_bwrite(struct xfs_mount *mp, struct xfs_buf *bp); -extern int xfs_bawrite(void *mp, xfs_buf_t *bp); extern void xfs_bdwrite(void *mp, xfs_buf_t *bp); extern void xfsbdstrat(struct xfs_mount *, struct xfs_buf *); -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:30 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_66 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329Tp8204061 for ; Tue, 2 Feb 2010 20:09:30 -0600 X-ASG-Debug-ID: 1265163035-1da700e90002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7FCAB1371D5E for ; Tue, 2 Feb 2010 18:10:37 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id sFjwDCzyGoheCGyn for ; Tue, 02 Feb 2010 18:10:37 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207822-1927428 for ; Wed, 03 Feb 2010 12:40:36 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7h-0003pr-BW for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:33 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006iu-9c for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 04/10] xfs: Sort delayed write buffers before dispatch Subject: [PATCH 04/10] xfs: Sort delayed write buffers before dispatch Date: Wed, 3 Feb 2010 10:24:58 +1100 Message-Id: <1265153104-29680-5-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163038 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Currently when the xfsbufd writes delayed write buffers, it pushes them to disk in the order they come off the delayed write list. If there are lots of buffers Ñ•pread widely over the disk, this results in overwhelming the elevator sort queues in the block layer and we end up losing the posibility of merging adjacent buffers to minimise the number of IOs. Use the new generic list_sort function to sort the delwri dispatch queue before issue to ensure that the buffers are pushed in the most friendly order possible to the lower layers. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_buf.c | 87 ++++++++++++++++++++++++++++++-------------- 1 files changed, 60 insertions(+), 27 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index b306265..4556a4c 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "xfs_sb.h" #include "xfs_inum.h" @@ -1877,14 +1878,42 @@ xfs_buf_delwri_split( } +/* + * Compare function is more complex than it needs to be because + * the return value is only 32 bits and we are doing comparisons + * on 64 bit values + */ +static int +xfs_buf_cmp( + void *priv, + struct list_head *a, + struct list_head *b) +{ + struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list); + struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list); + xfs_daddr_t diff; + + diff = ap->b_bn - bp->b_bn; + if (diff < 0) + return -1; + if (diff > 0) + return 1; + return 0; +} + +void +xfs_buf_delwri_sort( + xfs_buftarg_t *target, + struct list_head *list) +{ + list_sort(NULL, list, xfs_buf_cmp); +} + STATIC int xfsbufd( void *data) { - struct list_head tmp; - xfs_buftarg_t *target = (xfs_buftarg_t *)data; - int count; - xfs_buf_t *bp; + xfs_buftarg_t *target = (xfs_buftarg_t *)data; current->flags |= PF_MEMALLOC; @@ -1893,6 +1922,8 @@ xfsbufd( do { long age = xfs_buf_age_centisecs * msecs_to_jiffies(10); long tout = xfs_buf_timer_centisecs * msecs_to_jiffies(10); + int count = 0; + struct list_head tmp; if (unlikely(freezing(current))) { set_bit(XBT_FORCE_SLEEP, &target->bt_flags); @@ -1907,11 +1938,10 @@ xfsbufd( schedule_timeout_interruptible(tout); xfs_buf_delwri_split(target, &tmp, age); - count = 0; + list_sort(NULL, &tmp, xfs_buf_cmp); while (!list_empty(&tmp)) { - bp = list_entry(tmp.next, xfs_buf_t, b_list); - ASSERT(target == bp->b_target); - + struct xfs_buf *bp; + bp = list_first_entry(&tmp, struct xfs_buf, b_list); list_del_init(&bp->b_list); xfs_buf_iostrategy(bp); count++; @@ -1937,42 +1967,45 @@ xfs_flush_buftarg( xfs_buftarg_t *target, int wait) { - struct list_head tmp; - xfs_buf_t *bp, *n; + xfs_buf_t *bp; int pincount = 0; + LIST_HEAD(tmp_list); + LIST_HEAD(wait_list); xfs_buf_runall_queues(xfsconvertd_workqueue); xfs_buf_runall_queues(xfsdatad_workqueue); xfs_buf_runall_queues(xfslogd_workqueue); set_bit(XBT_FORCE_FLUSH, &target->bt_flags); - pincount = xfs_buf_delwri_split(target, &tmp, 0); + pincount = xfs_buf_delwri_split(target, &tmp_list, 0); /* - * Dropped the delayed write list lock, now walk the temporary list + * Dropped the delayed write list lock, now walk the temporary list. + * All I/O is issued async and then if we need to wait for completion + * we do that after issuing all the IO. */ - list_for_each_entry_safe(bp, n, &tmp, b_list) { + list_sort(NULL, &tmp_list, xfs_buf_cmp); + while (!list_empty(&tmp_list)) { + bp = list_first_entry(&tmp_list, struct xfs_buf, b_list); ASSERT(target == bp->b_target); - if (wait) + list_del_init(&bp->b_list); + if (wait) { bp->b_flags &= ~XBF_ASYNC; - else - list_del_init(&bp->b_list); - + list_add(&bp->b_list, &wait_list); + } xfs_buf_iostrategy(bp); } - if (wait) + if (wait) { + /* Expedite and wait for IO to complete. */ blk_run_address_space(target->bt_mapping); + while (!list_empty(&wait_list)) { + bp = list_first_entry(&wait_list, struct xfs_buf, b_list); - /* - * Remaining list items must be flushed before returning - */ - while (!list_empty(&tmp)) { - bp = list_entry(tmp.next, xfs_buf_t, b_list); - - list_del_init(&bp->b_list); - xfs_iowait(bp); - xfs_buf_relse(bp); + list_del_init(&bp->b_list); + xfs_iowait(bp); + xfs_buf_relse(bp); + } } return pincount; -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329V6g204081 for ; Tue, 2 Feb 2010 20:09:31 -0600 X-ASG-Debug-ID: 1265163035-1da700e90004-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 48B591371DC4 for ; Tue, 2 Feb 2010 18:10:39 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id pfGB9etN4N5IBim6 for ; Tue, 02 Feb 2010 18:10:39 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207830-1927428 for ; Wed, 03 Feb 2010 12:40:38 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003nr-KP for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006j9-I7 for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Date: Wed, 3 Feb 2010 10:25:03 +1100 Message-Id: <1265153104-29680-10-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163040 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When an inode has already be flushed delayed write, xfs_inode_clean() returns true and hence xfs_fs_write_inode() can return on a synchronous inode write without having written the inode. Currently these sycnhronous writes only come sync(1), unmount, a sycnhronous NFS export and cachefiles so should be relatively rare and out of common performance paths. Realistically, a synchronous inode write is not necessary here; we can treat this like fsync where we either force the log if there are no unlogged changes, or do a sync transaction if there are unlogged changes. The will result real synchronous semantics as the fsync will issue barriers, but may slow down the above two configurations as a result. However, if the inode is not pinned and has no unlogged changes, then the fsync code is a no-op and hence it may be faster than the existing code. The only thing we need tobe careful of here is if the inode has not yet been flushed to the inode buffer bulkstat scans will fail to see it. Hence even on a sync write, we should try to flush the inode to the backing buffer. This is only a best-effort delwri flush - it doesn't guarantee that the flush happens but races with other inode operations should not happen as the inode lock is not dropped between the fsync operation and the flush. For the asynchronous write, move the clean check until after we have locked up the inode. With the inode locked and the flush lock held, we know that if the inodis clean there are no pending changes in the log and there are no current outstanding delayed writes or IO in progress, so if it reports clean now it really is clean. This matches the order of locking and checks in xfs_sync_inode_attr(). Version 2: - ilock is now held external to the fsync call to close race windows between the fsync and delwri flush to the backing buffer. Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_super.c | 54 ++++++++++++++++++++++++----------------- 1 files changed, 32 insertions(+), 22 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 226fe20..1257a5f 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -1045,35 +1045,45 @@ xfs_fs_write_inode( error = xfs_wait_on_pages(ip, 0, -1); if (error) goto out; - } - - /* - * Bypass inodes which have already been cleaned by - * the inode flush clustering code inside xfs_iflush - */ - if (xfs_inode_clean(ip)) - goto out; - - /* - * We make this non-blocking if the inode is contended, return - * EAGAIN to indicate to the caller that they did not succeed. - * This prevents the flush path from blocking on inodes inside - * another operation right now, they get caught later by xfs_sync. - */ - if (sync) { + /* + * The fsync operation makes inode changes stable and it + * reduces the IOs we have to do here from two (log and inode) + * to just the log. We still need to do a delwri write of the + * inode after this to flush it to the bacing buffer so that + * bulkstat works properly. + */ xfs_ilock(ip, XFS_ILOCK_SHARED); - xfs_iflock(ip); - - error = xfs_iflush(ip, SYNC_WAIT); + error = xfs_fsync(ip, XFS_ILOCK_SHARED); + if (error) + goto out_unlock; + error = EAGAIN; } else { + /* + * We make this non-blocking if the inode is contended, return + * EAGAIN to indicate to the caller that they did not succeed. + * This prevents the flush path from blocking on inodes inside + * another operation right now, they get caught later by xfs_sync. + */ error = EAGAIN; if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) goto out; - if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) - goto out_unlock; + } + + if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) + goto out_unlock; - error = xfs_iflush(ip, 0); + /* + * Now we have the flush lock and the inode is not pinned, we can check + * if the inode is really clean as we know that there are no pending + * transaction completions, it is not waiting on the delayed write + * queue and there is no IO in progress. + */ + error = 0; + if (xfs_inode_clean(ip)) { + xfs_ifunlock(ip); + goto out_unlock; } + error = xfs_iflush(ip, 0); out_unlock: xfs_iunlock(ip, XFS_ILOCK_SHARED); -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329Vao204083 for ; Tue, 2 Feb 2010 20:09:31 -0600 X-ASG-Debug-ID: 1265163035-525703e10004-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C404C1A72E5 for ; Tue, 2 Feb 2010 18:10:39 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id oHQh4ljxcCsG2Eho for ; Tue, 02 Feb 2010 18:10:39 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207816-1927428 for ; Wed, 03 Feb 2010 12:40:35 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7h-0003ps-Cf for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:33 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006ix-BH for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 05/10] xfs: Use delay write promotion for dquot flushing Subject: [PATCH 05/10] xfs: Use delay write promotion for dquot flushing Date: Wed, 3 Feb 2010 10:24:59 +1100 Message-Id: <1265153104-29680-6-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163040 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean xfs_qm_dqflock_pushbuf_wait() does a very similar trick to item pushing used to do to flush out delayed write dquot buffers. Change it to use the new promotion method rather than an async flush. Also, xfs_qm_dqflock_pushbuf_wait() can return without the flush lock held, yet the callers make the assumption that after this call the flush lock is held. Always return with the flush lock held. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/quota/xfs_dquot.c | 25 ++++++++++--------------- 1 files changed, 10 insertions(+), 15 deletions(-) diff --git a/fs/xfs/quota/xfs_dquot.c b/fs/xfs/quota/xfs_dquot.c index f9baeed..1620a56 100644 --- a/fs/xfs/quota/xfs_dquot.c +++ b/fs/xfs/quota/xfs_dquot.c @@ -1528,21 +1528,16 @@ xfs_qm_dqflock_pushbuf_wait( */ bp = xfs_incore(dqp->q_mount->m_ddev_targp, dqp->q_blkno, XFS_QI_DQCHUNKLEN(dqp->q_mount), XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - int error; - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(dqp->q_mount, 0); - error = xfs_bawrite(dqp->q_mount, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, dqp->q_mount, - "xfs_qm_dqflock_pushbuf_wait: " - "pushbuf error %d on dqp %p, bp %p", - error, dqp, bp); - } else { - xfs_buf_relse(bp); - } + if (!bp) + goto out_lock; + + if (XFS_BUF_ISDELAYWRITE(bp)) { + if (XFS_BUF_ISPINNED(bp)) + xfs_log_force(dqp->q_mount, 0); + xfs_buf_delwri_promote(bp); + wake_up_process(bp->b_target->bt_task); } + xfs_buf_relse(bp); +out_lock: xfs_dqflock(dqp); } -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:33 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329X2P204105 for ; Tue, 2 Feb 2010 20:09:33 -0600 X-ASG-Debug-ID: 1265163035-525703e10006-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5BFD21A72E9 for ; Tue, 2 Feb 2010 18:10:41 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id GNDM3Qz4JB3ughFQ for ; Tue, 02 Feb 2010 18:10:41 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207846-1927428 for ; Wed, 03 Feb 2010 12:40:40 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003ne-7o for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006ik-3V for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 01/10] xfs: Make inode reclaim states explicit Subject: [PATCH 01/10] xfs: Make inode reclaim states explicit Date: Wed, 3 Feb 2010 10:24:55 +1100 Message-Id: <1265153104-29680-2-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163042 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean A.K.A.: don't rely on xfs_iflush() return value in reclaim We have gradually been moving checks out of the reclaim code because they are duplicated in xfs_iflush(). We've had a history of problems in this area, and many of them stem from the overloading of the return values from xfs_iflush() and interaction with inode flush locking to determine if the inode is safe to reclaim. With the desire to move to delayed write flushing of inodes and non-blocking inode tree reclaim walks, the overloading of the return value of xfs_iflush makes it very difficult to determine the correct thing to do next. This patch explicitly re-adds the checks to the inode reclaim code, removing the reliance on the return value of xfs_iflush() to determine what to do next. It also means that we can clearly document all the inode states that reclaim must handle and hence we can easily see that we handled all the necessary cases. This also removes the need for the xfs_inode_clean() check in xfs_iflush() as all callers now check this first (safely). Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_sync.c | 80 ++++++++++++++++++++++++++++++++---------- fs/xfs/xfs_inode.c | 11 +----- fs/xfs/xfs_inode.h | 1 + 3 files changed, 63 insertions(+), 29 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c index c9b863e..98b8937 100644 --- a/fs/xfs/linux-2.6/xfs_sync.c +++ b/fs/xfs/linux-2.6/xfs_sync.c @@ -706,12 +706,43 @@ __xfs_inode_clear_reclaim_tag( XFS_INO_TO_AGINO(mp, ip->i_ino), XFS_ICI_RECLAIM_TAG); } +/* + * Inodes in different states need to be treated differently, and the return + * value of xfs_iflush is not sufficient to get this right. The following table + * lists the inode states and the reclaim actions necessary for non-blocking + * reclaim: + * + * + * inode state iflush ret required action + * --------------- ---------- --------------- + * bad - reclaim + * shutdown EIO unpin and reclaim + * clean, unpinned 0 reclaim + * stale, unpinned 0 reclaim + * clean, pinned(*) 0 unpin and reclaim + * stale, pinned 0 unpin and reclaim + * dirty, async 0 block on flush lock, reclaim + * dirty, sync flush 0 block on flush lock, reclaim + * + * (*) dgc: I don't think the clean, pinned state is possible but it gets + * handled anyway given the order of checks implemented. + * + * Hence the order of actions after gaining the locks should be: + * bad => reclaim + * shutdown => unpin and reclaim + * pinned => unpin + * stale => reclaim + * clean => reclaim + * dirty => flush, wait and reclaim + */ STATIC int xfs_reclaim_inode( struct xfs_inode *ip, struct xfs_perag *pag, int sync_mode) { + int error; + /* * The radix tree lock here protects a thread in xfs_iget from racing * with us starting reclaim on the inode. Once we have the @@ -729,30 +760,41 @@ xfs_reclaim_inode( spin_unlock(&ip->i_flags_lock); write_unlock(&pag->pag_ici_lock); - /* - * If the inode is still dirty, then flush it out. If the inode - * is not in the AIL, then it will be OK to flush it delwri as - * long as xfs_iflush() does not keep any references to the inode. - * We leave that decision up to xfs_iflush() since it has the - * knowledge of whether it's OK to simply do a delwri flush of - * the inode or whether we need to wait until the inode is - * pulled from the AIL. - * We get the flush lock regardless, though, just to make sure - * we don't free it while it is being flushed. - */ xfs_ilock(ip, XFS_ILOCK_EXCL); xfs_iflock(ip); - /* - * In the case of a forced shutdown we rely on xfs_iflush() to - * wait for the inode to be unpinned before returning an error. - */ - if (!is_bad_inode(VFS_I(ip)) && xfs_iflush(ip, sync_mode) == 0) { - /* synchronize with xfs_iflush_done */ - xfs_iflock(ip); - xfs_ifunlock(ip); + if (is_bad_inode(VFS_I(ip))) + goto reclaim; + if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { + xfs_iunpin_wait(ip); + goto reclaim; + } + if (xfs_ipincount(ip)) + xfs_iunpin_wait(ip); + if (xfs_iflags_test(ip, XFS_ISTALE)) + goto reclaim; + if (xfs_inode_clean(ip)) + goto reclaim; + + /* Now we have an inode that needs flushing */ + error = xfs_iflush(ip, sync_mode); + if (!error) { + switch(sync_mode) { + case XFS_IFLUSH_DELWRI_ELSE_ASYNC: + case XFS_IFLUSH_DELWRI: + case XFS_IFLUSH_ASYNC: + case XFS_IFLUSH_DELWRI_ELSE_SYNC: + case XFS_IFLUSH_SYNC: + /* IO issued, synchronise with IO completion */ + xfs_iflock(ip); + default: + ASSERT(0); + break; + } } +reclaim: + xfs_ifunlock(ip); xfs_iunlock(ip, XFS_ILOCK_EXCL); xfs_ireclaim(ip); return 0; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index d0d1b5a..8d0666d 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2493,7 +2493,7 @@ __xfs_iunpin_wait( wait_event(ip->i_ipin_wait, (atomic_read(&ip->i_pincount) == 0)); } -static inline void +void xfs_iunpin_wait( xfs_inode_t *ip) { @@ -2849,15 +2849,6 @@ xfs_iflush( mp = ip->i_mount; /* - * If the inode isn't dirty, then just release the inode flush lock and - * do nothing. - */ - if (xfs_inode_clean(ip)) { - xfs_ifunlock(ip); - return 0; - } - - /* * We can't flush the inode until it is unpinned, so wait for it if we * are allowed to block. We know noone new can pin it, because we are * holding the inode lock shared and you need to hold it exclusively to diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index ec1f28c..8b618ea 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -483,6 +483,7 @@ int xfs_iunlink(struct xfs_trans *, xfs_inode_t *); void xfs_iext_realloc(xfs_inode_t *, int, int); void xfs_ipin(xfs_inode_t *); void xfs_iunpin(xfs_inode_t *); +void xfs_iunpin_wait(xfs_inode_t *); int xfs_iflush(xfs_inode_t *, uint); void xfs_ichgtime(xfs_inode_t *, int); void xfs_lock_inodes(xfs_inode_t **, int, uint); -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:35 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329Ua9204063 for ; Tue, 2 Feb 2010 20:09:35 -0600 X-ASG-Debug-ID: 1265163035-525703e10002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1C9981A72E5 for ; Tue, 2 Feb 2010 18:10:38 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id HSHzon25eAb5NBWs for ; Tue, 02 Feb 2010 18:10:38 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207825-1927428 for ; Wed, 03 Feb 2010 12:40:37 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003nq-IY for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006j6-GI for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Subject: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Date: Wed, 3 Feb 2010 10:25:02 +1100 Message-Id: <1265153104-29680-9-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163039 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We have a need for a delayed write inode flush operation to be made atomically with an fsync to avoid physically writing inodes but still keeping inode buffer information up to date for bulkstat. Move the inode locking outside xfs_fsync() to allow this to be done. Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_file.c | 6 +++++- fs/xfs/linux-2.6/xfs_lrw.c | 5 +++-- fs/xfs/xfs_vnodeops.c | 34 ++++++++++++++++------------------ fs/xfs/xfs_vnodeops.h | 2 +- 4 files changed, 25 insertions(+), 22 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_file.c b/fs/xfs/linux-2.6/xfs_file.c index e4caeb2..a2c8321 100644 --- a/fs/xfs/linux-2.6/xfs_file.c +++ b/fs/xfs/linux-2.6/xfs_file.c @@ -177,9 +177,13 @@ xfs_file_fsync( int datasync) { struct xfs_inode *ip = XFS_I(dentry->d_inode); + int error; xfs_iflags_clear(ip, XFS_ITRUNCATED); - return -xfs_fsync(ip); + xfs_ilock(ip, XFS_ILOCK_SHARED); + error = -xfs_fsync(ip, XFS_ILOCK_SHARED); + xfs_iunlock(ip, XFS_ILOCK_SHARED); + return error; } STATIC int diff --git a/fs/xfs/linux-2.6/xfs_lrw.c b/fs/xfs/linux-2.6/xfs_lrw.c index c80fa00..508135c 100644 --- a/fs/xfs/linux-2.6/xfs_lrw.c +++ b/fs/xfs/linux-2.6/xfs_lrw.c @@ -754,11 +754,12 @@ write_retry: error = error2; if (need_i_mutex) mutex_lock(&inode->i_mutex); - xfs_ilock(xip, iolock); + xfs_ilock(xip, iolock | XFS_ILOCK_SHARED); - error2 = xfs_fsync(xip); + error2 = xfs_fsync(xip, XFS_ILOCK_SHARED); if (!error) error = error2; + xfs_iunlock(xip, XFS_ILOCK_SHARED); } out_unlock_internal: diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index 43241e2..afcb635 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -590,10 +590,22 @@ xfs_readlink( * the I/O lock while flushing the data, and the inode lock while flushing the * inode. The inode lock CANNOT be held while flushing the data, so acquire * after we're done with that. + * + * We always need to make sure that the required inode state is safe on disk. + * The inode might be clean but we still might need to force the log because of + * committed transactions that haven't hit the disk yet. Likewise, there could + * be unflushed non-transactional changes to the inode core that have to go to + * disk and this requires us to issue a synchronous transaction to capture + * these changes correctly. + * + * This code relies on the assumption that if the update_* fields of the inode + * are clear and the inode is unpinned then it is clean and no action is + * required. */ int xfs_fsync( - xfs_inode_t *ip) + xfs_inode_t *ip, + int lock_mode) { xfs_trans_t *tp; int error = 0; @@ -604,20 +616,6 @@ xfs_fsync( if (XFS_FORCED_SHUTDOWN(ip->i_mount)) return XFS_ERROR(EIO); - /* - * We always need to make sure that the required inode state is safe on - * disk. The inode might be clean but we still might need to force the - * log because of committed transactions that haven't hit the disk yet. - * Likewise, there could be unflushed non-transactional changes to the - * inode core that have to go to disk and this requires us to issue - * a synchronous transaction to capture these changes correctly. - * - * This code relies on the assumption that if the update_* fields - * of the inode are clear and the inode is unpinned then it is clean - * and no action is required. - */ - xfs_ilock(ip, XFS_ILOCK_SHARED); - if (!ip->i_update_core) { /* * Timestamps/size haven't changed since last inode flush or @@ -627,7 +625,6 @@ xfs_fsync( * disk yet, the inode will be still be pinned. If it is, * force the log. */ - xfs_iunlock(ip, XFS_ILOCK_SHARED); if (xfs_ipincount(ip)) { error = _xfs_log_force(ip->i_mount, XFS_LOG_SYNC, &log_flushed); @@ -637,7 +634,7 @@ xfs_fsync( * Kick off a transaction to log the inode core to get the * updates. The sync transaction will also force the log. */ - xfs_iunlock(ip, XFS_ILOCK_SHARED); + xfs_iunlock(ip, lock_mode); tp = xfs_trans_alloc(ip->i_mount, XFS_TRANS_FSYNC_TS); error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(ip->i_mount), 0, 0, 0); @@ -662,7 +659,8 @@ xfs_fsync( xfs_trans_set_sync(tp); error = _xfs_trans_commit(tp, 0, &log_flushed); - xfs_iunlock(ip, XFS_ILOCK_EXCL); + if (lock_mode != XFS_ILOCK_EXCL) + xfs_ilock_demote(ip, XFS_ILOCK_EXCL); } if (ip->i_mount->m_flags & XFS_MOUNT_BARRIER) { diff --git a/fs/xfs/xfs_vnodeops.h b/fs/xfs/xfs_vnodeops.h index 774f407..eacd297 100644 --- a/fs/xfs/xfs_vnodeops.h +++ b/fs/xfs/xfs_vnodeops.h @@ -21,7 +21,7 @@ int xfs_setattr(struct xfs_inode *ip, struct iattr *vap, int flags); #define XFS_ATTR_NOACL 0x08 /* Don't call xfs_acl_chmod */ int xfs_readlink(struct xfs_inode *ip, char *link); -int xfs_fsync(struct xfs_inode *ip); +int xfs_fsync(struct xfs_inode *ip, int lock_mode); int xfs_release(struct xfs_inode *ip); int xfs_inactive(struct xfs_inode *ip); int xfs_lookup(struct xfs_inode *dp, struct xfs_name *name, -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:34 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329XiD204108 for ; Tue, 2 Feb 2010 20:09:33 -0600 X-ASG-Debug-ID: 1265163040-163b021e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D195B1C9B81C for ; Tue, 2 Feb 2010 18:10:41 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id cm9XUFBOHV79s4LF for ; Tue, 02 Feb 2010 18:10:41 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207807-1927428 for multiple; Wed, 03 Feb 2010 12:40:34 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7h-0003pu-H3; Wed, 03 Feb 2010 10:25:33 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006j3-Ee; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: Christoph Hellwig X-ASG-Orig-Subj: [PATCH 07/10] xfs: remove invalid barrier optimization from xfs_fsync Subject: [PATCH 07/10] xfs: remove invalid barrier optimization from xfs_fsync Date: Wed, 3 Feb 2010 10:25:01 +1100 Message-Id: <1265153104-29680-8-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163042 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean From: Christoph Hellwig We always need to flush the disk write cache and can't skip it just because the no inode attributes have changed. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Signed-off-by: Dave Chinner --- fs/xfs/xfs_vnodeops.c | 12 ++---------- 1 files changed, 2 insertions(+), 10 deletions(-) diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index fd108b7..43241e2 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -597,7 +597,7 @@ xfs_fsync( { xfs_trans_t *tp; int error = 0; - int log_flushed = 0, changed = 1; + int log_flushed = 0; xfs_itrace_entry(ip); @@ -627,18 +627,10 @@ xfs_fsync( * disk yet, the inode will be still be pinned. If it is, * force the log. */ - xfs_iunlock(ip, XFS_ILOCK_SHARED); - if (xfs_ipincount(ip)) { error = _xfs_log_force(ip->i_mount, XFS_LOG_SYNC, &log_flushed); - } else { - /* - * If the inode is not pinned and nothing has changed - * we don't need to flush the cache. - */ - changed = 0; } } else { /* @@ -673,7 +665,7 @@ xfs_fsync( xfs_iunlock(ip, XFS_ILOCK_EXCL); } - if ((ip->i_mount->m_flags & XFS_MOUNT_BARRIER) && changed) { + if (ip->i_mount->m_flags & XFS_MOUNT_BARRIER) { /* * If the log write didn't issue an ordered tag we need * to flush the disk cache for the data device now. -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:37 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329a2a204153 for ; Tue, 2 Feb 2010 20:09:37 -0600 X-ASG-Debug-ID: 1265163035-1da700e90006-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0072E1371DD6 for ; Tue, 2 Feb 2010 18:10:40 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id SfLQNDwwuLxHIoBk for ; Tue, 02 Feb 2010 18:10:40 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207832-1927428 for ; Wed, 03 Feb 2010 12:40:39 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003ng-AP for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006iq-7w for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 03/10] xfs: Don't issue buffer IO direct from AIL push V2 Subject: [PATCH 03/10] xfs: Don't issue buffer IO direct from AIL push V2 Date: Wed, 3 Feb 2010 10:24:57 +1100 Message-Id: <1265153104-29680-4-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163042 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean All buffers logged into the AIL are marked as delayed write. When the AIL needs to push the buffer out, it issues an async write of the buffer. This means that IO patterns are dependent on the order of buffers in the AIL. Instead of flushing the buffer, promote the buffer in the delayed write list so that the next time the xfsbufd is run the buffer will be flushed by the xfsbufd. Return the state to the xfsaild that the buffer was promoted so that the xfsaild knows that it needs to cause the xfsbufd to run to flush the buffers that were promoted. Using the xfsbufd for issuing the IO allows us to dispatch all buffer IO from the one queue. This means that we can make much more enlightened decisions on what order to flush buffers to disk as we don't have multiple places issuing IO. Optimisations to xfsbufd will be in a future patch. Version 2 - kill XFS_ITEM_FLUSHING as it is now unused. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_buf.c | 29 ++++++++++++ fs/xfs/linux-2.6/xfs_buf.h | 2 + fs/xfs/linux-2.6/xfs_trace.h | 1 + fs/xfs/quota/xfs_dquot_item.c | 85 +++++------------------------------ fs/xfs/quota/xfs_dquot_item.h | 4 -- fs/xfs/xfs_buf_item.c | 64 +++++++++++++++------------ fs/xfs/xfs_inode_item.c | 98 ++++++---------------------------------- fs/xfs/xfs_inode_item.h | 6 --- fs/xfs/xfs_trans.h | 3 +- fs/xfs/xfs_trans_ail.c | 13 +++--- 10 files changed, 102 insertions(+), 203 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index 44e20e5..b306265 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -1778,6 +1778,35 @@ xfs_buf_delwri_dequeue( trace_xfs_buf_delwri_dequeue(bp, _RET_IP_); } +/* + * If a delwri buffer needs to be pushed before it has aged out, then promote + * it to the head of the delwri queue so that it will be flushed on the next + * xfsbufd run. We do this by resetting the queuetime of the buffer to be older + * than the age currently needed to flush the buffer. Hence the next time the + * xfsbufd sees it is guaranteed to be considered old enough to flush. + */ +void +xfs_buf_delwri_promote( + struct xfs_buf *bp) +{ + struct xfs_buftarg *btp = bp->b_target; + long age = xfs_buf_age_centisecs * msecs_to_jiffies(10) + 1; + + ASSERT(bp->b_flags & XBF_DELWRI); + ASSERT(bp->b_flags & _XBF_DELWRI_Q); + + /* + * Check the buffer age before locking the delayed write queue as we + * don't need to promote buffers that are already past the flush age. + */ + if (bp->b_queuetime < jiffies - age) + return; + bp->b_queuetime = jiffies - age; + spin_lock(&btp->bt_delwrite_lock); + list_move(&bp->b_list, &btp->bt_delwrite_queue); + spin_unlock(&btp->bt_delwrite_lock); +} + STATIC void xfs_buf_runall_queues( struct workqueue_struct *queue) diff --git a/fs/xfs/linux-2.6/xfs_buf.h b/fs/xfs/linux-2.6/xfs_buf.h index ea8c198..be45e8c 100644 --- a/fs/xfs/linux-2.6/xfs_buf.h +++ b/fs/xfs/linux-2.6/xfs_buf.h @@ -266,6 +266,7 @@ extern int xfs_buf_ispin(xfs_buf_t *); /* Delayed Write Buffer Routines */ extern void xfs_buf_delwri_dequeue(xfs_buf_t *); +extern void xfs_buf_delwri_promote(xfs_buf_t *); /* Buffer Daemon Setup Routines */ extern int xfs_buf_init(void); @@ -395,6 +396,7 @@ extern void xfs_free_buftarg(struct xfs_mount *, struct xfs_buftarg *); extern void xfs_wait_buftarg(xfs_buftarg_t *); extern int xfs_setsize_buftarg(xfs_buftarg_t *, unsigned int, unsigned int); extern int xfs_flush_buftarg(xfs_buftarg_t *, int); + #ifdef CONFIG_KDB_MODULES extern struct list_head *xfs_get_buftarg_list(void); #endif diff --git a/fs/xfs/linux-2.6/xfs_trace.h b/fs/xfs/linux-2.6/xfs_trace.h index 1bb09e7..a4574dc 100644 --- a/fs/xfs/linux-2.6/xfs_trace.h +++ b/fs/xfs/linux-2.6/xfs_trace.h @@ -483,6 +483,7 @@ DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock_stale); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_committed); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_push); +DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pushbuf); DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf); DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf_recur); DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb); diff --git a/fs/xfs/quota/xfs_dquot_item.c b/fs/xfs/quota/xfs_dquot_item.c index 1b56437..dda0fb0 100644 --- a/fs/xfs/quota/xfs_dquot_item.c +++ b/fs/xfs/quota/xfs_dquot_item.c @@ -212,66 +212,31 @@ xfs_qm_dquot_logitem_pushbuf( xfs_dquot_t *dqp; xfs_mount_t *mp; xfs_buf_t *bp; - uint dopush; dqp = qip->qli_dquot; ASSERT(XFS_DQ_IS_LOCKED(dqp)); /* - * The qli_pushbuf_flag keeps others from - * trying to duplicate our effort. - */ - ASSERT(qip->qli_pushbuf_flag != 0); - ASSERT(qip->qli_push_owner == current_pid()); - - /* * If flushlock isn't locked anymore, chances are that the * inode flush completed and the inode was taken off the AIL. * So, just get out. */ if (completion_done(&dqp->q_flush) || ((qip->qli_item.li_flags & XFS_LI_IN_AIL) == 0)) { - qip->qli_pushbuf_flag = 0; xfs_dqunlock(dqp); return; } mp = dqp->q_mount; bp = xfs_incore(mp->m_ddev_targp, qip->qli_format.qlf_blkno, XFS_QI_DQCHUNKLEN(mp), XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - dopush = ((qip->qli_item.li_flags & XFS_LI_IN_AIL) && - !completion_done(&dqp->q_flush)); - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(mp, 0); - - if (dopush) { - int error; -#ifdef XFSRACEDEBUG - delay_for_intr(); - delay(300); -#endif - error = xfs_bawrite(mp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, mp, - "xfs_qm_dquot_logitem_pushbuf: pushbuf error %d on qip %p, bp %p", - error, qip, bp); - } else { - xfs_buf_relse(bp); - } - } else { - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); - xfs_buf_relse(bp); - } + xfs_dqunlock(dqp); + if (!bp) return; - } + if (XFS_BUF_ISDELAYWRITE(bp)) + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); + return; - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); } /* @@ -289,50 +254,24 @@ xfs_qm_dquot_logitem_trylock( xfs_dq_logitem_t *qip) { xfs_dquot_t *dqp; - uint retval; dqp = qip->qli_dquot; if (atomic_read(&dqp->q_pincount) > 0) - return (XFS_ITEM_PINNED); + return XFS_ITEM_PINNED; if (! xfs_qm_dqlock_nowait(dqp)) - return (XFS_ITEM_LOCKED); + return XFS_ITEM_LOCKED; - retval = XFS_ITEM_SUCCESS; if (!xfs_dqflock_nowait(dqp)) { /* - * The dquot is already being flushed. It may have been - * flushed delayed write, however, and we don't want to - * get stuck waiting for that to complete. So, we want to check - * to see if we can lock the dquot's buffer without sleeping. - * If we can and it is marked for delayed write, then we - * hold it and send it out from the push routine. We don't - * want to do that now since we might sleep in the device - * strategy routine. We also don't want to grab the buffer lock - * here because we'd like not to call into the buffer cache - * while holding the AIL lock. - * Make sure to only return PUSHBUF if we set pushbuf_flag - * ourselves. If someone else is doing it then we don't - * want to go to the push routine and duplicate their efforts. + * dquot has already been flushed to the backing buffer, + * leave it locked, pushbuf routine will unlock it. */ - if (qip->qli_pushbuf_flag == 0) { - qip->qli_pushbuf_flag = 1; - ASSERT(qip->qli_format.qlf_blkno == dqp->q_blkno); -#ifdef DEBUG - qip->qli_push_owner = current_pid(); -#endif - /* - * The dquot is left locked. - */ - retval = XFS_ITEM_PUSHBUF; - } else { - retval = XFS_ITEM_FLUSHING; - xfs_dqunlock_nonotify(dqp); - } + return XFS_ITEM_PUSHBUF; } ASSERT(qip->qli_item.li_flags & XFS_LI_IN_AIL); - return (retval); + return XFS_ITEM_SUCCESS; } diff --git a/fs/xfs/quota/xfs_dquot_item.h b/fs/xfs/quota/xfs_dquot_item.h index 5a63253..5acae2a 100644 --- a/fs/xfs/quota/xfs_dquot_item.h +++ b/fs/xfs/quota/xfs_dquot_item.h @@ -27,10 +27,6 @@ typedef struct xfs_dq_logitem { xfs_log_item_t qli_item; /* common portion */ struct xfs_dquot *qli_dquot; /* dquot ptr */ xfs_lsn_t qli_flush_lsn; /* lsn at last flush */ - unsigned short qli_pushbuf_flag; /* 1 bit used in push_ail */ -#ifdef DEBUG - uint64_t qli_push_owner; -#endif xfs_dq_logformat_t qli_format; /* logged structure */ } xfs_dq_logitem_t; diff --git a/fs/xfs/xfs_buf_item.c b/fs/xfs/xfs_buf_item.c index e0a1158..f3c49e6 100644 --- a/fs/xfs/xfs_buf_item.c +++ b/fs/xfs/xfs_buf_item.c @@ -467,8 +467,10 @@ xfs_buf_item_unpin_remove( /* * This is called to attempt to lock the buffer associated with this * buf log item. Don't sleep on the buffer lock. If we can't get - * the lock right away, return 0. If we can get the lock, pull the - * buffer from the free list, mark it busy, and return 1. + * the lock right away, return 0. If we can get the lock, take a + * reference to the buffer. If this is a delayed write buffer that + * needs AIL help to be written back, invoke the pushbuf routine + * rather than the normal success path. */ STATIC uint xfs_buf_item_trylock( @@ -477,24 +479,18 @@ xfs_buf_item_trylock( xfs_buf_t *bp; bp = bip->bli_buf; - - if (XFS_BUF_ISPINNED(bp)) { + if (XFS_BUF_ISPINNED(bp)) return XFS_ITEM_PINNED; - } - - if (!XFS_BUF_CPSEMA(bp)) { + if (!XFS_BUF_CPSEMA(bp)) return XFS_ITEM_LOCKED; - } - /* - * Remove the buffer from the free list. Only do this - * if it's on the free list. Private buffers like the - * superblock buffer are not. - */ + /* take a reference to the buffer. */ XFS_BUF_HOLD(bp); ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); trace_xfs_buf_item_trylock(bip); + if (XFS_BUF_ISDELAYWRITE(bp)) + return XFS_ITEM_PUSHBUF; return XFS_ITEM_SUCCESS; } @@ -626,11 +622,9 @@ xfs_buf_item_committed( } /* - * This is called to asynchronously write the buffer associated with this - * buf log item out to disk. The buffer will already have been locked by - * a successful call to xfs_buf_item_trylock(). If the buffer still has - * B_DELWRI set, then get it going out to disk with a call to bawrite(). - * If not, then just release the buffer. + * The buffer is locked, but is not a delayed write buffer. This happens + * if we race with IO completion and hence we don't want to try to write it + * again. Just release the buffer. */ STATIC void xfs_buf_item_push( @@ -642,17 +636,29 @@ xfs_buf_item_push( trace_xfs_buf_item_push(bip); bp = bip->bli_buf; + ASSERT(!XFS_BUF_ISDELAYWRITE(bp)); + xfs_buf_relse(bp); +} - if (XFS_BUF_ISDELAYWRITE(bp)) { - int error; - error = xfs_bawrite(bip->bli_item.li_mountp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, bip->bli_item.li_mountp, - "xfs_buf_item_push: pushbuf error %d on bip %p, bp %p", - error, bip, bp); - } else { - xfs_buf_relse(bp); - } +/* + * The buffer is locked and is a delayed write buffer. Promote the buffer + * in the delayed write queue as the caller knows that they must invoke + * the xfsbufd to get this buffer written. We have to unlock the buffer + * to allow the xfsbufd to write it, too. + */ +STATIC void +xfs_buf_item_pushbuf( + xfs_buf_log_item_t *bip) +{ + xfs_buf_t *bp; + + ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); + trace_xfs_buf_item_pushbuf(bip); + + bp = bip->bli_buf; + ASSERT(XFS_BUF_ISDELAYWRITE(bp)); + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); } /* ARGSUSED */ @@ -677,7 +683,7 @@ static struct xfs_item_ops xfs_buf_item_ops = { .iop_committed = (xfs_lsn_t(*)(xfs_log_item_t*, xfs_lsn_t)) xfs_buf_item_committed, .iop_push = (void(*)(xfs_log_item_t*))xfs_buf_item_push, - .iop_pushbuf = NULL, + .iop_pushbuf = (void(*)(xfs_log_item_t*))xfs_buf_item_pushbuf, .iop_committing = (void(*)(xfs_log_item_t*, xfs_lsn_t)) xfs_buf_item_committing }; diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index 207553e..d4dc063 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -602,33 +602,20 @@ xfs_inode_item_trylock( if (!xfs_iflock_nowait(ip)) { /* - * If someone else isn't already trying to push the inode - * buffer, we get to do it. + * inode has already been flushed to the backing buffer, + * leave it locked in shared mode, pushbuf routine will + * unlock it. */ - if (iip->ili_pushbuf_flag == 0) { - iip->ili_pushbuf_flag = 1; -#ifdef DEBUG - iip->ili_push_owner = current_pid(); -#endif - /* - * Inode is left locked in shared mode. - * Pushbuf routine gets to unlock it. - */ - return XFS_ITEM_PUSHBUF; - } else { - /* - * We hold the AIL lock, so we must specify the - * NONOTIFY flag so that we won't double trip. - */ - xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); - return XFS_ITEM_FLUSHING; - } - /* NOTREACHED */ + return XFS_ITEM_PUSHBUF; } /* Stale items should force out the iclog */ if (ip->i_flags & XFS_ISTALE) { xfs_ifunlock(ip); + /* + * we hold the AIL lock - notify the unlock routine of this + * so it doesn't try to get the lock again. + */ xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); return XFS_ITEM_PINNED; } @@ -746,11 +733,8 @@ xfs_inode_item_committed( * This gets called by xfs_trans_push_ail(), when IOP_TRYLOCK * failed to get the inode flush lock but did get the inode locked SHARED. * Here we're trying to see if the inode buffer is incore, and if so whether it's - * marked delayed write. If that's the case, we'll initiate a bawrite on that - * buffer to expedite the process. - * - * We aren't holding the AIL lock (or the flush lock) when this gets called, - * so it is inherently race-y. + * marked delayed write. If that's the case, we'll promote it and that will + * allow the caller to write the buffer by triggering the xfsbufd to run. */ STATIC void xfs_inode_item_pushbuf( @@ -759,26 +743,16 @@ xfs_inode_item_pushbuf( xfs_inode_t *ip; xfs_mount_t *mp; xfs_buf_t *bp; - uint dopush; ip = iip->ili_inode; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED)); /* - * The ili_pushbuf_flag keeps others from - * trying to duplicate our effort. - */ - ASSERT(iip->ili_pushbuf_flag != 0); - ASSERT(iip->ili_push_owner == current_pid()); - - /* * If a flush is not in progress anymore, chances are that the * inode was taken off the AIL. So, just get out. */ if (completion_done(&ip->i_flush) || ((iip->ili_item.li_flags & XFS_LI_IN_AIL) == 0)) { - iip->ili_pushbuf_flag = 0; xfs_iunlock(ip, XFS_ILOCK_SHARED); return; } @@ -787,53 +761,12 @@ xfs_inode_item_pushbuf( bp = xfs_incore(mp->m_ddev_targp, iip->ili_format.ilf_blkno, iip->ili_format.ilf_len, XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - /* - * We were racing with iflush because we don't hold - * the AIL lock or the flush lock. However, at this point, - * we have the buffer, and we know that it's dirty. - * So, it's possible that iflush raced with us, and - * this item is already taken off the AIL. - * If not, we can flush it async. - */ - dopush = ((iip->ili_item.li_flags & XFS_LI_IN_AIL) && - !completion_done(&ip->i_flush)); - iip->ili_pushbuf_flag = 0; - xfs_iunlock(ip, XFS_ILOCK_SHARED); - - trace_xfs_inode_item_push(bp, _RET_IP_); - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(mp, 0); - - if (dopush) { - int error; - error = xfs_bawrite(mp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, mp, - "xfs_inode_item_pushbuf: pushbuf error %d on iip %p, bp %p", - error, iip, bp); - } else { - xfs_buf_relse(bp); - } - } else { - iip->ili_pushbuf_flag = 0; - xfs_iunlock(ip, XFS_ILOCK_SHARED); - xfs_buf_relse(bp); - } - return; - } - /* - * We have to be careful about resetting pushbuf flag too early (above). - * Even though in theory we can do it as soon as we have the buflock, - * we don't want others to be doing work needlessly. They'll come to - * this function thinking that pushing the buffer is their - * responsibility only to find that the buffer is still locked by - * another doing the same thing - */ - iip->ili_pushbuf_flag = 0; xfs_iunlock(ip, XFS_ILOCK_SHARED); + if (!bp) + return; + if (XFS_BUF_ISDELAYWRITE(bp)) + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); return; } @@ -937,7 +870,6 @@ xfs_inode_item_init( /* We have zeroed memory. No need ... iip->ili_extents_buf = NULL; - iip->ili_pushbuf_flag = 0; */ iip->ili_format.ilf_type = XFS_LI_INODE; diff --git a/fs/xfs/xfs_inode_item.h b/fs/xfs/xfs_inode_item.h index cc8df1a..9a46795 100644 --- a/fs/xfs/xfs_inode_item.h +++ b/fs/xfs/xfs_inode_item.h @@ -144,12 +144,6 @@ typedef struct xfs_inode_log_item { data exts */ struct xfs_bmbt_rec *ili_aextents_buf; /* array of logged attr exts */ - unsigned int ili_pushbuf_flag; /* one bit used in push_ail */ - -#ifdef DEBUG - uint64_t ili_push_owner; /* one who sets pushbuf_flag - above gets to push the buf */ -#endif #ifdef XFS_TRANS_DEBUG int ili_root_size; char *ili_orig_root; diff --git a/fs/xfs/xfs_trans.h b/fs/xfs/xfs_trans.h index ca64f33..c93e3a1 100644 --- a/fs/xfs/xfs_trans.h +++ b/fs/xfs/xfs_trans.h @@ -861,8 +861,7 @@ typedef struct xfs_item_ops { #define XFS_ITEM_SUCCESS 0 #define XFS_ITEM_PINNED 1 #define XFS_ITEM_LOCKED 2 -#define XFS_ITEM_FLUSHING 3 -#define XFS_ITEM_PUSHBUF 4 +#define XFS_ITEM_PUSHBUF 3 /* * This structure is used to maintain a list of block ranges that have been diff --git a/fs/xfs/xfs_trans_ail.c b/fs/xfs/xfs_trans_ail.c index d7b1af8..e799824 100644 --- a/fs/xfs/xfs_trans_ail.c +++ b/fs/xfs/xfs_trans_ail.c @@ -253,6 +253,7 @@ xfsaild_push( int flush_log, count, stuck; xfs_mount_t *mp = ailp->xa_mount; struct xfs_ail_cursor *cur = &ailp->xa_cursors; + int push_xfsbufd = 0; spin_lock(&ailp->xa_lock); xfs_trans_ail_cursor_init(ailp, cur); @@ -308,6 +309,7 @@ xfsaild_push( XFS_STATS_INC(xs_push_ail_pushbuf); IOP_PUSHBUF(lip); last_pushed_lsn = lsn; + push_xfsbufd = 1; break; case XFS_ITEM_PINNED: @@ -322,12 +324,6 @@ xfsaild_push( stuck++; break; - case XFS_ITEM_FLUSHING: - XFS_STATS_INC(xs_push_ail_flushing); - last_pushed_lsn = lsn; - stuck++; - break; - default: ASSERT(0); break; @@ -374,6 +370,11 @@ xfsaild_push( xfs_log_force(mp, 0); } + if (push_xfsbufd) { + /* we've got delayed write buffers to flush */ + wake_up_process(mp->m_ddev_targp->bt_task); + } + if (!count) { /* We're past our target or empty, so idle */ last_pushed_lsn = 0; -- 1.6.5 From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:36 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_54,J_CHICKENPOX_72 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329Z6u204147 for ; Tue, 2 Feb 2010 20:09:36 -0600 X-ASG-Debug-ID: 1265163040-163b021e0002-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A97791C9B81E for ; Tue, 2 Feb 2010 18:10:43 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id 02JsoSRNQr7JT6HS for ; Tue, 02 Feb 2010 18:10:43 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207864-1927428 for ; Wed, 03 Feb 2010 12:40:42 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003nd-7k for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006ih-1X for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 0/10] Delayed write metadata writeback V4 Subject: [PATCH 0/10] Delayed write metadata writeback V4 Date: Wed, 3 Feb 2010 10:24:54 +1100 Message-Id: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163044 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean While I started with killing async inode writeback, the series has grown. It's not really limited to inode writeback - it touches dquot flushing, changes the way the AIL pushes on buffers, adds xfsbufd sortingi for delayed write buffers, adds a real non-blocking mode to inode reclaim and avoids physical inode writeback from the VFS while fixing bugs in handling delayed write inodes. Hence this is more about enabling efficient delayed write metadata than it is able killing async inode writeback. The idea behind this series is to make metadata buffers get written from xfsbufd via the delayed write queue rather than being issued asynchronously from all over the place. To do this, async buffer writeback is almost entirely removed from XFS, replaced instead by delayed writes and a method to expedite flushing of delayed write buffers when required. The result of funnelling all the buffer IO into a single place is that we can more tightly control and therefore optimise the submission of metadata IO. Aggregating the buffers before dispatch allows much better sort efficiency of the buffers as the sort window is not limited to the size of the elevator congestion hysteresis limit. Hence we can approach 100% merge effeciency on large numbers of buffers when dispatched for IO and greatly reduce the amount of seeking metadata writeback causes. The major change is to the inode flushing and reclaim code. Delayed write inodes hold the flush lock for much longer than for async writeback, and hence blocking on the flush lock can cause extremely long latencies without other mechanisms to expedite the release of the flush locks. To prevent needing to flush inodes immeidately, all operations are done non-blocking unless synchronous. THis required a significant rework of the inode reclaim code, but it greatly simplified other pieces of code (e.g. log item pushing). Version 4 - rework inode reclaim checks for better legibility - add warning to reclaim code when delwri flush errors occur - kill XFS_ITEM_FLUSHING now it is not used - clean up sync_mode flags being pushed into xfs_iflush() - kill the now unused xfs_bawrite() function - include Christoph's fsync cache flush fix - rework the inode locking and call to xfs_fsync() when doing synchronous inode writes to close races between the fsync and the background delwri flush afterwards. Version 3 - rework inode reclaim to: - separate it from xfs_iflush return values - provide a non-blocking mode for background operation - apply delwri buffer promotion tricks to dquot flushing - kill unneeded dquot flushing flags, similar to inode flushing flag removal - fix sync inode flush bug when trying to flush delwri inodes Version 2: - use generic list sort function - when unmounting, push the delwri buffers first, then do sync inode reclaim so that reclaim doesn't block for 15 seconds waiting for delwri inode buffers to be aged and written before the inodes can be reclaimed. Performance numbers for this version are the same as V2, which were as follows: Perf results (average of 3 runs) on a debug XFS build (means allocation patterns are randomly varied, so runtimes are also a bit variable): Untar 2.6.32 kernel tarball, sync, then remove: Untar+sync rm -rf xfs-dev: 25.2s 13.0s xfs-dev-delwri-1: 22.5s 9.1s xfs-dev-delwri-2: 21.9s 8.4s 4 processes each creating 100,000, five byte files in separate directories concurrently, then 4 processes removing a directory each concurrently. create rm -rf xfs-dev: 8m32s 4m10s xfs-dev-delwri-1: 4m55s 3m42s xfs-dev-delwri-2: 4m56s 3m33s The patch series (plus the couple of previous bug fixes that haven't been pulled into the tree on oss yet) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/dgc/xfs for-2.6.34 Christoph Hellwig (1): xfs: remove invalid barrier optimization from xfs_fsync Dave Chinner (11): xfs: don't hold onto reserved blocks on remount,ro xfs: turn off sign warnings xfs: Make inode reclaim states explicit xfs: Use delayed write for inodes rather than async V2 xfs: Don't issue buffer IO direct from AIL push V2 xfs: Sort delayed write buffers before dispatch xfs: Use delay write promotion for dquot flushing xfs: kill the unused XFS_QMOPT_* flush flags V2 xfs: move the inode locking outside xfs_fsync() xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 xfs: kill xfs_bawrite fs/xfs/Makefile | 2 +- fs/xfs/linux-2.6/xfs_buf.c | 135 ++++++++++++++++++++++++++-------------- fs/xfs/linux-2.6/xfs_buf.h | 3 +- fs/xfs/linux-2.6/xfs_file.c | 6 ++- fs/xfs/linux-2.6/xfs_lrw.c | 5 +- fs/xfs/linux-2.6/xfs_super.c | 82 ++++++++++++++++++------- fs/xfs/linux-2.6/xfs_sync.c | 138 +++++++++++++++++++++++++++++++++------- fs/xfs/linux-2.6/xfs_trace.h | 1 + fs/xfs/quota/xfs_dquot.c | 38 +++++------- fs/xfs/quota/xfs_dquot_item.c | 87 ++++---------------------- fs/xfs/quota/xfs_dquot_item.h | 4 - fs/xfs/quota/xfs_qm.c | 14 ++--- fs/xfs/xfs_buf_item.c | 64 ++++++++++--------- fs/xfs/xfs_inode.c | 86 ++------------------------ fs/xfs/xfs_inode.h | 11 +--- fs/xfs/xfs_inode_item.c | 108 +++++++------------------------- fs/xfs/xfs_inode_item.h | 6 -- fs/xfs/xfs_mount.c | 13 ++++- fs/xfs/xfs_mount.h | 1 + fs/xfs/xfs_quota.h | 8 +-- fs/xfs/xfs_trans.h | 3 +- fs/xfs/xfs_trans_ail.c | 13 ++-- fs/xfs/xfs_vnodeops.c | 46 +++++-------- fs/xfs/xfs_vnodeops.h | 2 +- 24 files changed, 415 insertions(+), 461 deletions(-) From SRS0+4mlS+63+fromorbit.com=david@internode.on.net Tue Feb 2 20:09:42 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=AWL,BAYES_00, DATE_IN_PAST_24_48,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329f7I204182 for ; Tue, 2 Feb 2010 20:09:42 -0600 X-ASG-Debug-ID: 1265163049-6fe903540000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E73881C9B823 for ; Tue, 2 Feb 2010 18:10:49 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id sbNWRmucSSRvzwA7 for ; Tue, 02 Feb 2010 18:10:49 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207897-1927428 for multiple; Wed, 03 Feb 2010 12:40:48 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1Nc5Ed-000707-7C; Tue, 02 Feb 2010 09:59:11 +1100 Date: Tue, 2 Feb 2010 09:59:11 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 2/7] xfs: Use delayed write for inodes rather than async Subject: Re: [PATCH 2/7] xfs: Use delayed write for inodes rather than async Message-ID: <20100201225911.GA26252@discord.disaster> References: <1264400564-19704-1-git-send-email-david@fromorbit.com> <1264400564-19704-3-git-send-email-david@fromorbit.com> <20100125115308.GC8595@infradead.org> <20100125223140.GN25842@discord.disaster> <20100126161045.GB24152@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100126161045.GB24152@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163050 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.53 X-Barracuda-Spam-Status: No, SCORE=-1.53 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DATE_IN_PAST_24_48, DATE_IN_PAST_24_48_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 DATE_IN_PAST_24_48 Date: is 24 to 48 hours before Received: date 0.48 DATE_IN_PAST_24_48_2 DATE_IN_PAST_24_48_2 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Jan 26, 2010 at 11:10:45AM -0500, Christoph Hellwig wrote: > > > Also in this context > > > the requeue label name doesn't fit too well, even if it's the same > > > action as the requeue. > > > > I'm open to suggestions for a better name ;) > > What do you think about this version? Yeah, that looks a lot better. I'll incorporate that into the patch. Thanks. Cheers, Dave. -- Dave Chinner david@fromorbit.com From SRS0+DEPi+63+fromorbit.com=dave@internode.on.net Tue Feb 2 20:09:36 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329Zta204133 for ; Tue, 2 Feb 2010 20:09:35 -0600 X-ASG-Debug-ID: 1265163035-1da700e90008-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A3DCB1371DE8 for ; Tue, 2 Feb 2010 18:10:42 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id YFzxGiejz0uiJQVN for ; Tue, 02 Feb 2010 18:10:42 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207854-1927428 for ; Wed, 03 Feb 2010 12:40:41 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NcS7X-0003nf-8Y for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:23 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NcS7W-0006in-64 for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:25:22 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 02/10] xfs: Use delayed write for inodes rather than async V2 Subject: [PATCH 02/10] xfs: Use delayed write for inodes rather than async V2 Date: Wed, 3 Feb 2010 10:24:56 +1100 Message-Id: <1265153104-29680-3-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265153104-29680-1-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163043 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We currently do background inode flush asynchronously, resulting in inodes being written in whatever order the background writeback issues them. Not only that, there are also blocking and non-blocking asynchronous inode flushes, depending on where the flush comes from. This patch completely removes asynchronous inode writeback. It removes all the strange writeback modes and replaces them with either a synchronous flush or a non-blocking delayed write flush. That is, inode flushes will only issue IO directly if they are synchronous, and background flushing may do nothing if the operation would block (e.g. on a pinned inode or buffer lock). Delayed write flushes will now result in the inode buffer sitting in the delwri queue of the buffer cache to be flushed by either an AIL push or by the xfsbufd timing out the buffer. This will allow accumulation of dirty inode buffers in memory and allow optimisation of inode cluster writeback at the xfsbufd level where we have much greater queue depths than the block layer elevators. We will also get adjacent inode cluster buffer IO merging for free when a later patch in the series allows sorting of the delayed write buffers before dispatch. This effectively means that any inode that is written back by background writeback will be seen as flush locked during AIL pushing, and will result in the buffers being pushed from there. This writeback path is currently non-optimal, but the next patch in the series will fix that problem. A side effect of this delayed write mechanism is that background inode reclaim will no longer directly flush inodes, nor can it wait on the flush lock. The result is that inode reclaim must leave the inode in the reclaimable state until it is clean. Hence attempts to reclaim a dirty inode in the background will simply skip the inode until it is clean and this allows other mechanisms (i.e. xfsbufd) to do more optimal writeback of the dirty buffers. As a result, the inode reclaim code has been rewritten so that it no longer relies on the ambiguous return values of xfs_iflush() to determine whether it is safe to reclaim an inode. Portions of this patch are derived from patches by Christoph Hellwig. Version 2: - cleanup reclaim code as suggested by Christoph - log background reclaim inode flush errors - just pass sync flags to xfs_iflush Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_super.c | 4 +- fs/xfs/linux-2.6/xfs_sync.c | 104 ++++++++++++++++++++++++++++++----------- fs/xfs/xfs_inode.c | 75 ++---------------------------- fs/xfs/xfs_inode.h | 10 ---- fs/xfs/xfs_inode_item.c | 10 +++- fs/xfs/xfs_mount.c | 13 +++++- 6 files changed, 102 insertions(+), 114 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index e9c2145..226fe20 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -1064,7 +1064,7 @@ xfs_fs_write_inode( xfs_ilock(ip, XFS_ILOCK_SHARED); xfs_iflock(ip); - error = xfs_iflush(ip, XFS_IFLUSH_SYNC); + error = xfs_iflush(ip, SYNC_WAIT); } else { error = EAGAIN; if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) @@ -1072,7 +1072,7 @@ xfs_fs_write_inode( if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) goto out_unlock; - error = xfs_iflush(ip, XFS_IFLUSH_ASYNC_NOBLOCK); + error = xfs_iflush(ip, 0); } out_unlock: diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c index 98b8937..a9f6d20 100644 --- a/fs/xfs/linux-2.6/xfs_sync.c +++ b/fs/xfs/linux-2.6/xfs_sync.c @@ -270,8 +270,7 @@ xfs_sync_inode_attr( goto out_unlock; } - error = xfs_iflush(ip, (flags & SYNC_WAIT) ? - XFS_IFLUSH_SYNC : XFS_IFLUSH_DELWRI); + error = xfs_iflush(ip, flags); out_unlock: xfs_iunlock(ip, XFS_ILOCK_SHARED); @@ -460,16 +459,18 @@ xfs_quiesce_fs( { int count = 0, pincount; + xfs_reclaim_inodes(mp, 0); xfs_flush_buftarg(mp->m_ddev_targp, 0); - xfs_reclaim_inodes(mp, XFS_IFLUSH_DELWRI_ELSE_ASYNC); /* * This loop must run at least twice. The first instance of the loop * will flush most meta data but that will generate more meta data * (typically directory updates). Which then must be flushed and - * logged before we can write the unmount record. + * logged before we can write the unmount record. We also so sync + * reclaim of inodes to catch any that the above delwri flush skipped. */ do { + xfs_reclaim_inodes(mp, SYNC_WAIT); xfs_sync_attr(mp, SYNC_WAIT); pincount = xfs_flush_buftarg(mp->m_ddev_targp, 1); if (!pincount) { @@ -585,7 +586,7 @@ xfs_sync_worker( if (!(mp->m_flags & XFS_MOUNT_RDONLY)) { xfs_log_force(mp, 0); - xfs_reclaim_inodes(mp, XFS_IFLUSH_DELWRI_ELSE_ASYNC); + xfs_reclaim_inodes(mp, 0); /* dgc: errors ignored here */ error = xfs_qm_sync(mp, SYNC_TRYLOCK); error = xfs_sync_fsdata(mp, SYNC_TRYLOCK); @@ -719,21 +720,42 @@ __xfs_inode_clear_reclaim_tag( * shutdown EIO unpin and reclaim * clean, unpinned 0 reclaim * stale, unpinned 0 reclaim - * clean, pinned(*) 0 unpin and reclaim - * stale, pinned 0 unpin and reclaim - * dirty, async 0 block on flush lock, reclaim - * dirty, sync flush 0 block on flush lock, reclaim + * clean, pinned(*) 0 requeue + * stale, pinned EAGAIN requeue + * dirty, delwri ok 0 requeue + * dirty, delwri blocked EAGAIN requeue + * dirty, sync flush 0 reclaim * * (*) dgc: I don't think the clean, pinned state is possible but it gets * handled anyway given the order of checks implemented. * + * As can be seen from the table, the return value of xfs_iflush() is not + * sufficient to correctly decide the reclaim action here. The checks in + * xfs_iflush() might look like duplicates, but they are not. + * + * Also, because we get the flush lock first, we know that any inode that has + * been flushed delwri has had the flush completed by the time we check that + * the inode is clean. The clean inode check needs to be done before flushing + * the inode delwri otherwise we would loop forever requeuing clean inodes as + * we cannot tell apart a successful delwri flush and a clean inode from the + * return value of xfs_iflush(). + * + * Note that because the inode is flushed delayed write by background + * writeback, the flush lock may already be held here and waiting on it can + * result in very long latencies. Hence for sync reclaims, where we wait on the + * flush lock, the caller should push out delayed write inodes first before + * trying to reclaim them to minimise the amount of time spent waiting. For + * background relaim, we just requeue the inode for the next pass. + * * Hence the order of actions after gaining the locks should be: * bad => reclaim * shutdown => unpin and reclaim - * pinned => unpin + * pinned, delwri => requeue + * pinned, sync => unpin * stale => reclaim * clean => reclaim - * dirty => flush, wait and reclaim + * dirty, delwri => flush and requeue + * dirty, sync => flush, wait and reclaim */ STATIC int xfs_reclaim_inode( @@ -741,7 +763,7 @@ xfs_reclaim_inode( struct xfs_perag *pag, int sync_mode) { - int error; + int error = 0; /* * The radix tree lock here protects a thread in xfs_iget from racing @@ -761,7 +783,11 @@ xfs_reclaim_inode( write_unlock(&pag->pag_ici_lock); xfs_ilock(ip, XFS_ILOCK_EXCL); - xfs_iflock(ip); + if (!xfs_iflock_nowait(ip)) { + if (!(sync_mode & SYNC_WAIT)) + goto out; + xfs_iflock(ip); + } if (is_bad_inode(VFS_I(ip))) goto reclaim; @@ -769,8 +795,13 @@ xfs_reclaim_inode( xfs_iunpin_wait(ip); goto reclaim; } - if (xfs_ipincount(ip)) + if (xfs_ipincount(ip)) { + if (!(sync_mode & SYNC_WAIT)) { + xfs_ifunlock(ip); + goto out; + } xfs_iunpin_wait(ip); + } if (xfs_iflags_test(ip, XFS_ISTALE)) goto reclaim; if (xfs_inode_clean(ip)) @@ -778,26 +809,43 @@ xfs_reclaim_inode( /* Now we have an inode that needs flushing */ error = xfs_iflush(ip, sync_mode); - if (!error) { - switch(sync_mode) { - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - case XFS_IFLUSH_DELWRI: - case XFS_IFLUSH_ASYNC: - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - case XFS_IFLUSH_SYNC: - /* IO issued, synchronise with IO completion */ - xfs_iflock(ip); - default: - ASSERT(0); - break; - } + if (sync_mode & SYNC_WAIT) { + xfs_iflock(ip); + goto reclaim; } + /* + * When we have to flush an inode but don't have SYNC_WAIT set, we + * flush the inode out using a delwri buffer and wait for the next + * call into reclaim to find it in a clean state instead of waiting for + * it now. We also don't return errors here - if the error is transient + * then the next reclaim pass will flush the inode, and if the error + * is permanent then the next sync reclaim will relcaim the inode and + * pass on the error. + */ + if (error && !XFS_FORCED_SHUTDOWN(ip->i_mount)) { + xfs_fs_cmn_err(CE_WARN, ip->i_mount, + "inode 0x%llx background reclaim flush failed with %d", + (long long)ip->i_ino, error); + } +out: + xfs_iflags_clear(ip, XFS_IRECLAIM); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + /* + * We could return EAGAIN here to make reclaim rescan the inode tree in + * a short while. However, this just burns CPU time scanning the tree + * waiting for IO to complete and xfssyncd never goes back to the idle + * state. Instead, return 0 to let the next scheduled background reclaim + * attempt to reclaim the inode again. + */ + return 0; + reclaim: xfs_ifunlock(ip); xfs_iunlock(ip, XFS_ILOCK_EXCL); xfs_ireclaim(ip); - return 0; + return error; + } int diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 8d0666d..fa31360 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2835,8 +2835,6 @@ xfs_iflush( xfs_dinode_t *dip; xfs_mount_t *mp; int error; - int noblock = (flags == XFS_IFLUSH_ASYNC_NOBLOCK); - enum { INT_DELWRI = (1 << 0), INT_ASYNC = (1 << 1) }; XFS_STATS_INC(xs_iflush_count); @@ -2859,7 +2857,7 @@ xfs_iflush( * in the same cluster are dirty, they will probably write the inode * out for us if they occur after the log force completes. */ - if (noblock && xfs_ipincount(ip)) { + if (!(flags & SYNC_WAIT) && xfs_ipincount(ip)) { xfs_iunpin_nowait(ip); xfs_ifunlock(ip); return EAGAIN; @@ -2893,60 +2891,10 @@ xfs_iflush( } /* - * Decide how buffer will be flushed out. This is done before - * the call to xfs_iflush_int because this field is zeroed by it. - */ - if (iip != NULL && iip->ili_format.ilf_fields != 0) { - /* - * Flush out the inode buffer according to the directions - * of the caller. In the cases where the caller has given - * us a choice choose the non-delwri case. This is because - * the inode is in the AIL and we need to get it out soon. - */ - switch (flags) { - case XFS_IFLUSH_SYNC: - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - flags = 0; - break; - case XFS_IFLUSH_ASYNC_NOBLOCK: - case XFS_IFLUSH_ASYNC: - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - flags = INT_ASYNC; - break; - case XFS_IFLUSH_DELWRI: - flags = INT_DELWRI; - break; - default: - ASSERT(0); - flags = 0; - break; - } - } else { - switch (flags) { - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - case XFS_IFLUSH_DELWRI: - flags = INT_DELWRI; - break; - case XFS_IFLUSH_ASYNC_NOBLOCK: - case XFS_IFLUSH_ASYNC: - flags = INT_ASYNC; - break; - case XFS_IFLUSH_SYNC: - flags = 0; - break; - default: - ASSERT(0); - flags = 0; - break; - } - } - - /* * Get the buffer containing the on-disk inode. */ error = xfs_itobp(mp, NULL, ip, &dip, &bp, - noblock ? XBF_TRYLOCK : XBF_LOCK); + (flags & SYNC_WAIT) ? XBF_LOCK : XBF_TRYLOCK); if (error || !bp) { xfs_ifunlock(ip); return error; @@ -2974,13 +2922,10 @@ xfs_iflush( if (error) goto cluster_corrupt_out; - if (flags & INT_DELWRI) { - xfs_bdwrite(mp, bp); - } else if (flags & INT_ASYNC) { - error = xfs_bawrite(mp, bp); - } else { + if (flags & SYNC_WAIT) error = xfs_bwrite(mp, bp); - } + else + xfs_bdwrite(mp, bp); return error; corrupt_out: @@ -3015,16 +2960,6 @@ xfs_iflush_int( iip = ip->i_itemp; mp = ip->i_mount; - - /* - * If the inode isn't dirty, then just release the inode - * flush lock and do nothing. - */ - if (xfs_inode_clean(ip)) { - xfs_ifunlock(ip); - return 0; - } - /* set *dip = inode's place in the buffer */ dip = (xfs_dinode_t *)xfs_buf_offset(bp, ip->i_imap.im_boffset); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 8b618ea..6c912b0 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -420,16 +420,6 @@ static inline void xfs_ifunlock(xfs_inode_t *ip) #define XFS_ILOCK_DEP(flags) (((flags) & XFS_ILOCK_DEP_MASK) >> XFS_ILOCK_SHIFT) /* - * Flags for xfs_iflush() - */ -#define XFS_IFLUSH_DELWRI_ELSE_SYNC 1 -#define XFS_IFLUSH_DELWRI_ELSE_ASYNC 2 -#define XFS_IFLUSH_SYNC 3 -#define XFS_IFLUSH_ASYNC 4 -#define XFS_IFLUSH_DELWRI 5 -#define XFS_IFLUSH_ASYNC_NOBLOCK 6 - -/* * Flags for xfs_itruncate_start(). */ #define XFS_ITRUNC_DEFINITE 0x1 diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index 48ec1c0..207553e 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -866,10 +866,14 @@ xfs_inode_item_push( iip->ili_format.ilf_fields != 0); /* - * Write out the inode. The completion routine ('iflush_done') will - * pull it from the AIL, mark it clean, unlock the flush lock. + * Push the inode to it's backing buffer. This will not remove the + * inode from the AIL - a further push will be required to trigger a + * buffer push. However, this allows all the dirty inodes to be pushed + * to the buffer before it is pushed to disk. THe buffer IO completion + * will pull th einode from the AIL, mark it clean and unlock the flush + * lock. */ - (void) xfs_iflush(ip, XFS_IFLUSH_ASYNC); + (void) xfs_iflush(ip, 0); xfs_iunlock(ip, XFS_ILOCK_SHARED); return; diff --git a/fs/xfs/xfs_mount.c b/fs/xfs/xfs_mount.c index 7f81ed7..9a036f4 100644 --- a/fs/xfs/xfs_mount.c +++ b/fs/xfs/xfs_mount.c @@ -1456,7 +1456,18 @@ xfs_unmountfs( * need to force the log first. */ xfs_log_force(mp, XFS_LOG_SYNC); - xfs_reclaim_inodes(mp, XFS_IFLUSH_ASYNC); + + /* + * Do a delwri reclaim pass first so that as many dirty inodes are + * queued up for IO as possible. Then flush the buffers before making + * a synchronous path to catch all the remaining inodes are reclaimed. + * This makes the reclaim process as quick as possible by avoiding + * synchronous writeout and blocking on inodes already in the delwri + * state as much as possible. + */ + xfs_reclaim_inodes(mp, 0); + XFS_bflush(mp->m_ddev_targp); + xfs_reclaim_inodes(mp, SYNC_WAIT); xfs_qm_unmount(mp); -- 1.6.5 From SRS0+4mlS+63+fromorbit.com=david@internode.on.net Tue Feb 2 20:09:45 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=AWL,BAYES_00, DATE_IN_PAST_24_48,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1329jfM204210 for ; Tue, 2 Feb 2010 20:09:45 -0600 X-ASG-Debug-ID: 1265163049-6fc9035f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E7BF1C9B826 for ; Tue, 2 Feb 2010 18:10:50 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id UCSoAlCTlxLpa8qb for ; Tue, 02 Feb 2010 18:10:50 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12207902-1927428 for multiple; Wed, 03 Feb 2010 12:40:49 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1Nc51C-0006ez-6B; Tue, 02 Feb 2010 09:45:18 +1100 Date: Tue, 2 Feb 2010 09:45:18 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] xfs: remove invalid barrier optimization from xfs_fsync Subject: Re: [PATCH] xfs: remove invalid barrier optimization from xfs_fsync Message-ID: <20100201224518.GA28368@discord.disaster> References: <20100126213851.GA14927@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100126213851.GA14927@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265163051 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0208 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.53 X-Barracuda-Spam-Status: No, SCORE=-1.53 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DATE_IN_PAST_24_48, DATE_IN_PAST_24_48_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21506 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 DATE_IN_PAST_24_48 Date: is 24 to 48 hours before Received: date 0.48 DATE_IN_PAST_24_48_2 DATE_IN_PAST_24_48_2 X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Jan 26, 2010 at 04:38:51PM -0500, Christoph Hellwig wrote: > We always need to flush the disk write cache and can't skip it just because > the no inode attributes have changed. > > > Signed-off-by: Christoph Hellwig Looks good. Reviewed-by: Dave Chinner -- Dave Chinner david@fromorbit.com From morfic@gmail.com Tue Feb 2 20:18:24 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o132IN7B204444 for ; Tue, 2 Feb 2010 20:18:23 -0600 X-ASG-Debug-ID: 1265163570-058303280000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5A64F1371EE1 for ; Tue, 2 Feb 2010 18:19:30 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id CneUJEdzhJrdsKXf for ; Tue, 02 Feb 2010 18:19:30 -0800 (PST) Received: by ewy27 with SMTP id 27so827138ewy.18 for ; Tue, 02 Feb 2010 18:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=E4xbZtYNtWH/KOhT/znPcpsO5wxxOvGBF2zG34fjORA=; b=BJWKnfhe1YcInLhFDZ2aOXuExuNCVU2I5BlF0JrTdh/BTb/Vc4lKs6iNuW0ucB3t0e apR+WPVeIAHQLHqcLcKmew7CQFuauXNPlQWW4BONyprY0VqSYLhN0zm1PEQe/juZTvIA De7NVAwQjlUDfIkFT1m7vAjCyd4uJ1eOZuQPk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=d4nePjnaiP+4MXI3ikCbfib0rGbx4+oFGSpymA6P3iG3WvClQhdMQPY+rFniDgUlH+ 1E40MTYp8gaya/jylcl8VKBuSRMPcJ5OqoUJPA3dG46ioX73GLXMTjFOU70Q7M8FZ0gm N/60cJT7CQ8KC6VDUYhzXFNzg2u7uMKzlWFTI= MIME-Version: 1.0 Received: by 10.216.162.142 with SMTP id y14mr4090054wek.192.1265163569469; Tue, 02 Feb 2010 18:19:29 -0800 (PST) In-Reply-To: <20100202112300.GA23809@infradead.org> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> Date: Tue, 2 Feb 2010 20:19:29 -0600 Message-ID: <13bb8ce11002021819l7e17dac4n5f58a9887a16fae2@mail.gmail.com> X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM From: Daniel Goller To: Christoph Hellwig Cc: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265163571 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.02 X-Barracuda-Spam-Status: No, SCORE=-1.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M, BSF_RULE_7582B, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21507 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.50 BSF_RULE7568M Custom Rule 7568M 0.50 BSF_RULE_7582B Custom Rule 7582B X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Thanks for the many replys, it also made me realize how basic and general the question was, without giving much details about the hardware. I see my "can't mount a xfs partition after successful unmounting without needing to clear the log" in 2.6.32, 2.6.33-rc2 and now 2.6.33-rc6. The arm device: Processor : Feroceon 88FR131 rev 1 (v5l) BogoMIPS : 1192.75 Features : swp half thumb fastmult edsp CPU implementer : 0x56 CPU architecture: 5TE CPU variant : 0x2 CPU part : 0x131 CPU revision : 1 Hardware : Marvell SheevaPlug Reference Board Revision : 0000 Serial : 0000000000000000 Typically i just can't mount without a xfs_repair -L which seems to be all i need to get it back and lose no data in the process. In 2.6.33-rc6 i had it for the first time do this: (which basically stopped the boot process cold as it would not go past this point) keeping this partition from being mounted at boot then allowed me to record the following when manually mounting it) * Mounting local filesystems... Internal error: Oops: 5 [#1] last sysfs file: /sys/kernel/uevent_seqnum Modules linked in: CPU: 0 Not tainted (2.6.33-rc6 #1) PC is at xlog_recover_process_data+0x1c4/0x304 LR is at xlog_recover_unmount_trans+0x10/0x1c pc : [] lr : [] psr: 20000013 sp : df091c78 ip : 0000000a fp : 00000001 r10: df0794d0 r9 : df091d14 r8 : e0ffea00 r7 : df091d04 r6 : e0ffe034 r5 : 00000010 r4 : 02e00a34 r3 : 00000001 r2 : c0563190 r1 : c05631b8 r0 : 00000000 Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user Control: 0005317f Table: 1f314000 DAC: 00000015 Process mount (pid: 857, stack limit =3D 0xdf090270) Stack: (0xdf091c78 to 0xdf092000) 1c60: 00000007 df2910= 00 1c80: 00000005 df3329c0 00000000 00000001 00000005 df3329c0 df291000 000000= 05 1ca0: 00000001 00000000 df332cc0 0000000f 00000000 c0206eb0 00000001 df332c= c0 1cc0: df091d44 ffffffff c0563174 c003d13c df091d44 c058dddc c0563174 000000= 02 1ce0: 00000002 00000000 00000000 00000000 df332cc0 c003d66c 00000028 df332c= 00 1d00: c04f0121 00000000 00000000 00000000 00000000 df0ade00 00000000 000000= 00 1d20: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000= 00 1d40: 00000000 e0ffe000 00000000 df3329c0 00000000 00000028 00000000 000000= 06 1d60: 00000000 a3d7b000 00000000 c02074d8 00000006 00000000 00000001 c04277= 58 1d80: 60000013 df3329c0 00000000 00000006 00000000 00408100 00000000 c02075= 34 1da0: 00000006 00000000 00000000 df3329c0 00000000 00000006 00000000 004081= 00 1dc0: 00000000 c0208be4 00000006 00000000 00000006 00000000 00000028 000000= 00 1de0: df0fd800 000001c0 00000000 c0202b58 00020000 df0fd800 c0220a64 df0fd8= 00 1e00: c0220a64 073904d0 00000000 073904cc 00000000 c020ac34 00020000 000002= d0 1e20: 00000001 00000000 51eb851f c02154fc 00000000 00000058 df296000 c01f1b= 1c 1e40: 0000000a 00000058 df189120 c01f1b1c 0000000a df0fda9c df296000 c01f1b= 1c 1e60: 00000000 df0fd800 df218a00 00000000 df28a000 00000000 00000000 df28a0= 00 1e80: 00000000 c0220a64 df218a00 00000000 df6b8340 00000003 df6b839c 000000= 00 1ea0: df218a00 c00991fc 33616473 df096f00 df096da0 000000d0 df096da0 df096f= 40 1ec0: 0000000a c007cdd8 df296d00 df296d00 c056dfc4 df096da0 df28a000 000000= 00 1ee0: df28a000 c021ed60 c02208e8 df296d00 00000000 c00981fc df296d00 c00ab3= 04 1f00: c056dfc4 df096d60 df28a000 df096da0 00000000 c00982d0 00000000 000000= 30 1f20: df091f48 df096da0 df096d60 c00addc4 00000a18 00200800 df28a000 df0900= 00 1f40: 00000000 0001e560 dfbebe00 df705d80 c007186c df241000 0001e560 002008= 00 1f60: 00000000 c002b044 df090000 00000000 0001e560 c00ae3f4 df28a000 c00476= c8 1f80: 00000000 df28a000 df096da0 df096d60 00000000 0001ea18 00200800 0001e5= 70 1fa0: 00000015 c002aec0 0001ea18 00200800 0001e560 0001e570 0001e988 002008= 00 1fc0: 0001ea18 00200800 0001e570 00000015 0001c444 00000000 00000000 0001e5= 60 1fe0: 00200800 befff5cc 0000b0f4 40115384 60000010 0001e560 00000000 000000= 00 [] (xlog_recover_process_data+0x1c4/0x304) from [] (xlog_do_recovery_pass+0x1b0/0x794) [] (xlog_do_recovery_pass+0x1b0/0x794) from [] (xlog_do_log_recovery+0x44/0= x88) [] (xlog_do_log_recovery+0x44/0x88) from [] (xlog_do_recover+0x18/0x124) [] (xlog_do_recover+0x18/0x124) from [] (xlog_recover+0x7c/0xc4) [] (xlog_recover+0x7c/0xc4) from [] (xfs_log_mount+0x154/0x18c) [] (xfs_log_mount+0x154/0x18c) from [] (xfs_mountfs+0x34c/0x670) [] (xfs_mountfs+0x34c/0x670) from [] (xfs_fs_fill_super+0x17c/0x2c0) [] (xfs_fs_fill_super+0x17c/0x2c0) from [] (get_sb_bdev+0x138/0x170) [] (get_sb_bdev+0x138/0x170) from [] (xfs_fs_get_sb+0x1c/0x28) [] (xfs_fs_get_sb+0x1c/0x28) from [] (vfs_kern_mount+0x50/0xe0) [] (vfs_kern_mount+0x50/0xe0) from [] (do_kern_mount+0x34/0xdc) [] (do_kern_mount+0x34/0xdc) from [] (do_mount+0x154/0x700) [] (do_mount+0x154/0x700) from [] (sys_mount+0x84/0xc4) [] (sys_mount+0x84/0xc4) from [] (ret_fast_syscall+0x0/0x28) Code: 93a03000 82033001 e3530000 0a000048 (e5d43008) ---[ end trace b80cd4d01963a09f ]--- /etc/init.d/localmount: line 15: 857 Segmentation fault mount -at "$types" it seems to operate fine since running a xfs_repair -L on it from the x86_64 laptop (It is a disk in a usb enclosure, which makes the moving of the drive to clear the log rather easy) > The git tree with the current versions of the patches to fix this is > here: > > =A0 =A0 =A0 =A0http://git.kernel.org/?p=3Dlinux/kernel/git/jejb/xfs-vipt/= .git;a=3Dsummary > > I will probably not get a chance to try pulling this from git before the weekend, if the Sheevaplug however the type of arm machine you were referring to (i honestly never looked at its cache), i would be interested in trying the fix and (hopefully) stick to xfs for this partition. Thanks, Daniel Goller Let me know if there is further information you would like to look at for this issue. From sandeen@sandeen.net Tue Feb 2 20:48:15 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o132mFs6205613 for ; Tue, 2 Feb 2010 20:48:15 -0600 X-ASG-Debug-ID: 1265165363-6a16002d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 72BB31C9B883 for ; Tue, 2 Feb 2010 18:49:24 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id yH9rD321Jl0XDePx for ; Tue, 02 Feb 2010 18:49:24 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id A3780109E852 for ; Tue, 2 Feb 2010 20:49:23 -0600 (CST) Message-ID: <4B68E433.9050309@sandeen.net> Date: Tue, 02 Feb 2010 20:49:23 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss X-ASG-Orig-Subj: Re: [PATCH V2] xfstests: routine to create scratch of certain size Subject: Re: [PATCH V2] xfstests: routine to create scratch of certain size References: <4B621D89.8070800@sandeen.net> <4B68D75A.9020105@sandeen.net> In-Reply-To: <4B68D75A.9020105@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265165364 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21508 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Eric Sandeen wrote: > This is needed for later enospc tests to be generic > > Signed-off-by: Eric Sandeen > --- > > V2: Address Christoph's comment about use of MKFS_OPTIONS; > just do it like the other geom helper and explicitly -set- > MKFS_OPTIONS for scratch, to set the size, then just call > _scratch_mkfs > > (this would override any global MKFS_OPTIONS setting, but > I think that's ok for now, and not sure how these options might > interact w/ previously set MKFS_OPTIONS anyway) Scratch that, I'm not being diligent enough and didn't test this :/ (yay for "simple" changes) mke2fs takes the size arg at the end, after the device, so we cannot just set up MKFS_OPTIONS Sorry for the noise. -Eric (who will try to be more dilient now...) From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 04:54:40 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13AsbXx223826 for ; Wed, 3 Feb 2010 04:54:40 -0600 X-ASG-Debug-ID: 1265194546-079a02e30000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 578C213732E3 for ; Wed, 3 Feb 2010 02:55:46 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id etcdaNkFIDviGg17 for ; Wed, 03 Feb 2010 02:55:46 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Ncctd-00019X-Lj for xfs@oss.sgi.com; Wed, 03 Feb 2010 10:55:45 +0000 Date: Wed, 3 Feb 2010 05:55:45 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100203105545.GA1047@infradead.org> References: <20100201220813.GA3519@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100201220813.GA3519@infradead.org> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265194547 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Actually I'll take this one back - the log buffers aren't delwri so xfs_flush_buftarg probably only helped by timing or the xfs_buf_runall_queues on xfslogd_workqueue. Need to think about this a bit more. From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 04:55:41 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13AteTc223873 for ; Wed, 3 Feb 2010 04:55:41 -0600 X-ASG-Debug-ID: 1265194609-1ac101d50000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2CA5813732FD for ; Wed, 3 Feb 2010 02:56:49 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 9pF9Q0xBJySYqe0A for ; Wed, 03 Feb 2010 02:56:49 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nccuf-0001Jn-Fh; Wed, 03 Feb 2010 10:56:49 +0000 Date: Wed, 3 Feb 2010 05:56:49 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs-oss X-ASG-Orig-Subj: Re: [PATCH V3] xfstests: filter selinux output in _acl_ls etc Subject: Re: [PATCH V3] xfstests: filter selinux output in _acl_ls etc Message-ID: <20100203105649.GB1047@infradead.org> References: <4B61C1BC.4050800@sandeen.net> <4B626697.3080006@sandeen.net> <4B68B007.1020705@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B68B007.1020705@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265194610 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 02, 2010 at 05:06:47PM -0600, Eric Sandeen wrote: > When selinux is on, ls -l gives us a "." to indicate selinux > attrs, which breaks some tests: > > === Test minimal ACE === > Setup file > --rwxrw-r-- id1 id2 file1 > +-rwxrw-r--. id1 id2 file1 > > so make an _ls_l helper to filter that out. > > Signed-off-by: Eric Sandeen Looks good, Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 04:57:20 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13AvJh0223948 for ; Wed, 3 Feb 2010 04:57:20 -0600 X-ASG-Debug-ID: 1265194708-079e02fa0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 46872137330D for ; Wed, 3 Feb 2010 02:58:29 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id B5UnZYyynZvTWQ82 for ; Wed, 03 Feb 2010 02:58:29 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NccwG-0001jS-NL; Wed, 03 Feb 2010 10:58:28 +0000 Date: Wed, 3 Feb 2010 05:58:28 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: Christoph Hellwig , xfs-oss X-ASG-Orig-Subj: Re: [PATCH V3] xfstests: 223 - test file alignment on stripe geometry Subject: Re: [PATCH V3] xfstests: 223 - test file alignment on stripe geometry Message-ID: <20100203105828.GC1047@infradead.org> References: <4B621529.20708@sandeen.net> <20100130105236.GA18286@infradead.org> <4B68B532.9000103@sandeen.net> <4B68D5DF.20808@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B68D5DF.20808@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265194709 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Looks good to me, and although I haven't tested the respin yet myself it looks close enough to the original one. Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 05:16:12 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13BGCIE224673 for ; Wed, 3 Feb 2010 05:16:12 -0600 X-ASG-Debug-ID: 1265195841-635f00ac0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E20891A8437 for ; Wed, 3 Feb 2010 03:17:21 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id UZOQKzXXggDob4Q4 for ; Wed, 03 Feb 2010 03:17:21 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcdEW-0002du-Az; Wed, 03 Feb 2010 11:17:20 +0000 Date: Wed, 3 Feb 2010 06:17:20 -0500 From: Christoph Hellwig To: Dave Chinner Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 02/10] xfs: Use delayed write for inodes rather than async V2 Subject: Re: [PATCH 02/10] xfs: Use delayed write for inodes rather than async V2 Message-ID: <20100203111720.GA22957@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-3-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265153104-29680-3-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265195841 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Still looks good with the recent changes, Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 05:16:46 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13BGkVA224701 for ; Wed, 3 Feb 2010 05:16:46 -0600 X-ASG-Debug-ID: 1265195875-7e39015b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DD8541C9CA36 for ; Wed, 3 Feb 2010 03:17:55 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id Y2vrSR7SDlID0GuE for ; Wed, 03 Feb 2010 03:17:55 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcdF5-0003Xf-40; Wed, 03 Feb 2010 11:17:55 +0000 Date: Wed, 3 Feb 2010 06:17:55 -0500 From: Christoph Hellwig To: Dave Chinner Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 06/10] xfs: kill the unused XFS_QMOPT_* flush flags V2 Subject: Re: [PATCH 06/10] xfs: kill the unused XFS_QMOPT_* flush flags V2 Message-ID: <20100203111755.GB22957@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-7-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265153104-29680-7-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265195875 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 10:25:00AM +1100, Dave Chinner wrote: > dquots are never flushed asynchronously. Remove the flag and the > async write support from the flush function. Make the default flush > a delwri flush to make the inode flush code, which leaves the > XFS_QMOPT_SYNC the only flag remaining. Convert that to use > SYNC_WAIT instead, just like the inode flush code. > > V2: > - just pass flush flags straight through Still looks good with that minor change, Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 05:18:07 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13BI69M224769 for ; Wed, 3 Feb 2010 05:18:06 -0600 X-ASG-Debug-ID: 1265195956-637b00b80000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5E9A91A8440 for ; Wed, 3 Feb 2010 03:19:16 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id B8kAajtIShrFOHFg for ; Wed, 03 Feb 2010 03:19:16 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcdGO-0006YW-2p; Wed, 03 Feb 2010 11:19:16 +0000 Date: Wed, 3 Feb 2010 06:19:16 -0500 From: Christoph Hellwig To: Dave Chinner Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 10/10] xfs: kill xfs_bawrite Subject: Re: [PATCH 10/10] xfs: kill xfs_bawrite Message-ID: <20100203111916.GC22957@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-11-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265153104-29680-11-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265195956 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 10:25:04AM +1100, Dave Chinner wrote: > There are no more users of this function left in the XFS code > now that we've switched everything to delayed write flushing. > Remove it. Actually we still write the superblock async using xfs_bwrite, but getting rid of that is a separate project, so Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 05:26:45 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13BQi7M225072 for ; Wed, 3 Feb 2010 05:26:45 -0600 X-ASG-Debug-ID: 1265196474-0e2601240000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 51E371C9C797; Wed, 3 Feb 2010 03:27:54 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id Y0G7kco2hYJdMUad; Wed, 03 Feb 2010 03:27:54 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcdOj-0007NC-S5; Wed, 03 Feb 2010 11:27:53 +0000 Date: Wed, 3 Feb 2010 06:27:53 -0500 From: Christoph Hellwig To: Dave Chinner Cc: bpm@sgi.com, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100203112753.GA19996@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265153104-29680-10-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265196474 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Still not entirely happy with this one. The first one is that I think the barriers in fsync are still too heavy for the normal sync use case. I'd be more happy with exporting the body of xfs_fsync without the cache flushes (and a ebtter name than xfs_fsync) and use that for write_inode. That leaves open the NFSD case thought. I'd prefer to have that fixed if possibly. Ben, any chance you could send your patch to use fsync to the nfs list ASAP? I think we'd be even better off to just force -o wsync and disable ->write_inode entirely for NFS, any chance you could test such a patch on your setup? Besides that the patch is missing the comment from the previous iteration why we're still doing the delwri iflush for the sync == 1 case. I think keeping that one is important to explain the really weird reason for it. From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 05:28:08 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13BS8cS225113 for ; Wed, 3 Feb 2010 05:28:08 -0600 X-ASG-Debug-ID: 1265196557-0cf001490000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D77501C9C7BE for ; Wed, 3 Feb 2010 03:29:17 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id gH8pBKwlpsyEgUpg for ; Wed, 03 Feb 2010 03:29:17 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcdQ5-0000zx-JU; Wed, 03 Feb 2010 11:29:17 +0000 Date: Wed, 3 Feb 2010 06:29:17 -0500 From: Christoph Hellwig To: Dave Chinner Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Subject: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Message-ID: <20100203112917.GB19996@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-9-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265153104-29680-9-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265196557 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 10:25:02AM +1100, Dave Chinner wrote: > We have a need for a delayed write inode flush operation > to be made atomically with an fsync to avoid physically > writing inodes but still keeping inode buffer information > up to date for bulkstat. > > Move the inode locking outside xfs_fsync() to allow this to > be done. What's the point of the lock_flags argument? It should always be IOLOCK_SHARED, so instead of passing it in as an argument I'd rather add an assert to enforce it. For more comments on if this is the right functionality to use in write_inode see my comment to the next patch. From sandeen@sandeen.net Wed Feb 3 11:49:08 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Hn79g237935 for ; Wed, 3 Feb 2010 11:49:07 -0600 X-ASG-Debug-ID: 1265219414-797900e90000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C20321C9DE79 for ; Wed, 3 Feb 2010 09:50:15 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id bH7lx58rpevhwmqG for ; Wed, 03 Feb 2010 09:50:15 -0800 (PST) Received: from int-mx04.intmail.prod.int.phx2.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.17]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13HoEPV005226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 3 Feb 2010 12:50:14 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx04.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13HoDHI026860 for ; Wed, 3 Feb 2010 12:50:13 -0500 Message-ID: <4B69B755.2030008@sandeen.net> Date: Wed, 03 Feb 2010 11:50:13 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: [PATCH] increase readdir buffer size Subject: [PATCH] increase readdir buffer size Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.17 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265219415 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21562 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean While doing some testing of readdir perf a while back, I noticed that the buffer size we're using internally is smaller than what glibc gives us by default. Upping this size helped a bit, and seems safe. glibc's __alloc_dir() does: const size_t default_allocation = (4 * BUFSIZ < sizeof (struct dirent64) ? sizeof (struct dirent64) : 4 * BUFSIZ); const size_t small_allocation = (BUFSIZ < sizeof (struct dirent64) ? sizeof (struct dirent64) : BUFSIZ); size_t allocation = default_allocation; #ifdef _STATBUF_ST_BLKSIZE if (statp != NULL && default_allocation < statp->st_blksize) allocation = statp->st_blksize; #endif and #define _G_BUFSIZ 8192 #define _IO_BUFSIZ _G_BUFSIZ # define BUFSIZ _IO_BUFSIZ so the default buffer is 4 * 8192 = 32768 (except in the unlikely case of blocks > 32k....) Signed-off-by: Eric Sandeen --- diff --git a/fs/xfs/linux-2.6/xfs_file.c b/fs/xfs/linux-2.6/xfs_file.c index e4caeb2..44bcc67 100644 --- a/fs/xfs/linux-2.6/xfs_file.c +++ b/fs/xfs/linux-2.6/xfs_file.c @@ -203,9 +203,9 @@ xfs_file_readdir( * * Try to give it an estimate that's good enough, maybe at some * point we can change the ->readdir prototype to include the - * buffer size. + * buffer size. For now we use the current glibc buffer size. */ - bufsize = (size_t)min_t(loff_t, PAGE_SIZE, ip->i_d.di_size); + bufsize = (size_t)min_t(loff_t, 32768, ip->i_d.di_size); error = xfs_readdir(ip, dirent, bufsize, (xfs_off_t *)&filp->f_pos, filldir); From bpm@sgi.com Wed Feb 3 12:05:26 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13I5QiP238437 for ; Wed, 3 Feb 2010 12:05:26 -0600 Received: from whiskey.americas.sgi.com (whiskey.americas.sgi.com [128.162.233.19]) by relay3.corp.sgi.com (Postfix) with ESMTP id CA74FAC036; Wed, 3 Feb 2010 10:06:32 -0800 (PST) Received: by whiskey.americas.sgi.com (Postfix, from userid 4600) id EEB5F4266A1; Wed, 3 Feb 2010 12:07:06 -0600 (CST) Date: Wed, 3 Feb 2010 12:07:06 -0600 From: bpm@sgi.com To: Christoph Hellwig Cc: Dave Chinner , xfs@oss.sgi.com Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100203180706.GH5702@sgi.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> <20100203112753.GA19996@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203112753.GA19996@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hey Christoph, On Wed, Feb 03, 2010 at 06:27:53AM -0500, Christoph Hellwig wrote: > That leaves open the NFSD case thought. I'd prefer to have that fixed > if possibly. Ben, any chance you could send your patch to use fsync > to the nfs list ASAP? I'll make some time to work on it. > I think we'd be even better off to just force > -o wsync and disable ->write_inode entirely for NFS, any chance you > could test such a patch on your setup? Sure. IIRC, previous tests were w/ -o wsync and write_inode switched to fsync passing 1 for fdatasync. I'll try this too. -Ben From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 13:42:23 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13JgMbq241215 for ; Wed, 3 Feb 2010 13:42:23 -0600 X-ASG-Debug-ID: 1265226211-541f02140000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7B48B1C9E846 for ; Wed, 3 Feb 2010 11:43:32 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id DKgDPNNNKOZ3WaNO for ; Wed, 03 Feb 2010 11:43:32 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Ncl8N-0004Bw-Rv for xfs@oss.sgi.com; Wed, 03 Feb 2010 19:43:31 +0000 Date: Wed, 3 Feb 2010 14:43:31 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH] xfs: only clear the suid bit once in xfs_write Subject: [PATCH] xfs: only clear the suid bit once in xfs_write Message-ID: <20100203194331.GA20199@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265226212 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean file_remove_suid already calls into ->setattr to clear the suid and sgid bits if needed, no need to start a second transaction to do it ourselves. Note that xfs_write_clear_setuid issues a sync transaction while the path through ->setattr doesn't, but that is consistant with the other filesystems. Signed-off-by: Christoph Hellwig Index: xfs/fs/xfs/linux-2.6/xfs_lrw.c =================================================================== --- xfs.orig/fs/xfs/linux-2.6/xfs_lrw.c 2010-02-03 20:37:45.956003749 +0100 +++ xfs/fs/xfs/linux-2.6/xfs_lrw.c 2010-02-03 20:37:49.348003520 +0100 @@ -630,18 +630,9 @@ start: * by root. This keeps people from modifying setuid and * setgid binaries. */ - - if (((xip->i_d.di_mode & S_ISUID) || - ((xip->i_d.di_mode & (S_ISGID | S_IXGRP)) == - (S_ISGID | S_IXGRP))) && - !capable(CAP_FSETID)) { - error = xfs_write_clear_setuid(xip); - if (likely(!error)) - error = -file_remove_suid(file); - if (unlikely(error)) { - goto out_unlock_internal; - } - } + error = -file_remove_suid(file); + if (unlikely(error)) + goto out_unlock_internal; /* We can write back this queue in page reclaim */ current->backing_dev_info = mapping->backing_dev_info; Index: xfs/fs/xfs/xfs_rw.c =================================================================== --- xfs.orig/fs/xfs/xfs_rw.c 2010-02-03 20:37:45.965004126 +0100 +++ xfs/fs/xfs/xfs_rw.c 2010-02-03 20:37:49.349010212 +0100 @@ -47,48 +47,6 @@ #include "xfs_trace.h" /* - * This is a subroutine for xfs_write() and other writers (xfs_ioctl) - * which clears the setuid and setgid bits when a file is written. - */ -int -xfs_write_clear_setuid( - xfs_inode_t *ip) -{ - xfs_mount_t *mp; - xfs_trans_t *tp; - int error; - - mp = ip->i_mount; - tp = xfs_trans_alloc(mp, XFS_TRANS_WRITEID); - if ((error = xfs_trans_reserve(tp, 0, - XFS_WRITEID_LOG_RES(mp), - 0, 0, 0))) { - xfs_trans_cancel(tp, 0); - return error; - } - xfs_ilock(ip, XFS_ILOCK_EXCL); - xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); - xfs_trans_ihold(tp, ip); - ip->i_d.di_mode &= ~S_ISUID; - - /* - * Note that we don't have to worry about mandatory - * file locking being disabled here because we only - * clear the S_ISGID bit if the Group execute bit is - * on, but if it was on then mandatory locking wouldn't - * have been enabled. - */ - if (ip->i_d.di_mode & S_IXGRP) { - ip->i_d.di_mode &= ~S_ISGID; - } - xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); - xfs_trans_set_sync(tp); - error = xfs_trans_commit(tp, 0); - xfs_iunlock(ip, XFS_ILOCK_EXCL); - return 0; -} - -/* * Force a shutdown of the filesystem instantly while keeping * the filesystem consistent. We don't do an unmount here; just shutdown * the shop, make sure that absolutely nothing persistent happens to Index: xfs/fs/xfs/xfs_rw.h =================================================================== --- xfs.orig/fs/xfs/xfs_rw.h 2010-02-03 20:37:53.085003394 +0100 +++ xfs/fs/xfs/xfs_rw.h 2010-02-03 20:37:57.946033654 +0100 @@ -39,7 +39,6 @@ xfs_fsb_to_db(struct xfs_inode *ip, xfs_ /* * Prototypes for functions in xfs_rw.c. */ -extern int xfs_write_clear_setuid(struct xfs_inode *ip); extern int xfs_read_buf(struct xfs_mount *mp, xfs_buftarg_t *btp, xfs_daddr_t blkno, int len, uint flags, struct xfs_buf **bpp); From aelder@sgi.com Wed Feb 3 13:52:58 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Jqw0t241543 for ; Wed, 3 Feb 2010 13:52:58 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay2.corp.sgi.com (Postfix) with ESMTP id B27E83040B2 for ; Wed, 3 Feb 2010 11:54:05 -0800 (PST) Received: from [128.162.232.162] ([128.162.232.162]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 3 Feb 2010 13:54:05 -0600 Subject: xfstests: 219: fix awk filter for duplicate users From: Alex Elder Reply-To: aelder@sgi.com To: xfs@oss.sgi.com Content-Type: text/plain; charset="UTF-8" Date: Wed, 03 Feb 2010 13:54:04 -0600 Message-ID: <1265226844.2848.27.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 03 Feb 2010 19:54:05.0646 (UTC) FILETIME=[A431BEE0:01CAA50A] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The filter I added for removing duplicate users from the output of repquota didn't do the job very well. This fixes that, making it so the first time a user is seen its line is printed, not thereafter. Signed-off-by: Alex Elder --- 219 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Index: b/219 =================================================================== --- a/219 +++ b/219 @@ -86,7 +86,7 @@ test_accounting() done repquota -$type -s -n $SCRATCH_MNT | grep -v "^#0" | filter_scratch | - awk '/^#/ { if (! seen[$1]) { seen[$1]++; next; } } { print }' + awk '/^#/ { if (seen[$1]) next; seen[$1]++; } } { print; }' } # real QA test starts here From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 14:30:01 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13KU1D6243100 for ; Wed, 3 Feb 2010 14:30:01 -0600 X-ASG-Debug-ID: 1265229070-3a83002b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 199451C9E986 for ; Wed, 3 Feb 2010 12:31:10 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 824A2RNaoDysYjru for ; Wed, 03 Feb 2010 12:31:10 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NclsU-0006cJ-Ec; Wed, 03 Feb 2010 20:31:10 +0000 Date: Wed, 3 Feb 2010 15:31:10 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH] increase readdir buffer size Subject: Re: [PATCH] increase readdir buffer size Message-ID: <20100203203110.GA24818@infradead.org> References: <4B69B755.2030008@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B69B755.2030008@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265229071 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 11:50:13AM -0600, Eric Sandeen wrote: > so the default buffer is 4 * 8192 = 32768 > (except in the unlikely case of blocks > 32k....) > > Signed-off-by: Eric Sandeen Looks good, Reviewed-by: Christoph Hellwig From morfic@gmail.com Wed Feb 3 14:33:35 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13KXZEt243219 for ; Wed, 3 Feb 2010 14:33:35 -0600 X-ASG-Debug-ID: 1265229284-3ab000360000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-ew0-f227.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4309A1AA2F8 for ; Wed, 3 Feb 2010 12:34:44 -0800 (PST) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by cuda.sgi.com with ESMTP id Z014WhiIZ6ReOME3 for ; Wed, 03 Feb 2010 12:34:44 -0800 (PST) Received: by ewy27 with SMTP id 27so1810776ewy.18 for ; Wed, 03 Feb 2010 12:34:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=qXbPImkbiDbaTOjPdB1FrqQ0DHC50AQwfRjokLmot0Q=; b=cbaCOLBj1kJQx9Bah6hSgVgy3ALASSbEtvIBxJ0HKRN8e6Hku8ASGjASs6cQ5UA4fW 6qIBWhlUtjAWpzORRu83JSd8GP8A+pjmqVfz8gBqEj04v306qa7bcaLijL1UsskN30am wXiFopGzJCzu/E/7rOghfqCOO9ocvjopLBy2g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=G0+YKgjjyboa/Y0jl5ssaWpIQA1FZ59di4OU30uCB/Pza9NZ3gKylU/isISW9wALiZ fTxeq7X+xQhRBPpYeGT1V1Zq53BxFfsIUatMW4EJvt7jpG4dgciWdpYx22s5DVLzx0el wDWWEBf2zQNBqKthD3GEDSgmKmh7f6yPJb4eg= MIME-Version: 1.0 Received: by 10.216.87.6 with SMTP id x6mr42593wee.174.1265229283455; Wed, 03 Feb 2010 12:34:43 -0800 (PST) In-Reply-To: <13bb8ce11002021819l7e17dac4n5f58a9887a16fae2@mail.gmail.com> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> <13bb8ce11002021819l7e17dac4n5f58a9887a16fae2@mail.gmail.com> Date: Wed, 3 Feb 2010 14:34:43 -0600 Message-ID: <13bb8ce11002031234t4b02b199g5b8dfd65ea48aa13@mail.gmail.com> X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM From: Daniel Goller To: xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: mail-ew0-f227.google.com[209.85.219.227] X-Barracuda-Start-Time: 1265229285 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0334 1.0000 -1.8049 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.80 X-Barracuda-Spam-Status: No, SCORE=-1.80 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21572 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Any particular stress test you can suggest, to use once i try git for xfs on this Marvell box? To try to help bringing up issues faster than through normal use of this box? Thanks in advance for any suggestions, Daniel From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 14:37:24 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13KbN8I243397 for ; Wed, 3 Feb 2010 14:37:24 -0600 X-ASG-Debug-ID: 1265229513-54d303070000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D62E01C9EA39 for ; Wed, 3 Feb 2010 12:38:33 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id HjUgrfoPZpeYTLE5 for ; Wed, 03 Feb 2010 12:38:33 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nclzd-0008Qf-I8; Wed, 03 Feb 2010 20:38:33 +0000 Date: Wed, 3 Feb 2010 15:38:33 -0500 From: Christoph Hellwig To: Daniel Goller Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM Message-ID: <20100203203833.GA31908@infradead.org> References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> <13bb8ce11002021819l7e17dac4n5f58a9887a16fae2@mail.gmail.com> <13bb8ce11002031234t4b02b199g5b8dfd65ea48aa13@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <13bb8ce11002031234t4b02b199g5b8dfd65ea48aa13@mail.gmail.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265229513 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 02:34:43PM -0600, Daniel Goller wrote: > Any particular stress test you can suggest, to use once i try git for > xfs on this Marvell box? > To try to help bringing up issues faster than through normal use of this box? The primary use of the large buffers where this matters is the log. So you just need to cleanly unmount and mount the filesystem again, and the issues your previously reported should be gone. From sandeen@sandeen.net Wed Feb 3 14:42:46 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Kgk86243532 for ; Wed, 3 Feb 2010 14:42:46 -0600 X-ASG-Debug-ID: 1265229834-3aff003f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C42C51AA39A; Wed, 3 Feb 2010 12:43:54 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 1fwizFhxY863zvG6; Wed, 03 Feb 2010 12:43:54 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13KhsLG003138 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 3 Feb 2010 15:43:54 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13Khp16001049; Wed, 3 Feb 2010 15:43:53 -0500 Message-ID: <4B69E006.9020807@sandeen.net> Date: Wed, 03 Feb 2010 14:43:50 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: aelder@sgi.com CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfstests: 219: fix awk filter for duplicate users Subject: Re: xfstests: 219: fix awk filter for duplicate users References: <1265226844.2848.27.camel@doink1> In-Reply-To: <1265226844.2848.27.camel@doink1> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265229835 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0062 1.0000 -1.9802 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.98 X-Barracuda-Spam-Status: No, SCORE=-1.98 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21572 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Alex Elder wrote: > The filter I added for removing duplicate users from the > output of repquota didn't do the job very well. This > fixes that, making it so the first time a user is seen > its line is printed, not thereafter. > > Signed-off-by: Alex Elder > > --- > 219 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > Index: b/219 > =================================================================== > --- a/219 > +++ b/219 > @@ -86,7 +86,7 @@ test_accounting() > done > > repquota -$type -s -n $SCRATCH_MNT | grep -v "^#0" | > filter_scratch | > - awk '/^#/ { if (! seen[$1]) { seen[$1]++; next; } } { print }' > + awk '/^#/ { if (seen[$1]) next; seen[$1]++; } } { print; }' > } +awk: /^#/ { if (seen[$1]) next; seen[$1]++; } } { print; } +awk: ^ syntax error but thanks for making me feel better ;) w/o the extra "}" it works for me. (incidentally is a line continuation " \ " needed too? I guess it works without...) -Eric From sandeen@sandeen.net Wed Feb 3 14:45:05 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_56 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Kj5gN243633 for ; Wed, 3 Feb 2010 14:45:05 -0600 X-ASG-Debug-ID: 1265229973-3ed4005d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 979CA137607E for ; Wed, 3 Feb 2010 12:46:13 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id T5cn1SmYeds8yBoN for ; Wed, 03 Feb 2010 12:46:13 -0800 (PST) Received: from int-mx03.intmail.prod.int.phx2.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13KkCSH003726 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 3 Feb 2010 15:46:12 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx03.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13KkBQf024039; Wed, 3 Feb 2010 15:46:11 -0500 Message-ID: <4B69E093.6000105@sandeen.net> Date: Wed, 03 Feb 2010 14:46:11 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Daniel Goller CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: State of XFS on ARM Subject: Re: State of XFS on ARM References: <13bb8ce11002011924h611099feh4955eedcc6e588a6@mail.gmail.com> <20100202112300.GA23809@infradead.org> <13bb8ce11002021819l7e17dac4n5f58a9887a16fae2@mail.gmail.com> <13bb8ce11002031234t4b02b199g5b8dfd65ea48aa13@mail.gmail.com> In-Reply-To: <13bb8ce11002031234t4b02b199g5b8dfd65ea48aa13@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.16 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265229974 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21573 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Daniel Goller wrote: > Any particular stress test you can suggest, to use once i try git for > xfs on this Marvell box? > To try to help bringing up issues faster than through normal use of this box? Run it through the xfstests suite: http://git.kernel.org/?p=fs/xfs/xfstests-dev.git;a=summary (I have a sheevaplug too, thanks to some nice folks @Marvell, and I really need to get regular qa going on it in return... have been too busy so far) :( There's a readme in the tree, but basically just create a local.config like: # - setenv TEST_DEV "device containing TEST PARTITION" # - setenv TEST_DIR "mount point of TEST PARTITION" # - setenv SCRATCH_DEV "device containing SCRATCH PARTITION" # - setenv SCRATCH_MNT "mount point for SCRATCH PARTITION" TEST_DEV=/dev/sdb1 TEST_DIR=/mnt/test SCRATCH_DEV=/dev/sdb3 SCRATCH_MNT=/mnt/scratch mkfs.xfs $TEST_DEV, and then run #./check -g auto -Eric > Thanks in advance for any suggestions, > > Daniel > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 14:50:43 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13KogGQ243761 for ; Wed, 3 Feb 2010 14:50:42 -0600 X-ASG-Debug-ID: 1265230311-3ade005f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 24AD91AA47C; Wed, 3 Feb 2010 12:51:52 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id VkHqkhslpIrFQHCD; Wed, 03 Feb 2010 12:51:52 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcmCV-0004dH-RW; Wed, 03 Feb 2010 20:51:51 +0000 Date: Wed, 3 Feb 2010 15:51:51 -0500 From: Christoph Hellwig To: Alex Elder Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: xfstests: 219: fix awk filter for duplicate users Subject: Re: xfstests: 219: fix awk filter for duplicate users Message-ID: <20100203205151.GA11337@infradead.org> References: <1265226844.2848.27.camel@doink1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265226844.2848.27.camel@doink1> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265230312 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 01:54:04PM -0600, Alex Elder wrote: > The filter I added for removing duplicate users from the > output of repquota didn't do the job very well. This > fixes that, making it so the first time a user is seen > its line is printed, not thereafter. > > Signed-off-by: Alex Elder > > --- > 219 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > Index: b/219 > =================================================================== > --- a/219 > +++ b/219 > @@ -86,7 +86,7 @@ test_accounting() > done > > repquota -$type -s -n $SCRATCH_MNT | grep -v "^#0" | > filter_scratch | > - awk '/^#/ { if (! seen[$1]) { seen[$1]++; next; } } { print }' > + awk '/^#/ { if (seen[$1]) next; seen[$1]++; } } { print; }' The patch seems whitespace damages, and there's a "}" too many in the added line. After fixing that up 219 passes again for me. So more or less: Reviewed-by: Christoph Hellwig From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 14:54:26 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13KsP3m243862 for ; Wed, 3 Feb 2010 14:54:25 -0600 X-ASG-Debug-ID: 1265230535-3ae000660000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 70D841AA4DE; Wed, 3 Feb 2010 12:55:35 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id n8B2i2fS0LPHg1KA; Wed, 03 Feb 2010 12:55:35 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcmG5-000601-SE; Wed, 03 Feb 2010 20:55:33 +0000 Date: Wed, 3 Feb 2010 15:55:33 -0500 From: Christoph Hellwig To: bpm@sgi.com Cc: Christoph Hellwig , Dave Chinner , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100203205533.GB19415@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> <20100203112753.GA19996@infradead.org> <20100203180706.GH5702@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203180706.GH5702@sgi.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265230535 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 12:07:06PM -0600, bpm@sgi.com wrote: > Hey Christoph, > > On Wed, Feb 03, 2010 at 06:27:53AM -0500, Christoph Hellwig wrote: > > That leaves open the NFSD case thought. I'd prefer to have that fixed > > if possibly. Ben, any chance you could send your patch to use fsync > > to the nfs list ASAP? > > I'll make some time to work on it. > > > I think we'd be even better off to just force > > -o wsync and disable ->write_inode entirely for NFS, any chance you > > could test such a patch on your setup? > > Sure. IIRC, previous tests were w/ -o wsync and write_inode switched to > fsync passing 1 for fdatasync. I'll try this too. -o wsync should be very similar to using ->fsync or Dave's new ->write_inode. What the synchronous transaction does is causing a log force, which is exactly what fsync does in case the inode is still pinned (except not quite as optimal, but I'll send a patch for that) From BATV+db9c69c6839b08361670+2355+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 3 14:55:39 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Ktdsi243899 for ; Wed, 3 Feb 2010 14:55:39 -0600 X-ASG-Debug-ID: 1265230608-3b0e00630000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EEAEA1AA4F0; Wed, 3 Feb 2010 12:56:48 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id CwIxZ4EWPcQtT8ub; Wed, 03 Feb 2010 12:56:48 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NcmHI-0006GM-MG; Wed, 03 Feb 2010 20:56:48 +0000 Date: Wed, 3 Feb 2010 15:56:48 -0500 From: Christoph Hellwig To: Dave Chinner Cc: bpm@sgi.com, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100203205648.GA23116@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> <20100203112753.GA19996@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203112753.GA19996@infradead.org> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265230608 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 06:27:53AM -0500, Christoph Hellwig wrote: > Still not entirely happy with this one. The first one is that I think > the barriers in fsync are still too heavy for the normal sync use > case. I'd be more happy with exporting the body of xfs_fsync without > the cache flushes (and a ebtter name than xfs_fsync) and use that > for write_inode. > > That leaves open the NFSD case thought. I'd prefer to have that fixed > if possibly. Ben, any chance you could send your patch to use fsync > to the nfs list ASAP? I think we'd be even better off to just force > -o wsync and disable ->write_inode entirely for NFS, any chance you > could test such a patch on your setup? Thinking about it, we usually do cause a log buffer write from ->fsync which means we submit the barrier anyway. That might be the reason why you're not seeing the performance hit in your testing. With that I'm okay with the patch as-is for now, we can micro-optimize it later. Reviewed-by: Christoph Hellwig From sandeen@sandeen.net Wed Feb 3 15:34:24 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13LYN06245496 for ; Wed, 3 Feb 2010 15:34:24 -0600 X-ASG-Debug-ID: 1265232932-3aeb00d30000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9B0241AA772 for ; Wed, 3 Feb 2010 13:35:33 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id mahnfvFOJYja4NZi for ; Wed, 03 Feb 2010 13:35:33 -0800 (PST) Received: from int-mx03.intmail.prod.int.phx2.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13LZWns013460 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 3 Feb 2010 16:35:32 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx03.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13LZVNf004794 for ; Wed, 3 Feb 2010 16:35:32 -0500 Message-ID: <4B69EC23.8040207@sandeen.net> Date: Wed, 03 Feb 2010 15:35:31 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: [PATCH 0/3] resending a few xfstests patches as a series Subject: [PATCH 0/3] resending a few xfstests patches as a series Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.16 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265232933 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0249 1.0000 -1.8597 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.86 X-Barracuda-Spam-Status: No, SCORE=-1.86 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21576 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean These should be cleaned up & sane by now :) -Eric From sandeen@sandeen.net Wed Feb 3 15:36:07 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13La7Lv245566 for ; Wed, 3 Feb 2010 15:36:07 -0600 X-ASG-Debug-ID: 1265233036-3ed2010d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B62D81376462 for ; Wed, 3 Feb 2010 13:37:16 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id nqCyWPbM1sWwL99O for ; Wed, 03 Feb 2010 13:37:16 -0800 (PST) Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13LbG4V004051 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 3 Feb 2010 16:37:16 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13LbDFw003157 for ; Wed, 3 Feb 2010 16:37:15 -0500 Message-ID: <4B69EC89.5050400@sandeen.net> Date: Wed, 03 Feb 2010 15:37:13 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: [PATCH 1/3] xfstests: routine to create scratch of certain size Subject: [PATCH 1/3] xfstests: routine to create scratch of certain size References: <4B69EC23.8040207@sandeen.net> In-Reply-To: <4B69EC23.8040207@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265233036 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21577 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is needed for later enospc tests to be generic We have to explicitly call the mkfs rather than _scratch_mkfs since extN wants nr-of-blocks as an argument -after- the device name. Signed-off-by: Eric Sandeen --- diff --git a/common.rc b/common.rc index 1edea2f..0a02a2b 100644 --- a/common.rc +++ b/common.rc @@ -305,6 +305,29 @@ _scratch_mkfs() esac } +# Create fs of certain size on scratch device +# _scratch_mkfs_sized [optional blocksize] +_scratch_mkfs_sized() +{ + fssize=$1 + blocksize=$2 + [ -z "$blocksize" ] && blocksize=4096 + let blocks=$fssize/$blocksize + + case $FSTYP in + xfs) + _scratch_mkfs_xfs -d size=$fssize -b size=$blocksize + ;; + ext2|ext3|ext4) + /sbin/mkfs.$FSTYP -b $blocksize $SCRATCH_DEV $blocks + ;; + *) + _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized" + ;; + esac + _scratch_mkfs +} + # Emulate an N-data-disk stripe w/ various stripe units # _scratch_mkfs_geom [optional blocksize] _scratch_mkfs_geom() From sandeen@sandeen.net Wed Feb 3 15:37:31 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13LbUe2245631 for ; Wed, 3 Feb 2010 15:37:30 -0600 X-ASG-Debug-ID: 1265233119-3a8c01000000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 501611C9EDF7 for ; Wed, 3 Feb 2010 13:38:39 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 6WfDqYw4KPDcG5jr for ; Wed, 03 Feb 2010 13:38:39 -0800 (PST) Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13LcdwD000617 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 3 Feb 2010 16:38:39 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13LcbY8020879 for ; Wed, 3 Feb 2010 16:38:38 -0500 Message-ID: <4B69ECDD.8050806@sandeen.net> Date: Wed, 03 Feb 2010 15:38:37 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: [PATCH 2/3] xfstests: make 053 and 077 generic Subject: [PATCH 2/3] xfstests: make 053 and 077 generic References: <4B69EC23.8040207@sandeen.net> In-Reply-To: <4B69EC23.8040207@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265233120 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MARKETING_SUBJECT X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21576 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean 053 and 077 can be generic w/ a little tweaking. Also change 077's filler to something more likely to be on a random system under test, and add it to the enospc group. Signed-off-by: Eric Sandeen --- diff --git a/053 b/053 index 98644a7..439cbe6 100755 --- a/053 +++ b/053 @@ -38,18 +38,17 @@ trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 . ./common.attr # real QA test starts here -_supported_fs xfs +_supported_fs generic _supported_os Linux -[ ! -x /bin/chacl -a ! -x /usr/bin/chacl ] && _notrun "chacl command not found" - _require_scratch +_acl_requirements _acl_setup_ids _do_die_on_error=y test=$SCRATCH_MNT/test # make filesystem on scratch using the defaults -_do 'make filesystem on $SCRATCH_DEV' '_scratch_mkfs_xfs' +_do 'make filesystem on $SCRATCH_DEV' '_scratch_mkfs' _do 'mount filesytem' '_scratch_mount' # create test files and set acls @@ -84,7 +83,7 @@ list_acls() echo "acls before repair:" list_acls _do 'unmount $SCRATCH_DEV' 'umount $SCRATCH_DEV' -_do 'repair filesystem' '_scratch_xfs_repair' +_do 'repair filesystem' '_check_scratch_fs' _do 'mount filesytem' '_scratch_mount' echo "acls after repair: " list_acls diff --git a/077 b/077 index cdee8da..a3d9334 100755 --- a/077 +++ b/077 @@ -30,8 +30,8 @@ echo "QA output created by $seq" here=`pwd` tmp=/tmp/$$ status=1 -#filler=$here/../../linux -filler=/home/fsgqa/isms/2.4.x-xfs +# Something w/ enough data to fill 50M of fs... +filler=/lib/modules/ _cleanup() { @@ -44,14 +44,16 @@ trap "_cleanup; rm -f $tmp.*; exit \$status" 0 1 2 3 15 # get standard environment, filters and checks . ./common.rc . ./common.filter +. ./common.attr # real QA test starts here -_supported_fs xfs +_supported_fs generic _supported_os Linux -[ ! -d $filler ] && _notrun "No linux directory to source files from" +[ ! -d $filler ] && _notrun "No directory to source files from" _require_scratch +_acl_requirements echo "*** create filesystem" @@ -59,7 +61,8 @@ rm -f $seq.full umount $SCRATCH_DEV >/dev/null 2>&1 echo "*** MKFS ***" >>$seq.full echo "" >>$seq.full -_scratch_mkfs_xfs -dsize=50m >>$seq.full 2>&1 \ +let SIZE=50*1024*1024 +_scratch_mkfs_sized $SIZE >>$seq.full 2>&1 \ || _fail "mkfs failed" _scratch_mount >>$seq.full 2>&1 \ || _fail "mount failed" diff --git a/group b/group index 6b8528f..c66d965 100644 --- a/group +++ b/group @@ -187,7 +187,7 @@ deprecated 074 rw udf auto 075 rw udf auto quick 076 metadata rw udf auto quick -077 acl attr auto +077 acl attr auto enospc 078 growfs auto quick 079 acl attr ioctl metadata auto quick 080 rw ioctl From sandeen@sandeen.net Wed Feb 3 15:39:55 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13Ldsf9245713 for ; Wed, 3 Feb 2010 15:39:55 -0600 X-ASG-Debug-ID: 1265233263-05dd03490000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 219FD1AA7ED for ; Wed, 3 Feb 2010 13:41:03 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id SZLVtJwBn91yLe9m for ; Wed, 03 Feb 2010 13:41:03 -0800 (PST) Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o13Lf34U017053 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Wed, 3 Feb 2010 16:41:03 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o13Lf2HL021587 for ; Wed, 3 Feb 2010 16:41:02 -0500 Message-ID: <4B69ED6D.6030902@sandeen.net> Date: Wed, 03 Feb 2010 15:41:01 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: [PATCH 3/3] xfstests: rename _acl_requirements to _require_acls Subject: [PATCH 3/3] xfstests: rename _acl_requirements to _require_acls References: <4B69EC23.8040207@sandeen.net> In-Reply-To: <4B69EC23.8040207@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265233264 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21576 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Most requirement tests are named _require_foo, so just make this more consistent. Also remove a few redundant tests for /usr/bin/chacl since _require_acls covers that. Signed-off-by: Eric Sandeen --- diff --git a/051 b/051 index 238abe9..12febd3 100755 --- a/051 +++ b/051 @@ -73,7 +73,6 @@ _cleanup() _supported_fs xfs udf _supported_os Linux -[ -x /usr/bin/chacl ] || _notrun "chacl executable not found" [ -x $runas ] || _notrun "$runas executable not found" rm -f $seq.full @@ -82,7 +81,7 @@ _setup_testdir _need_to_be_root _acl_setup_ids -_acl_requirements +_require_acls # get dir cd $testdir diff --git a/053 b/053 index 439cbe6..ac2162d 100755 --- a/053 +++ b/053 @@ -42,7 +42,7 @@ _supported_fs generic _supported_os Linux _require_scratch -_acl_requirements +_require_acls _acl_setup_ids _do_die_on_error=y test=$SCRATCH_MNT/test diff --git a/067 b/067 index 5ab743e..f1c211f 100755 --- a/067 +++ b/067 @@ -41,10 +41,8 @@ trap "rm -f $tmp.*; exit \$status" 0 1 2 3 15 _supported_fs xfs _supported_os Linux -[ -x /usr/bin/chacl ] || _notrun "chacl executable not found" - _need_to_be_root -_acl_requirements +_require_acls _require_scratch # set up fs for 1K inodes diff --git a/077 b/077 index a3d9334..ea81c07 100755 --- a/077 +++ b/077 @@ -53,7 +53,7 @@ _supported_os Linux [ ! -d $filler ] && _notrun "No directory to source files from" _require_scratch -_acl_requirements +_require_acls echo "*** create filesystem" diff --git a/099 b/099 index 90af18f..b68bfcd 100755 --- a/099 +++ b/099 @@ -81,7 +81,8 @@ _supported_fs generic _supported_os IRIX _acl_setup_ids -_acl_requirements +_require_acls + [ -x $runas ] || _notrun "$runas executable not found" # get dir diff --git a/105 b/105 index 9544c66..aba1f6d 100755 --- a/105 +++ b/105 @@ -56,7 +56,7 @@ rm -f $seq.full _require_scratch _acl_setup_ids -_acl_requirements +_require_acls umount $SCRATCH_DEV >/dev/null 2>&1 echo "*** MKFS ***" >>$seq.full diff --git a/common.attr b/common.attr index d12cc02..5019884 100644 --- a/common.attr +++ b/common.attr @@ -112,7 +112,7 @@ _filter_aces_notypes() # test if acl code will work # -_acl_requirements() +_require_acls() { xfsdir=$TEST_DIR From aelder@sgi.com Wed Feb 3 16:34:26 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.4 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13MYQwJ248469 for ; Wed, 3 Feb 2010 16:34:26 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay3.corp.sgi.com (Postfix) with ESMTP id 41F06AC039; Wed, 3 Feb 2010 14:35:33 -0800 (PST) Received: from [128.162.232.155] ([128.162.232.155]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 3 Feb 2010 16:35:32 -0600 Subject: Re: xfstests: 219: fix awk filter for duplicate users From: Alex Elder Reply-To: aelder@sgi.com To: Christoph Hellwig Cc: xfs@oss.sgi.com In-Reply-To: <20100203205151.GA11337@infradead.org> References: <1265226844.2848.27.camel@doink1> <20100203205151.GA11337@infradead.org> Content-Type: text/plain; charset="UTF-8" Date: Wed, 03 Feb 2010 16:35:32 -0600 Message-ID: <1265236532.2585.0.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 03 Feb 2010 22:35:32.0770 (UTC) FILETIME=[322B9020:01CAA521] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 15:51 -0500, Christoph Hellwig wrote: > On Wed, Feb 03, 2010 at 01:54:04PM -0600, Alex Elder wrote: > > The filter I added for removing duplicate users from the > > output of repquota didn't do the job very well. This > > fixes that, making it so the first time a user is seen > > its line is printed, not thereafter. > > > > Signed-off-by: Alex Elder > > > > --- > > 219 | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > Index: b/219 > > =================================================================== > > --- a/219 > > +++ b/219 > > @@ -86,7 +86,7 @@ test_accounting() > > done > > > > repquota -$type -s -n $SCRATCH_MNT | grep -v "^#0" | > > filter_scratch | > > - awk '/^#/ { if (! seen[$1]) { seen[$1]++; next; } } { print }' > > + awk '/^#/ { if (seen[$1]) next; seen[$1]++; } } { print; }' > > The patch seems whitespace damages, and there's a "}" too many in > the added line. After fixing that up 219 passes again for me. > > So more or less: > > > Reviewed-by: Christoph Hellwig I copy and pasted the wrong thing into my e-mail. Sorry about that. I will fix both the white space and the curly brace before commit. Thanks for the review. -Alex From SRS0+Oto3+63+fromorbit.com=david@internode.on.net Wed Feb 3 17:01:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13N1V8U250597 for ; Wed, 3 Feb 2010 17:01:31 -0600 X-ASG-Debug-ID: 1265238158-3f49020d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A78B41376E6E for ; Wed, 3 Feb 2010 15:02:38 -0800 (PST) Received: from mail.internode.on.net (bld-mail13.adl6.internode.on.net [150.101.137.98]) by cuda.sgi.com with ESMTP id MdAZr5DEwMxju6Dm for ; Wed, 03 Feb 2010 15:02:38 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12574219-1927428 for multiple; Thu, 04 Feb 2010 09:32:37 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NcoF1-0002Sw-Az; Thu, 04 Feb 2010 10:02:35 +1100 Date: Thu, 4 Feb 2010 10:02:35 +1100 From: Dave Chinner To: Christoph Hellwig Cc: bpm@sgi.com, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100203230235.GB5332@discord.disaster> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> <20100203112753.GA19996@infradead.org> <20100203205648.GA23116@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203205648.GA23116@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail13.adl6.internode.on.net[150.101.137.98] X-Barracuda-Start-Time: 1265238160 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21581 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 03:56:48PM -0500, Christoph Hellwig wrote: > On Wed, Feb 03, 2010 at 06:27:53AM -0500, Christoph Hellwig wrote: > > Still not entirely happy with this one. The first one is that I think > > the barriers in fsync are still too heavy for the normal sync use > > case. I'd be more happy with exporting the body of xfs_fsync without > > the cache flushes (and a ebtter name than xfs_fsync) and use that > > for write_inode. > > > > That leaves open the NFSD case thought. I'd prefer to have that fixed > > if possibly. Ben, any chance you could send your patch to use fsync > > to the nfs list ASAP? I think we'd be even better off to just force > > -o wsync and disable ->write_inode entirely for NFS, any chance you > > could test such a patch on your setup? > > Thinking about it, we usually do cause a log buffer write from ->fsync > which means we submit the barrier anyway. That might be the reason > why you're not seeing the performance hit in your testing. With that > I'm okay with the patch as-is for now, we can micro-optimize it later. OK. I was thinking that a "inode cluster fsync" function might be appropriate here. i.e. a transaction that takes all the dirty inodes in the inode cluster and logs/forces them all in one transaction. That would substanitally reduce the number of log writes needed, I think. I'll look into this over the next few days. Thanks for the reviews, Christoph. Cheers, Dave. -- Dave Chinner david@fromorbit.com From SRS0+O7JB+63+fromorbit.com=david@internode.on.net Wed Feb 3 17:07:28 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o13N7Sxw251063 for ; Wed, 3 Feb 2010 17:07:28 -0600 X-ASG-Debug-ID: 1265238515-3a8002250000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C3B5C1C9F344 for ; Wed, 3 Feb 2010 15:08:36 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id nG1hZjoQx0xLyHHh for ; Wed, 03 Feb 2010 15:08:36 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12366963-1927428 for multiple; Thu, 04 Feb 2010 09:38:34 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NcoKn-0002TS-8Y; Thu, 04 Feb 2010 10:08:33 +1100 Date: Thu, 4 Feb 2010 10:08:33 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Subject: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Message-ID: <20100203230833.GC5332@discord.disaster> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-9-git-send-email-david@fromorbit.com> <20100203112917.GB19996@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203112917.GB19996@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1265238517 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21582 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 06:29:17AM -0500, Christoph Hellwig wrote: > On Wed, Feb 03, 2010 at 10:25:02AM +1100, Dave Chinner wrote: > > We have a need for a delayed write inode flush operation > > to be made atomically with an fsync to avoid physically > > writing inodes but still keeping inode buffer information > > up to date for bulkstat. > > > > Move the inode locking outside xfs_fsync() to allow this to > > be done. > > What's the point of the lock_flags argument? It should always > be IOLOCK_SHARED, so instead of passing it in as an argument > I'd rather add an assert to enforce it. Fair enough. Updated patch below. Cheers, Dave. -- Dave Chinner david@fromorbit.com xfs: move the inode locking outside xfs_fsync() V2 We have a need for a delayed write inode flush operation to be made atomically with an fsync to avoid physically writing inodes but still keeping inode buffer information up to date for bulkstat. Move the inode locking outside xfs_fsync() to allow this to be done. Version 2 - kill the lock_flags argument and simply assert XFS_IOLOCK_SHARED in xfs_fsync(). Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_file.c | 6 +++++- fs/xfs/linux-2.6/xfs_lrw.c | 3 ++- fs/xfs/xfs_vnodeops.c | 29 +++++++++++++---------------- 3 files changed, 20 insertions(+), 18 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_file.c b/fs/xfs/linux-2.6/xfs_file.c index e4caeb2..94d9d6d 100644 --- a/fs/xfs/linux-2.6/xfs_file.c +++ b/fs/xfs/linux-2.6/xfs_file.c @@ -177,9 +177,13 @@ xfs_file_fsync( int datasync) { struct xfs_inode *ip = XFS_I(dentry->d_inode); + int error; xfs_iflags_clear(ip, XFS_ITRUNCATED); - return -xfs_fsync(ip); + xfs_ilock(ip, XFS_ILOCK_SHARED); + error = -xfs_fsync(ip); + xfs_iunlock(ip, XFS_ILOCK_SHARED); + return error; } STATIC int diff --git a/fs/xfs/linux-2.6/xfs_lrw.c b/fs/xfs/linux-2.6/xfs_lrw.c index c80fa00..d7f1a71 100644 --- a/fs/xfs/linux-2.6/xfs_lrw.c +++ b/fs/xfs/linux-2.6/xfs_lrw.c @@ -754,11 +754,12 @@ write_retry: error = error2; if (need_i_mutex) mutex_lock(&inode->i_mutex); - xfs_ilock(xip, iolock); + xfs_ilock(xip, iolock | XFS_ILOCK_SHARED); error2 = xfs_fsync(xip); if (!error) error = error2; + xfs_iunlock(xip, XFS_ILOCK_SHARED); } out_unlock_internal: diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index 43241e2..b5689a4 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -590,6 +590,17 @@ xfs_readlink( * the I/O lock while flushing the data, and the inode lock while flushing the * inode. The inode lock CANNOT be held while flushing the data, so acquire * after we're done with that. + * + * We always need to make sure that the required inode state is safe on disk. + * The inode might be clean but we still might need to force the log because of + * committed transactions that haven't hit the disk yet. Likewise, there could + * be unflushed non-transactional changes to the inode core that have to go to + * disk and this requires us to issue a synchronous transaction to capture + * these changes correctly. + * + * This code relies on the assumption that if the update_* fields of the inode + * are clear and the inode is unpinned then it is clean and no action is + * required. */ int xfs_fsync( @@ -600,24 +611,11 @@ xfs_fsync( int log_flushed = 0; xfs_itrace_entry(ip); + ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED)); if (XFS_FORCED_SHUTDOWN(ip->i_mount)) return XFS_ERROR(EIO); - /* - * We always need to make sure that the required inode state is safe on - * disk. The inode might be clean but we still might need to force the - * log because of committed transactions that haven't hit the disk yet. - * Likewise, there could be unflushed non-transactional changes to the - * inode core that have to go to disk and this requires us to issue - * a synchronous transaction to capture these changes correctly. - * - * This code relies on the assumption that if the update_* fields - * of the inode are clear and the inode is unpinned then it is clean - * and no action is required. - */ - xfs_ilock(ip, XFS_ILOCK_SHARED); - if (!ip->i_update_core) { /* * Timestamps/size haven't changed since last inode flush or @@ -627,7 +625,6 @@ xfs_fsync( * disk yet, the inode will be still be pinned. If it is, * force the log. */ - xfs_iunlock(ip, XFS_ILOCK_SHARED); if (xfs_ipincount(ip)) { error = _xfs_log_force(ip->i_mount, XFS_LOG_SYNC, &log_flushed); @@ -662,7 +659,7 @@ xfs_fsync( xfs_trans_set_sync(tp); error = _xfs_trans_commit(tp, 0, &log_flushed); - xfs_iunlock(ip, XFS_ILOCK_EXCL); + xfs_ilock_demote(ip, XFS_ILOCK_EXCL); } if (ip->i_mount->m_flags & XFS_MOUNT_BARRIER) { From SRS0+KImH+64+fromorbit.com=david@internode.on.net Thu Feb 4 02:13:18 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o148DH75021666 for ; Thu, 4 Feb 2010 02:13:18 -0600 X-ASG-Debug-ID: 1265271264-4ced00580000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 070841ABBD5 for ; Thu, 4 Feb 2010 00:14:25 -0800 (PST) Received: from mail.internode.on.net (bld-mail18.adl2.internode.on.net [150.101.137.103]) by cuda.sgi.com with ESMTP id ShmQjAKduy193ocu for ; Thu, 04 Feb 2010 00:14:25 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12524248-1927428 for multiple; Thu, 04 Feb 2010 18:44:24 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1Ncwr0-0002yz-LC; Thu, 04 Feb 2010 19:14:22 +1100 Date: Thu, 4 Feb 2010 19:14:22 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100204081422.GF5332@discord.disaster> References: <20100201220813.GA3519@infradead.org> <20100203105545.GA1047@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203105545.GA1047@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail18.adl2.internode.on.net[150.101.137.103] X-Barracuda-Start-Time: 1265271267 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21616 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote: > Actually I'll take this one back - the log buffers aren't delwri > so xfs_flush_buftarg probably only helped by timing or the > xfs_buf_runall_queues on xfslogd_workqueue. Need to think about > this a bit more. Can you point me at the bug report? IIRC I've seen this in the past where we freed the log before we've done all the correct shutdown processing and they got fixed by correcting the order of shutdown to ensure the log is idle before freeing it... Cheers, Dave. -- Dave Chinner david@fromorbit.com From schulze@informatik.uni-tuebingen.de Thu Feb 4 05:09:19 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14B9IJa031753 for ; Thu, 4 Feb 2010 05:09:19 -0600 X-ASG-Debug-ID: 1265281827-0e0c00410000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx2.informatik.uni-tuebingen.de (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5FF1813790FA for ; Thu, 4 Feb 2010 03:10:27 -0800 (PST) Received: from mx2.informatik.uni-tuebingen.de (mx2.Informatik.Uni-Tuebingen.De [134.2.12.9]) by cuda.sgi.com with ESMTP id pn2iDz0vRTdrwri1 for ; Thu, 04 Feb 2010 03:10:27 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mx2.informatik.uni-tuebingen.de (Postfix) with ESMTP id 8B5DB345B for ; Thu, 4 Feb 2010 12:10:26 +0100 (MET) Received: from mx2.informatik.uni-tuebingen.de ([127.0.0.1]) by localhost (mx2.informatik.uni-tuebingen.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SONXO4gZcVjr for ; Thu, 4 Feb 2010 12:10:26 +0100 (MET) Received: from imap.informatik.uni-tuebingen.de (bluff.Informatik.Uni-Tuebingen.De [134.2.9.80]) by mx2.informatik.uni-tuebingen.de (Postfix) with ESMTP id 5FE173441 for ; Thu, 4 Feb 2010 12:10:26 +0100 (MET) Received: from zcs-bs.informatik.uni-tuebingen.de (zcs-bs.Informatik.Uni-Tuebingen.De [134.2.12.62]) by imap.informatik.uni-tuebingen.de (Postfix) with ESMTP id 4C0AC4CB8 for ; Thu, 4 Feb 2010 12:10:26 +0100 (MET) Received: from zcs-bs.informatik.uni-tuebingen.de (zcs-bs.Informatik.Uni-Tuebingen.De [134.2.12.62]) by zcs-bs.informatik.uni-tuebingen.de (Postfix) with ESMTP id 6CF4C364E8F for ; Thu, 4 Feb 2010 12:10:26 +0100 (CET) Date: Thu, 4 Feb 2010 12:10:26 +0100 (CET) From: Jan Schulze To: xfs@oss.sgi.com Message-ID: <13543215.8311265281826308.JavaMail.root@zcs-bs.Informatik.Uni-Tuebingen.De> X-ASG-Orig-Subj: re-enabling quota enforcement => invalid argument Subject: re-enabling quota enforcement => invalid argument MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [134.2.9.172] X-Mailer: Zimbra 5.0.19_GA_3083.MACOSXx86_10.5 (ZimbraWebClient - FF3.0 (Linux)/5.0.19_GA_3083.MACOSXx86_10.5) X-Barracuda-Connect: mx2.Informatik.Uni-Tuebingen.De[134.2.12.9] X-Barracuda-Start-Time: 1265281828 X-Barracuda-Bayes: INNOCENT GLOBAL 0.2409 1.0000 -0.6193 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -0.62 X-Barracuda-Spam-Status: No, SCORE=-0.62 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21626 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi all, I have an XFS filesystem with pquota accounting and enforcement. It was working well, but recently I had to disable enforcement for a short time. Now I want to switch it back on, but I get the error "XFS_QUOTAON: Invalid argument". Does anyone know, what to do? Any suggestions are welcome. Thanks in advance. [root@coffein raid]# xfs_quota -x -c "state -a" /raid User quota state on /raid (/dev/sdb1) Accounting: OFF Enforcement: OFF Inode: #18446744073709551615 (0 blocks, 0 extents) Group quota state on /raid (/dev/sdb1) Accounting: OFF Enforcement: OFF Inode: #259 (24 blocks, 3 extents) Project quota state on /raid (/dev/sdb1) Accounting: ON Enforcement: OFF Inode: #259 (24 blocks, 3 extents) Blocks grace time: [7 days 00:00:30] Inodes grace time: [7 days 00:00:30] Realtime Blocks grace time: [7 days 00:00:30] [root@coffein raid]# xfs_quota -x -c "enable -p -v" /raid XFS_QUOTAON: Invalid argument Best Regards, Jan From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 09:28:58 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14FSuod061237 for ; Thu, 4 Feb 2010 09:28:57 -0600 X-ASG-Debug-ID: 1265297406-0ce503390000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C19741CA2EC9; Thu, 4 Feb 2010 07:30:06 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id xtdA4AqJA7rOFsQu; Thu, 04 Feb 2010 07:30:06 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd3eg-0005H6-GS; Thu, 04 Feb 2010 15:30:06 +0000 Date: Thu, 4 Feb 2010 10:30:06 -0500 From: Christoph Hellwig To: Ben Myers Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 0/4] wsync export option Subject: Re: [RFC PATCH 0/4] wsync export option Message-ID: <20100204153006.GC22014@infradead.org> References: <20100203233755.17677.96582.stgit@case> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203233755.17677.96582.stgit@case> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265297406 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 05:44:24PM -0600, Ben Myers wrote: > The following series is adds a 'wsync' export option to nfsd. It is intended > to be used on XFS with the wsync mount option. When you already have a > synchronous log there is no need to sync metadata separately. You don't need the xfs wsync option, as the existing write_inode calls or your new fsync calls are doing the same as the wsync mount option, just from a higher layer. The wsync option causes the log to be synchronously forced up to the log sequence number of the commit for the metadata operation, that is make all the operations affected by it synchronous. That's exactly what we'll do using fsync (actually right now we force the whole log, but I have a patch to optimize it to only force nuntil the last commit lsn), and approqimately the same as we do using write_inode, just a lot less efficiently. > This is barely tested, YMMV, I could have this all wrong, etc, etc. Here are > some very unscientific measurements taken over gigabit ethernet. > > # time tar -xvf /mnt2/quilt-0.47.tar > /dev/null > > No XFS wsync, no NFS wsync: > 0m13.177s 0m13.301s 0m13.528s > > XFS wsync set, no NFS wsync: > 0m13.019s 0m13.232s 0m13.094s > > XFS wsync set, NFS wsync set: > 0m8.361s 0m8.400s 0m8.301s > > Curious to hear if this is a reasonable thing to do. Suggestions welcome. I think it's reasonable. What might be even better it to have an export operation call out into the filesystem so that we can force wsync and not let nfsd deal with it at all. There is a fair chance that the filesystem can do the sync more efficiently. And btw, not directly related to your patch, but getting rid of this write_inode call defintively makes the usage of ->write_inode more regular. From generic code we mostly use it for full inode writeback in writeback_single_inode, and one other special case in generic_detach_inode if a filesystem is unmounting. Only having writeback_single_inode and fs code use it will make this call much more predictable for the filesystem. From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 10:06:45 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14G6iYF072847 for ; Thu, 4 Feb 2010 10:06:44 -0600 X-ASG-Debug-ID: 1265299674-6c8703840000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 906621AF2D3 for ; Thu, 4 Feb 2010 08:07:54 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id jtDSlDS9nnuxAr23 for ; Thu, 04 Feb 2010 08:07:54 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd4FE-0005TF-Ol; Thu, 04 Feb 2010 16:07:52 +0000 Date: Thu, 4 Feb 2010 11:07:52 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Subject: Re: [PATCH 08/10] xfs: move the inode locking outside xfs_fsync() Message-ID: <20100204160752.GA11206@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-9-git-send-email-david@fromorbit.com> <20100203112917.GB19996@infradead.org> <20100203230833.GC5332@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203230833.GC5332@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265299674 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 10:08:33AM +1100, Dave Chinner wrote: > On Wed, Feb 03, 2010 at 06:29:17AM -0500, Christoph Hellwig wrote: > > On Wed, Feb 03, 2010 at 10:25:02AM +1100, Dave Chinner wrote: > > > We have a need for a delayed write inode flush operation > > > to be made atomically with an fsync to avoid physically > > > writing inodes but still keeping inode buffer information > > > up to date for bulkstat. > > > > > > Move the inode locking outside xfs_fsync() to allow this to > > > be done. > > > > What's the point of the lock_flags argument? It should always > > be IOLOCK_SHARED, so instead of passing it in as an argument > > I'd rather add an assert to enforce it. > > Fair enough. Updated patch below. > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > > xfs: move the inode locking outside xfs_fsync() V2 > > We have a need for a delayed write inode flush operation > to be made atomically with an fsync to avoid physically > writing inodes but still keeping inode buffer information > up to date for bulkstat. > > Move the inode locking outside xfs_fsync() to allow this to > be done. > > Version 2 > - kill the lock_flags argument and simply assert XFS_IOLOCK_SHARED > in xfs_fsync(). Looks good, Reviewed-by: Christoph Hellwig From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 10:09:41 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14G9fIf073359 for ; Thu, 4 Feb 2010 10:09:41 -0600 X-ASG-Debug-ID: 1265299851-729b010e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 91F85137AB36 for ; Thu, 4 Feb 2010 08:10:51 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id azdAHOhWzQHyNbWh for ; Thu, 04 Feb 2010 08:10:51 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd4I7-0006ed-0d; Thu, 04 Feb 2010 16:10:51 +0000 Date: Thu, 4 Feb 2010 11:10:50 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100204161050.GB11206@infradead.org> References: <20100201220813.GA3519@infradead.org> <20100203105545.GA1047@infradead.org> <20100204081422.GF5332@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204081422.GF5332@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265299851 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 07:14:22PM +1100, Dave Chinner wrote: > On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote: > > Actually I'll take this one back - the log buffers aren't delwri > > so xfs_flush_buftarg probably only helped by timing or the > > xfs_buf_runall_queues on xfslogd_workqueue. Need to think about > > this a bit more. > > Can you point me at the bug report? IIRC I've seen this in the > past where we freed the log before we've done all the correct > shutdown processing and they got fixed by correcting the > order of shutdown to ensure the log is idle before freeing > it... Yes, that I was going to look into next. The recent report is http://bugzilla.kernel.org/show_bug.cgi?id=15150 and there's also a very similar one from Ed Cashin in mail archives from November. From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 10:17:39 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_72 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14GHcwI074170 for ; Thu, 4 Feb 2010 10:17:39 -0600 X-ASG-Debug-ID: 1265300329-626e03c40000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 665A81AF382 for ; Thu, 4 Feb 2010 08:18:49 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id kwDuPgY2HvtoyW6c for ; Thu, 04 Feb 2010 08:18:49 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd4Pn-0000Oi-Li; Thu, 04 Feb 2010 16:18:47 +0000 Date: Thu, 4 Feb 2010 11:18:47 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Eric Sandeen , xfs mailing list X-ASG-Orig-Subj: Re: [PATCH, RFC] more reserved blocks fixups Subject: Re: [PATCH, RFC] more reserved blocks fixups Message-ID: <20100204161847.GA32102@infradead.org> References: <4B60C8EE.5080700@sandeen.net> <20100128015817.GH15853@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100128015817.GH15853@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265300329 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Jan 28, 2010 at 12:58:17PM +1100, Dave Chinner wrote: > On Wed, Jan 27, 2010 at 05:14:54PM -0600, Eric Sandeen wrote: > > This mangles the reserved blocks counts a little more. > > > > 1) add a helper function for the default reserved count > > 2) add helper functions to save/restore counts on ro/rw > > 3) save/restore reserved blocks on freeze/thaw > > 4) disallow changing reserved count while readonly > > > > for 2) - maybe better names (save_and_clear?) > > for 4) - maybe allow, but change the _ro field instead? > > > > (TBH not tested yet but wondered if this seems sane) > > I was wondering if the save/restore could be encapsualted entirely > within xfs_quiesce_attr(). i.e. save before the superblock write, > restore directly after. That removes the need for a variable in > the xfs_mount structure, and catches both remount,ro and freeze. Seems like a bit fragile. But I think we want Eric's version over yours for a start - we should have a clean filesystem not only after remount,ro but also for a frozen filesystem. From sandeen@redhat.com Thu Feb 4 10:32:58 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.4 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14GWvjK075103 for ; Thu, 4 Feb 2010 10:32:57 -0600 X-ASG-Debug-ID: 1265301246-4fa200e70000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id ED4571AF01C for ; Thu, 4 Feb 2010 08:34:06 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id zglE7WgDrQiknUzq for ; Thu, 04 Feb 2010 08:34:06 -0800 (PST) Received: from int-mx04.intmail.prod.int.phx2.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.17]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14GY5Ew013855 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 4 Feb 2010 11:34:06 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx04.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14GY5a5013670; Thu, 4 Feb 2010 11:34:05 -0500 Message-ID: <4B6AF6FC.6030101@redhat.com> Date: Thu, 04 Feb 2010 10:34:04 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list CC: Jim Meyering X-ASG-Orig-Subj: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid Subject: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.17 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265301246 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21646 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean from RH bug https://bugzilla.redhat.com/show_bug.cgi?id=561870 # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k # mkfs.xfs: probe of k failed, cannot detect existing filesystem. # mkfs.xfs: Use the -f option to force overwrite. blkid fails to do a probe of a regular file. I wish blkid would cope with this, but for now it might be better to just turn it off. Reported-by: Jim Meyering Signed-off-by: Eric Sandeen --- diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index 9baf116..de87647 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -300,10 +300,15 @@ check_overwrite( int fd; long long size; int bsz; + struct stat statbuf; if (!device || !*device) return 0; + /* blkid can't get info from a regular file */ + if (!stat(device, &statbuf) && S_ISREG(statbuf.st_mode)) + return 0; + ret = -1; /* will reset on success of all setup calls */ fd = open(device, O_RDONLY); From jim@meyering.net Thu Feb 4 10:45:16 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.9 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14GjFs5075961 for ; Thu, 4 Feb 2010 10:45:16 -0600 X-ASG-Debug-ID: 1265301984-4f2c01a70000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx.meyering.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9BBC41AF551 for ; Thu, 4 Feb 2010 08:46:25 -0800 (PST) Received: from mx.meyering.net (mx.meyering.net [82.230.74.64]) by cuda.sgi.com with ESMTP id irAlCiC5iTYgdhIx for ; Thu, 04 Feb 2010 08:46:25 -0800 (PST) Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id CF74F11EA; Thu, 4 Feb 2010 17:46:23 +0100 (CET) From: Jim Meyering To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid Subject: Re: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid In-Reply-To: <4B6AF6FC.6030101@redhat.com> (Eric Sandeen's message of "Thu, 04 Feb 2010 10:34:04 -0600") References: <4B6AF6FC.6030101@redhat.com> Date: Thu, 04 Feb 2010 17:46:23 +0100 Message-ID: <878wb9j6e8.fsf@meyering.net> Lines: 51 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Barracuda-Connect: mx.meyering.net[82.230.74.64] X-Barracuda-Start-Time: 1265301985 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21646 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Eric Sandeen wrote: > https://bugzilla.redhat.com/show_bug.cgi?id=561870 > > # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k > # mkfs.xfs: probe of k failed, cannot detect existing filesystem. > # mkfs.xfs: Use the -f option to force overwrite. > > blkid fails to do a probe of a regular file. > > I wish blkid would cope with this, but for now it might > be better to just turn it off. > > Reported-by: Jim Meyering > Signed-off-by: Eric Sandeen > --- > > diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c > index 9baf116..de87647 100644 > --- a/mkfs/xfs_mkfs.c > +++ b/mkfs/xfs_mkfs.c > @@ -300,10 +300,15 @@ check_overwrite( > int fd; > long long size; > int bsz; > + struct stat statbuf; > > if (!device || !*device) > return 0; > > + /* blkid can't get info from a regular file */ > + if (!stat(device, &statbuf) && S_ISREG(statbuf.st_mode)) > + return 0; > + > ret = -1; /* will reset on success of all setup calls */ > > fd = open(device, O_RDONLY); Hi Eric, Did you consider calling fstat after opening, rather than "stat" before? /* blkid can't get info from a regular file */ if (!fstat(fd, &statbuf) && S_ISREG(statbuf.st_mode)) { close(fd); return 0; } That's slightly more efficient, and not prone to confusion in the unlikely event that "device" changes inode between the stat and the open calls. From sandeen@sandeen.net Thu Feb 4 10:52:16 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14GqFIO076425 for ; Thu, 4 Feb 2010 10:52:16 -0600 X-ASG-Debug-ID: 1265302404-4ee902280000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8AF491AF3FA for ; Thu, 4 Feb 2010 08:53:24 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id SjwTRvTSRXe0K8t8 for ; Thu, 04 Feb 2010 08:53:24 -0800 (PST) Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14GrOfx013591 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 4 Feb 2010 11:53:24 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14GrNue020255 for ; Thu, 4 Feb 2010 11:53:23 -0500 Message-ID: <4B6AFB83.1070309@sandeen.net> Date: Thu, 04 Feb 2010 10:53:23 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: xfsprogs: to -DDEBUG or not to -DDEBUG? Subject: xfsprogs: to -DDEBUG or not to -DDEBUG? Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265302405 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21648 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Fedora, long ago, disabled debug in the xfsprogs specfile: http://cvs.fedoraproject.org/viewvc/F-12/xfsprogs/xfsprogs.spec?r1=1.5&r2=1.6 * Wed Apr 20 2005 Dave Jones - Disable debug. (#151438) per this bug: https://bugzilla.redhat.com/show_bug.cgi?id=151438 referencing this email thread: http://oss.sgi.com/archives/linux-xfs/2005-03/msg00038.html which no longer exists (grrrr) but is probably now here: http://oss.sgi.com/archives/xfs/2005-03/msg00416.html Fedora still builds with -DNDEBUG, but the upstream tarball has it on by default. I have seen several occasions where other distros had failing xfs_repairs which were "fixed" by disabling debug. I'd like to make a decision on whether DEBUG should be on or off by default for upstream releases. Any thoughts? Thanks, -Eric From sandeen@redhat.com Thu Feb 4 10:53:13 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14GrC3d076491 for ; Thu, 4 Feb 2010 10:53:13 -0600 X-ASG-Debug-ID: 1265302461-150301260000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 13C5E137B297 for ; Thu, 4 Feb 2010 08:54:21 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id dNfnfKULmc6IqO0e for ; Thu, 04 Feb 2010 08:54:21 -0800 (PST) Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14GsIuA014023 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 4 Feb 2010 11:54:19 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14GsI9s020669; Thu, 4 Feb 2010 11:54:18 -0500 Message-ID: <4B6AFBB9.7000701@redhat.com> Date: Thu, 04 Feb 2010 10:54:17 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Jim Meyering CC: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid Subject: Re: [PATCH] mkfs: don't try to detect filesystems on regular files via blkid References: <4B6AF6FC.6030101@redhat.com> <878wb9j6e8.fsf@meyering.net> In-Reply-To: <878wb9j6e8.fsf@meyering.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265302462 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21647 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Jim Meyering wrote: > Eric Sandeen wrote: >> https://bugzilla.redhat.com/show_bug.cgi?id=561870 >> >> # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k >> # mkfs.xfs: probe of k failed, cannot detect existing filesystem. >> # mkfs.xfs: Use the -f option to force overwrite. >> >> blkid fails to do a probe of a regular file. >> >> I wish blkid would cope with this, but for now it might >> be better to just turn it off. >> >> Reported-by: Jim Meyering >> Signed-off-by: Eric Sandeen >> --- >> >> diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c >> index 9baf116..de87647 100644 >> --- a/mkfs/xfs_mkfs.c >> +++ b/mkfs/xfs_mkfs.c >> @@ -300,10 +300,15 @@ check_overwrite( >> int fd; >> long long size; >> int bsz; >> + struct stat statbuf; >> >> if (!device || !*device) >> return 0; >> >> + /* blkid can't get info from a regular file */ >> + if (!stat(device, &statbuf) && S_ISREG(statbuf.st_mode)) >> + return 0; >> + >> ret = -1; /* will reset on success of all setup calls */ >> >> fd = open(device, O_RDONLY); > > Hi Eric, > > Did you consider calling fstat after opening, > rather than "stat" before? > > /* blkid can't get info from a regular file */ > if (!fstat(fd, &statbuf) && S_ISREG(statbuf.st_mode)) { > close(fd); > return 0; > } > > That's slightly more efficient, and not prone to confusion > in the unlikely event that "device" changes inode between > the stat and the open calls. Ok, yes that's probably better, will resend. thanks. -Eric From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 11:35:08 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14HZ6x8079462 for ; Thu, 4 Feb 2010 11:35:07 -0600 X-ASG-Debug-ID: 1265304976-5566008d0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9AFBD137B3FC; Thu, 4 Feb 2010 09:36:16 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id qAyUCGvitc9ZELFI; Thu, 04 Feb 2010 09:36:16 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd5ck-00047W-NT; Thu, 04 Feb 2010 17:36:14 +0000 Date: Thu, 4 Feb 2010 12:36:14 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , bpm@sgi.com, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Subject: Re: [PATCH 09/10] xfs: xfs_fs_write_inode() can fail to write inodes synchronously V2 Message-ID: <20100204173614.GA9498@infradead.org> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-10-git-send-email-david@fromorbit.com> <20100203112753.GA19996@infradead.org> <20100203205648.GA23116@infradead.org> <20100203230235.GB5332@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203230235.GB5332@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265304976 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean FYI I did some benchmarking on this, and the syncmodes 2 and 5 of fs_mark, which use sys_sync regress almost 10% on my test setup with this patch. The barriers are only a small part of it, from instrumentation it seems like the constant log forces don't really help. Now given that we only get data integrity writes from sync_filesystem do we really need to bother with catching all that pending I/O here? It would be much easier to rely on ->sync_fs to do that for us once, which is does anyway. From sandeen@sandeen.net Thu Feb 4 11:47:40 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14Hldw4080167 for ; Thu, 4 Feb 2010 11:47:40 -0600 X-ASG-Debug-ID: 1265305729-635600920000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D6977137B519 for ; Thu, 4 Feb 2010 09:48:49 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 96Cd6tqr4zVqOTj9 for ; Thu, 04 Feb 2010 09:48:49 -0800 (PST) Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14Hmn25023698 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 4 Feb 2010 12:48:49 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14HmmDf005889; Thu, 4 Feb 2010 12:48:48 -0500 Message-ID: <4B6B087F.80901@sandeen.net> Date: Thu, 04 Feb 2010 11:48:47 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Eric Sandeen CC: xfs mailing list , Jim Meyering X-ASG-Orig-Subj: [PATCH V2] mkfs: don't try to detect filesystems on regular files via blkid Subject: [PATCH V2] mkfs: don't try to detect filesystems on regular files via blkid References: <4B6AF6FC.6030101@redhat.com> In-Reply-To: <4B6AF6FC.6030101@redhat.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265305729 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21651 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean from RH bug https://bugzilla.redhat.com/show_bug.cgi?id=561870 # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k # mkfs.xfs: probe of k failed, cannot detect existing filesystem. # mkfs.xfs: Use the -f option to force overwrite. blkid fails to do a probe of a regular file. I wish blkid would cope with this, but for now it might be better to just turn it off. I kept the size==0 check just in case we stumble on a 0-sized device, blkid doesn't like that either... Reported-by: Jim Meyering Signed-off-by: Eric Sandeen --- V2: use fstat after the open diff --git a/mkfs/xfs_mkfs.c b/mkfs/xfs_mkfs.c index 9baf116..219c81e 100644 --- a/mkfs/xfs_mkfs.c +++ b/mkfs/xfs_mkfs.c @@ -300,6 +300,7 @@ check_overwrite( int fd; long long size; int bsz; + struct stat statbuf; if (!device || !*device) return 0; @@ -309,6 +310,14 @@ check_overwrite( fd = open(device, O_RDONLY); if (fd < 0) goto out; + + /* blkid can't get info from a regular file */ + if (!fstat(fd, &statbuf) && S_ISREG(statbuf.st_mode)) { + close(fd); + ret = 0; + goto out; + } + platform_findsizes(device, fd, &size, &bsz); close(fd); From bpm@sgi.com Thu Feb 4 12:13:47 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14IDk9M081747 for ; Thu, 4 Feb 2010 12:13:47 -0600 Received: from whiskey.americas.sgi.com (whiskey.americas.sgi.com [128.162.233.19]) by relay1.corp.sgi.com (Postfix) with ESMTP id 656C58F8066; Thu, 4 Feb 2010 10:14:54 -0800 (PST) Received: by whiskey.americas.sgi.com (Postfix, from userid 4600) id 957614266A1; Thu, 4 Feb 2010 12:15:29 -0600 (CST) Date: Thu, 4 Feb 2010 12:15:29 -0600 From: bpm@sgi.com To: Christoph Hellwig Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [RFC PATCH 0/4] wsync export option Message-ID: <20100204181529.GK5702@sgi.com> References: <20100203233755.17677.96582.stgit@case> <20100204153006.GC22014@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204153006.GC22014@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 10:30:06AM -0500, Christoph Hellwig wrote: > On Wed, Feb 03, 2010 at 05:44:24PM -0600, Ben Myers wrote: > > The following series is adds a 'wsync' export option to nfsd. It is intended > > to be used on XFS with the wsync mount option. When you already have a > > synchronous log there is no need to sync metadata separately. > > You don't need the xfs wsync option, as the existing write_inode calls > or your new fsync calls are doing the same as the wsync mount option, > just from a higher layer. Ok, I see that now. write_inode_now is basically a superset of what we need. The important thing is to force the log (which xfs_file_operations.xfs_file_fsync does do) but not to call xfs_super_operations.write_inode (via write_inode_now) which eventually gets into xfs_iflush and takes forever. We can take forever later-- when the log is full. ;) > The wsync option causes the log to be synchronously forced up to the > log sequence number of the commit for the metadata operation, that is > make all the operations affected by it synchronous. That's exactly > what we'll do using fsync (actually right now we force the whole log, > but I have a patch to optimize it to only force nuntil the last commit > lsn), and approqimately the same as we do using write_inode, just a > lot less efficiently. Rather than having X number of synchronous log transactions written separately as with wsync you have X number of log transactions written out together in one go by vfs_fsync (if datasync==0). That should be faster than wsync. > > Curious to hear if this is a reasonable thing to do. Suggestions > > welcome. > > I think it's reasonable. What might be even better it to have an > export operation call out into the filesystem so that we can force > wsync and not let nfsd deal with it at all. There is a fair chance > that the filesystem can do the sync more efficiently. Trond also suggested an export_operation and I think it's a good idea. I'll explore that and repost. Thanks, Ben From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 12:38:00 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14Ibws3083137 for ; Thu, 4 Feb 2010 12:38:00 -0600 X-ASG-Debug-ID: 1265308748-04a901120000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9F09D137BB97; Thu, 4 Feb 2010 10:39:08 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id D9R3Cjsl4DEoQawY; Thu, 04 Feb 2010 10:39:08 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd6bc-0003G8-3C; Thu, 04 Feb 2010 18:39:08 +0000 Date: Thu, 4 Feb 2010 13:39:08 -0500 From: Christoph Hellwig To: bpm@sgi.com Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 0/4] wsync export option Subject: Re: [RFC PATCH 0/4] wsync export option Message-ID: <20100204183908.GB9329@infradead.org> References: <20100203233755.17677.96582.stgit@case> <20100204153006.GC22014@infradead.org> <20100204181529.GK5702@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204181529.GK5702@sgi.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265308748 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 12:15:29PM -0600, bpm@sgi.com wrote: > Rather than having X number of synchronous log transactions written > separately as with wsync you have X number of log transactions written > out together in one go by vfs_fsync (if datasync==0). That should be > faster than wsync. Indeed, except that there aren't a lot of different transactions usually. - nfsd_setattr is one SETATTR transaction - nfsd_create might be multiple transactions, indeed - especially the nfsv3 variant that also adds a setattr transaction - nfsd_link should be a single one but yes, doing the log force from nfsd should be a benefit for the create side at least. The additional benefit is that we can just drive it from NFSD and don't need to force mount options on the fs. So yes, let's do it from nfsd. > Trond also suggested an export_operation and I think it's a good idea. > I'll explore that and repost. Indeed. For XFS that export_operation could probably be a lot simpler than xfs_fsync. I don't think we need to catch the non-transactional timestamp and size updates at all, and we're guaranteed the transaction has already commited. So the method might be as simple as: static int xfs_nfs_force_inode(struct inode *inode) { struct xfs_inode *ip = XFS_I(inode); xfs_ilock(ip, XFS_ILOCK_SHARED); if (xfs_ipincount(ip)) { xfs_lsn_t force_lsn = ip->i_itemp->ili_last_lsn; ASSERT(force_lsn); xfs_log_force_lsn(ip->i_mount, force_lsn, XFS_LOG_SYNC); } xfs_iunlock(ip, XFS_ILOCK_SHARED); return 0; } From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 13:38:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14JcoEL086373 for ; Thu, 4 Feb 2010 13:38:53 -0600 X-ASG-Debug-ID: 1265312400-4b4400520000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D46CA137C185 for ; Thu, 4 Feb 2010 11:40:00 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id eLJrXEZPJscbUibq for ; Thu, 04 Feb 2010 11:40:00 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nd7YW-00027E-7d; Thu, 04 Feb 2010 19:40:00 +0000 Date: Thu, 4 Feb 2010 14:40:00 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs mailing list , Jim Meyering X-ASG-Orig-Subj: [PATCH] mkfs.xfs fix detection of empty devices Subject: [PATCH] mkfs.xfs fix detection of empty devices Message-ID: <20100204194000.GA7229@infradead.org> References: <4B6AF6FC.6030101@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B6AF6FC.6030101@redhat.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265312400 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We currently fail to detect that a device does indeed not contain any signature and we are indeed fine to proceed with it due to mishandling the return value of blkid_do_fullprobe. Fix that up and add some better diagnostics of the blkid detection. Also remove the size == 0 check in check_overwrite as blkid handles that just fine. from RH bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=561870 # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k # mkfs.xfs: probe of k failed, cannot detect existing filesystem. # mkfs.xfs: Use the -f option to force overwrite Signed-off-by: Christoph Hellwig Index: xfsprogs-dev/mkfs/xfs_mkfs.c =================================================================== --- xfsprogs-dev.orig/mkfs/xfs_mkfs.c 2010-02-04 19:19:36.000000000 +0000 +++ xfsprogs-dev/mkfs/xfs_mkfs.c 2010-02-04 19:33:36.000000000 +0000 @@ -297,48 +297,47 @@ check_overwrite( const char *type; blkid_probe pr = NULL; int ret; - int fd; - long long size; - int bsz; if (!device || !*device) return 0; ret = -1; /* will reset on success of all setup calls */ - fd = open(device, O_RDONLY); - if (fd < 0) - goto out; - platform_findsizes(device, fd, &size, &bsz); - close(fd); - - /* nothing to overwrite on a 0-length device */ - if (size == 0) { - ret = 0; - goto out; - } - pr = blkid_new_probe_from_filename(device); if (!pr) goto out; - if (blkid_probe_enable_partitions(pr, 1)) + ret = blkid_probe_enable_partitions(pr, 1); + if (ret < 0) + goto out; + + ret = blkid_do_fullprobe(pr); + if (ret < 0) goto out; - if (blkid_do_fullprobe(pr)) + /* + * Blkid returns 1 for nothing found and 0 when it finds a signature, + * but we want the exact opposite, so reverse the return value here. + * + * In addition print some useful diagnostics about what actually is + * on the device. + */ + ret = !ret; + if (!ret) goto out; - ret = 0; if (!blkid_probe_lookup_value(pr, "TYPE", &type, NULL)) { fprintf(stderr, _("%s: %s appears to contain an existing " "filesystem (%s).\n"), progname, device, type); - ret = 1; } else if (!blkid_probe_lookup_value(pr, "PTTYPE", &type, NULL)) { fprintf(stderr, _("%s: %s appears to contain a partition " "table (%s).\n"), progname, device, type); - ret = 1; + } else { + fprintf(stderr, + _("%s: %s appears to contain something weird " + "according to blkid\n"), progname, device); } out: From sandeen@sandeen.net Thu Feb 4 13:42:17 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14JgGsv086471 for ; Thu, 4 Feb 2010 13:42:17 -0600 X-ASG-Debug-ID: 1265312606-4b1a00ac0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 08457137C1DF for ; Thu, 4 Feb 2010 11:43:26 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id o0wdCG5IOO4qKR84 for ; Thu, 04 Feb 2010 11:43:26 -0800 (PST) Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14JhQoE014427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Thu, 4 Feb 2010 14:43:26 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14JhPqn022520; Thu, 4 Feb 2010 14:43:25 -0500 Message-ID: <4B6B235D.5060903@sandeen.net> Date: Thu, 04 Feb 2010 13:43:25 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Eric Sandeen CC: Jim Meyering , xfs mailing list X-ASG-Orig-Subj: Re: [PATCH V2] mkfs: don't try to detect filesystems on regular files via blkid Subject: Re: [PATCH V2] mkfs: don't try to detect filesystems on regular files via blkid References: <4B6AF6FC.6030101@redhat.com> <4B6B087F.80901@sandeen.net> In-Reply-To: <4B6B087F.80901@sandeen.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265312607 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21659 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Eric Sandeen wrote: > from RH bug > https://bugzilla.redhat.com/show_bug.cgi?id=561870 > > # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k > # mkfs.xfs: probe of k failed, cannot detect existing filesystem. > # mkfs.xfs: Use the -f option to force overwrite. > > blkid fails to do a probe of a regular file. > > I wish blkid would cope with this, but for now it might > be better to just turn it off. > > I kept the size==0 check just in case we stumble on a 0-sized > device, blkid doesn't like that either... > > Reported-by: Jim Meyering > Signed-off-by: Eric Sandeen > --- OK NAK on that, hch has a better patch, we were driving blkid wrong. -Eric From sandeen@redhat.com Thu Feb 4 14:14:02 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43,J_CHICKENPOX_46 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14KE11x087949 for ; Thu, 4 Feb 2010 14:14:01 -0600 X-ASG-Debug-ID: 1265314509-7ecf01040000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B4E9B1B078B for ; Thu, 4 Feb 2010 12:15:09 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id rshstfpXcjV4PUtR for ; Thu, 04 Feb 2010 12:15:09 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o14KF2vF018882 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 4 Feb 2010 15:15:02 -0500 Received: from neon.msp.redhat.com (neon.msp.redhat.com [10.15.80.10]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o14KF19p032518; Thu, 4 Feb 2010 15:15:01 -0500 Message-ID: <4B6B2AC5.1070803@redhat.com> Date: Thu, 04 Feb 2010 14:15:01 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.21 (X11/20090320) MIME-Version: 1.0 To: Christoph Hellwig CC: xfs mailing list , Jim Meyering X-ASG-Orig-Subj: Re: [PATCH] mkfs.xfs fix detection of empty devices Subject: Re: [PATCH] mkfs.xfs fix detection of empty devices References: <4B6AF6FC.6030101@redhat.com> <20100204194000.GA7229@infradead.org> In-Reply-To: <20100204194000.GA7229@infradead.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265314511 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21660 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Christoph Hellwig wrote: > We currently fail to detect that a device does indeed not contain any > signature and we are indeed fine to proceed with it due to mishandling > the return value of blkid_do_fullprobe. Fix that up and add some > better diagnostics of the blkid detection. Also remove the size == 0 > check in check_overwrite as blkid handles that just fine. Much better, thanks, minor comment below > from RH bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=561870 > > # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k > # mkfs.xfs: probe of k failed, cannot detect existing filesystem. > # mkfs.xfs: Use the -f option to force overwrite > > Signed-off-by: Christoph Hellwig > > Index: xfsprogs-dev/mkfs/xfs_mkfs.c > =================================================================== > --- xfsprogs-dev.orig/mkfs/xfs_mkfs.c 2010-02-04 19:19:36.000000000 +0000 > +++ xfsprogs-dev/mkfs/xfs_mkfs.c 2010-02-04 19:33:36.000000000 +0000 > @@ -297,48 +297,47 @@ check_overwrite( > const char *type; > blkid_probe pr = NULL; > int ret; > - int fd; > - long long size; > - int bsz; > > if (!device || !*device) > return 0; > > ret = -1; /* will reset on success of all setup calls */ > > - fd = open(device, O_RDONLY); > - if (fd < 0) > - goto out; > - platform_findsizes(device, fd, &size, &bsz); > - close(fd); > - > - /* nothing to overwrite on a 0-length device */ > - if (size == 0) { > - ret = 0; > - goto out; > - } > - > pr = blkid_new_probe_from_filename(device); > if (!pr) > goto out; > > - if (blkid_probe_enable_partitions(pr, 1)) > + ret = blkid_probe_enable_partitions(pr, 1); > + if (ret < 0) > + goto out; > + > + ret = blkid_do_fullprobe(pr); > + if (ret < 0) > goto out; > > - if (blkid_do_fullprobe(pr)) > + /* > + * Blkid returns 1 for nothing found and 0 when it finds a signature, > + * but we want the exact opposite, so reverse the return value here. > + * > + * In addition print some useful diagnostics about what actually is > + * on the device. > + */ > + ret = !ret; > + if (!ret) > goto out; that makes my brain hurt a little. Maybe: if (ret == 1) { /* blkid found nothing */ ret = 0; /* we return 0 for nothing found */ goto out; } else /* blkid found something */ ret = 1; /* we return 1 for something found */ it's wordy but at least not confusing. I'm ok either way, I guess, so: Reviewed-by: Eric Sandeen unless you change anything enough that you want another review. :) -Eric > - ret = 0; > if (!blkid_probe_lookup_value(pr, "TYPE", &type, NULL)) { > fprintf(stderr, > _("%s: %s appears to contain an existing " > "filesystem (%s).\n"), progname, device, type); > - ret = 1; > } else if (!blkid_probe_lookup_value(pr, "PTTYPE", &type, NULL)) { > fprintf(stderr, > _("%s: %s appears to contain a partition " > "table (%s).\n"), progname, device, type); > - ret = 1; > + } else { > + fprintf(stderr, > + _("%s: %s appears to contain something weird " > + "according to blkid\n"), progname, device); > } > > out: From f2006075@bits-goa.ac.in Thu Feb 4 15:34:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.2 required=5.0 tests=BAYES_50,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14LYTJ8092562 for ; Thu, 4 Feb 2010 15:34:32 -0600 X-ASG-Debug-ID: 1265319314-2d5a00f90000-w1Z2WR X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from warrior.bits-goa.ac.in (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0D066137CC97 for ; Thu, 4 Feb 2010 13:35:15 -0800 (PST) Received: from warrior.bits-goa.ac.in (mail.bits-goa.ac.in [210.212.160.113]) by cuda.sgi.com with ESMTP id AhvC7j4fSmLiKdNQ for ; Thu, 04 Feb 2010 13:35:15 -0800 (PST) Received: from localhost (localhost.localdomain [127.0.0.1]) by warrior.bits-goa.ac.in (Postfix) with ESMTP id 6A322B3C8990; Fri, 5 Feb 2010 03:00:21 +0530 (IST) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at bits-goa.ac.in Received: from warrior.bits-goa.ac.in ([127.0.0.1]) by localhost (warrior.bits-goa.ac.in [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ECp0oGt-hu2l; Fri, 5 Feb 2010 03:00:21 +0530 (IST) Received: from warrior.bits-goa.ac.in (localhost.localdomain [127.0.0.1]) by warrior.bits-goa.ac.in (Postfix) with ESMTP id 0F0D0B3C8985; Fri, 5 Feb 2010 03:00:19 +0530 (IST) Date: Fri, 5 Feb 2010 03:00:19 +0530 (IST) From: DEEPAK AGRAWAL Reply-To: Canadian Agent Message-ID: <1784735951.311701265319019055.JavaMail.root@warrior.bits-goa.ac.in> X-ASG-Orig-Subj: Good News!! Subject: Good News!! MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Priority: 1 Importance: high X-Mailer: Zimbra 6.0.1_GA_1816.RHEL5_64 (zclient/6.0.1_GA_1816.RHEL5_64) To: undisclosed-recipients:; X-Barracuda-Connect: mail.bits-goa.ac.in[210.212.160.113] X-Barracuda-Start-Time: 1265319316 X-Barracuda-Bayes: INNOCENT GLOBAL 0.6842 1.0000 1.2521 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: 1.25 X-Barracuda-Spam-Status: No, SCORE=1.25 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MAILTO_TO_SPAM_ADDR X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21667 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 MAILTO_TO_SPAM_ADDR URI: Includes a link to a likely spammer email X-Virus-Status: Clean Your e-mail address has made you, a winner of 2,500,000GBP in the Canadian Lottery Promo. Contact our British Agent for more details. Mr. Trevor Allan e-mail: trevorallan24@gala.net , Tel:+447031979972. Thank You. Lotto Co-ordinator (Deepak Agrawal) From BATV+197641c031149925fcd8+2356+infradead.org+hch@bombadil.srs.infradead.org Thu Feb 4 16:17:14 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_21,J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o14MHCNK094794 for ; Thu, 4 Feb 2010 16:17:14 -0600 X-ASG-Debug-ID: 1265321902-413a03920000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2CD801CA4D17 for ; Thu, 4 Feb 2010 14:18:22 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id UXD2StiEXwT20KFm for ; Thu, 04 Feb 2010 14:18:22 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdA1m-00071a-6d; Thu, 04 Feb 2010 22:18:22 +0000 Date: Thu, 4 Feb 2010 17:18:22 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: Jim Meyering , xfs mailing list X-ASG-Orig-Subj: Re: [PATCH v2] mkfs.xfs fix detection of empty devices Subject: Re: [PATCH v2] mkfs.xfs fix detection of empty devices Message-ID: <20100204221822.GA26670@infradead.org> References: <4B6AF6FC.6030101@redhat.com> <20100204194000.GA7229@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204194000.GA7229@infradead.org> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265321903 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We currently fail to detect that a device does indeed not contain any signature and we are indeed fine to proceed with it due to mishandling the return value of blkid_do_fullprobe. Fix that up and add some better diagnostics of the blkid detection. from RH bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=561870 # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k # mkfs.xfs: probe of k failed, cannot detect existing filesystem. # mkfs.xfs: Use the -f option to force overwrite Signed-off-by: Christoph Hellwig Index: xfsprogs-dev/mkfs/xfs_mkfs.c =================================================================== --- xfsprogs-dev.orig/mkfs/xfs_mkfs.c 2010-02-04 20:30:26.000000000 +0000 +++ xfsprogs-dev/mkfs/xfs_mkfs.c 2010-02-04 20:43:41.000000000 +0000 @@ -322,24 +322,40 @@ check_overwrite( if (!pr) goto out; - if (blkid_probe_enable_partitions(pr, 1)) + ret = blkid_probe_enable_partitions(pr, 1); + if (ret < 0) goto out; - if (blkid_do_fullprobe(pr)) + ret = blkid_do_fullprobe(pr); + if (ret < 0) goto out; - ret = 0; + /* + * Blkid returns 1 for nothing found and 0 when it finds a signature, + * but we want the exact opposite, so reverse the return value here. + * + * In addition print some useful diagnostics about what actually is + * on the device. + */ + if (ret) { + ret = 0; + goto out; + } + if (!blkid_probe_lookup_value(pr, "TYPE", &type, NULL)) { fprintf(stderr, _("%s: %s appears to contain an existing " "filesystem (%s).\n"), progname, device, type); - ret = 1; } else if (!blkid_probe_lookup_value(pr, "PTTYPE", &type, NULL)) { fprintf(stderr, _("%s: %s appears to contain a partition " "table (%s).\n"), progname, device, type); - ret = 1; + } else { + fprintf(stderr, + _("%s: %s appears to contain something weird " + "according to blkid\n"), progname, device); } + ret = 1; out: if (pr) From SRS0+SkGd+65+fromorbit.com=david@internode.on.net Thu Feb 4 20:30:30 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o152UTBx106048 for ; Thu, 4 Feb 2010 20:30:29 -0600 X-ASG-Debug-ID: 1265337096-091d02760000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E73491B1906 for ; Thu, 4 Feb 2010 18:31:37 -0800 (PST) Received: from mail.internode.on.net (bld-mail13.adl6.internode.on.net [150.101.137.98]) by cuda.sgi.com with ESMTP id 1NclDPbYEYggCNwB for ; Thu, 04 Feb 2010 18:31:37 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12739903-1927428 for multiple; Fri, 05 Feb 2010 13:01:36 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdDyo-0004CO-Gd; Fri, 05 Feb 2010 13:31:34 +1100 Date: Fri, 5 Feb 2010 13:31:34 +1100 From: Dave Chinner To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: xfsprogs: to -DDEBUG or not to -DDEBUG? Subject: Re: xfsprogs: to -DDEBUG or not to -DDEBUG? Message-ID: <20100205023134.GA11483@discord.disaster> References: <4B6AFB83.1070309@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B6AFB83.1070309@sandeen.net> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail13.adl6.internode.on.net[150.101.137.98] X-Barracuda-Start-Time: 1265337098 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21686 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 10:53:23AM -0600, Eric Sandeen wrote: > Fedora, long ago, disabled debug in the xfsprogs specfile: > > http://cvs.fedoraproject.org/viewvc/F-12/xfsprogs/xfsprogs.spec?r1=1.5&r2=1.6 > > * Wed Apr 20 2005 Dave Jones > - Disable debug. (#151438) > > per this bug: https://bugzilla.redhat.com/show_bug.cgi?id=151438 > > referencing this email thread: > > http://oss.sgi.com/archives/linux-xfs/2005-03/msg00038.html > which no longer exists (grrrr) but is probably now here: > > http://oss.sgi.com/archives/xfs/2005-03/msg00416.html > > Fedora still builds with -DNDEBUG, but the upstream tarball has > it on by default. I have seen several occasions where other > distros had failing xfs_repairs which were "fixed" by disabling > debug. I'd like to make a decision on whether DEBUG should be > on or off by default for upstream releases. Any thoughts? Personally I'd prefer the repair process to stop if it comes across inconsistencies it can't handle. Ignoring them is as likely to "fix" the crash as it is to corrupt the filesystem more. On top of that we get bug reports when those problems are hit so we can look into what caused the problem. We don't really have a repair test suite that covers all the possible corruptions that can occur, so we are kind of reliant on users reporting conditions we've never had to handle before so we can fix them... Cheers, Dave. -- Dave Chinner david@fromorbit.com From SRS0+i3iU+65+fromorbit.com=david@internode.on.net Thu Feb 4 20:57:06 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o152v5SY106759 for ; Thu, 4 Feb 2010 20:57:06 -0600 X-ASG-Debug-ID: 1265338693-334400f30000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 13992137DFB5 for ; Thu, 4 Feb 2010 18:58:14 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id pU4cs6qm0Hs1YGUo for ; Thu, 04 Feb 2010 18:58:14 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12657070-1927428 for multiple; Fri, 05 Feb 2010 13:28:12 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdEOZ-0004Dy-Da; Fri, 05 Feb 2010 13:58:11 +1100 Date: Fri, 5 Feb 2010 13:58:11 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100205025811.GB11483@discord.disaster> References: <20100201220813.GA3519@infradead.org> <20100203105545.GA1047@infradead.org> <20100204081422.GF5332@discord.disaster> <20100204161050.GB11206@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100204161050.GB11206@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1265338696 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21687 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Thu, Feb 04, 2010 at 11:10:50AM -0500, Christoph Hellwig wrote: > On Thu, Feb 04, 2010 at 07:14:22PM +1100, Dave Chinner wrote: > > On Wed, Feb 03, 2010 at 05:55:45AM -0500, Christoph Hellwig wrote: > > > Actually I'll take this one back - the log buffers aren't delwri > > > so xfs_flush_buftarg probably only helped by timing or the > > > xfs_buf_runall_queues on xfslogd_workqueue. Need to think about > > > this a bit more. > > > > Can you point me at the bug report? IIRC I've seen this in the > > past where we freed the log before we've done all the correct > > shutdown processing and they got fixed by correcting the > > order of shutdown to ensure the log is idle before freeing > > it... > > Yes, that I was going to look into next. The recent report > is http://bugzilla.kernel.org/show_bug.cgi?id=15150 That was from 2.6.27. I didn't think that we finished the mount/unmount path cleanup that fixed all those problems until 2.6.28 or .29... > and there's > also a very similar one from Ed Cashin in > mail archives from November. Yeah, that's clearly IO completion after the call to xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call which waits for all hashed buffers on the buftarg except for those that are XBF_FS_MANAGED - the superblock. Looking at xfs_fs_put_super(), the superblock buffer is freed after xfs_unmountfs() frees the log, so if the superblock was logged and is undergoing IO then xfs_unmountfs_wait() would not have blocked on it and it could leak past the "no more IO should bein progress" barrier that xfs_unmountfs_wait() is supposed to provide. Does this look like a plausible cause of the problem? Is so, given that the superblock is the only buffer that is marked XBF_FS_MANAGED now and it seems to me that xfs_wait_buftarg() should be waiting on it, maybe the fix needed is to xfs_wait_buftarg().... Cheers, Dave. -- Dave Chinner david@fromorbit.com From sandeen@sandeen.net Thu Feb 4 21:40:18 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_21,J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o153eIAW108469 for ; Thu, 4 Feb 2010 21:40:18 -0600 X-ASG-Debug-ID: 1265341287-5e1602390000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 445661CA5A70 for ; Thu, 4 Feb 2010 19:41:27 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id 03voz1jwDRmyac2I for ; Thu, 04 Feb 2010 19:41:27 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 331B8124CF0A; Thu, 4 Feb 2010 21:41:27 -0600 (CST) Message-ID: <4B6B9366.6020600@sandeen.net> Date: Thu, 04 Feb 2010 21:41:26 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: Eric Sandeen , Jim Meyering , xfs mailing list X-ASG-Orig-Subj: Re: [PATCH v2] mkfs.xfs fix detection of empty devices Subject: Re: [PATCH v2] mkfs.xfs fix detection of empty devices References: <4B6AF6FC.6030101@redhat.com> <20100204194000.GA7229@infradead.org> <20100204221822.GA26670@infradead.org> In-Reply-To: <20100204221822.GA26670@infradead.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265341288 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21689 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Christoph Hellwig wrote: > We currently fail to detect that a device does indeed not contain any > signature and we are indeed fine to proceed with it due to mishandling > the return value of blkid_do_fullprobe. Fix that up and add some > better diagnostics of the blkid detection. > > from RH bugzilla https://bugzilla.redhat.com/show_bug.cgi?id=561870 > > # dd if=/dev/zero of=k bs=1MB count=2 seek=20; mkfs.xfs k > # mkfs.xfs: probe of k failed, cannot detect existing filesystem. > # mkfs.xfs: Use the -f option to force overwrite > > Signed-off-by: Christoph Hellwig we discussed keeping the 0-size check until libblkid copes better; also the absense of ret = !ret looks more readable, thanks. Reviewed-by: Eric Sandeen > Index: xfsprogs-dev/mkfs/xfs_mkfs.c > =================================================================== > --- xfsprogs-dev.orig/mkfs/xfs_mkfs.c 2010-02-04 20:30:26.000000000 +0000 > +++ xfsprogs-dev/mkfs/xfs_mkfs.c 2010-02-04 20:43:41.000000000 +0000 > @@ -322,24 +322,40 @@ check_overwrite( > if (!pr) > goto out; > > - if (blkid_probe_enable_partitions(pr, 1)) > + ret = blkid_probe_enable_partitions(pr, 1); > + if (ret < 0) > goto out; > > - if (blkid_do_fullprobe(pr)) > + ret = blkid_do_fullprobe(pr); > + if (ret < 0) > goto out; > > - ret = 0; > + /* > + * Blkid returns 1 for nothing found and 0 when it finds a signature, > + * but we want the exact opposite, so reverse the return value here. > + * > + * In addition print some useful diagnostics about what actually is > + * on the device. > + */ > + if (ret) { > + ret = 0; > + goto out; > + } > + > if (!blkid_probe_lookup_value(pr, "TYPE", &type, NULL)) { > fprintf(stderr, > _("%s: %s appears to contain an existing " > "filesystem (%s).\n"), progname, device, type); > - ret = 1; > } else if (!blkid_probe_lookup_value(pr, "PTTYPE", &type, NULL)) { > fprintf(stderr, > _("%s: %s appears to contain a partition " > "table (%s).\n"), progname, device, type); > - ret = 1; > + } else { > + fprintf(stderr, > + _("%s: %s appears to contain something weird " > + "according to blkid\n"), progname, device); > } > + ret = 1; > > out: > if (pr) > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 03:21:22 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o159LKCj125074 for ; Fri, 5 Feb 2010 03:21:22 -0600 X-ASG-Debug-ID: 1265361750-67a600f20000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1E2B51CA0265 for ; Fri, 5 Feb 2010 01:22:31 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id HhIA1QpyNAqQXt2u for ; Fri, 05 Feb 2010 01:22:31 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdKOT-0000ic-30; Fri, 05 Feb 2010 09:22:29 +0000 Date: Fri, 5 Feb 2010 04:22:29 -0500 From: Christoph Hellwig To: Dave Chinner Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: make install in the brave new build system world Subject: make install in the brave new build system world Message-ID: <20100205092229.GA32454@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265361751 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean When doing make install in xfsprogs I get a lot of spew like this: /usr/bin/make -C include install make[1]: Entering directory `/root/xfsprogs-dev/include' make[1]: Nothing to be done for `install'. make[1]: Leaving directory `/root/xfsprogs-dev/include' /usr/bin/make -C libxfs install make[1]: Entering directory `/root/xfsprogs-dev/libxfs' [DEP] gcc -MM -I. -g -O2 -DNDEBUG -DVERSION=\"3.1.1\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -DENABLE_GETTEXT -D_GNU_SOURCE -D_XOPEN_SOURCE=500 -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall cache.c init.c kmem.c logitem.c rdwr.c trans.c util.c xfs_alloc.c xfs_ialloc.c xfs_inode.c xfs_btree.c xfs_alloc_btree.c xfs_ialloc_btree.c xfs_bmap_btree.c xfs_da_btree.c xfs_dir2.c xfs_dir2_leaf.c xfs_attr_leaf.c xfs_dir2_block.c xfs_dir2_node.c xfs_dir2_data.c xfs_dir2_sf.c xfs_bmap.c xfs_mount.c xfs_rtalloc.c xfs_trans.c xfs_attr.c linux.c | /bin/sed -e 's,^\([^:]*\)\.o,\1.lo,' > .dep make[1]: Leaving directory `/root/xfsprogs-dev/libxfs' So it seems like for some reason we do a) regenerate the dependencies in the install target (we already re-did them once before as part of the all target implied by make install) b) for some reason the new silent make rules don't apply to this. Any idea why? From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 03:36:04 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o159a4Hg125947 for ; Fri, 5 Feb 2010 03:36:04 -0600 X-ASG-Debug-ID: 1265362635-3f2201090000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 482B71B21CC for ; Fri, 5 Feb 2010 01:37:15 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 9bXkrBOBTK2JFSrN for ; Fri, 05 Feb 2010 01:37:15 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdKck-0003q5-HV; Fri, 05 Feb 2010 09:37:14 +0000 Date: Fri, 5 Feb 2010 04:37:14 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH 1/3] xfstests: routine to create scratch of certain size Subject: Re: [PATCH 1/3] xfstests: routine to create scratch of certain size Message-ID: <20100205093714.GA14508@infradead.org> References: <4B69EC23.8040207@sandeen.net> <4B69EC89.5050400@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B69EC89.5050400@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265362635 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 03:37:13PM -0600, Eric Sandeen wrote: > This is needed for later enospc tests to be generic > > We have to explicitly call the mkfs rather than > _scratch_mkfs since extN wants nr-of-blocks as > an argument -after- the device name. > > Signed-off-by: Eric Sandeen Looks good, Reviewed-by: Christoph Hellwig From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 03:36:49 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o159anBB125997 for ; Fri, 5 Feb 2010 03:36:49 -0600 X-ASG-Debug-ID: 1265362679-66ee014a0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5900618302EE for ; Fri, 5 Feb 2010 01:37:59 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id D4kE2j9viQVT031Y for ; Fri, 05 Feb 2010 01:37:59 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdKdT-0003xh-2J; Fri, 05 Feb 2010 09:37:59 +0000 Date: Fri, 5 Feb 2010 04:37:59 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH 2/3] xfstests: make 053 and 077 generic Subject: Re: [PATCH 2/3] xfstests: make 053 and 077 generic Message-ID: <20100205093759.GB14508@infradead.org> References: <4B69EC23.8040207@sandeen.net> <4B69ECDD.8050806@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B69ECDD.8050806@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265362679 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 03:38:37PM -0600, Eric Sandeen wrote: > 053 and 077 can be generic w/ a little tweaking. > > Also change 077's filler to something more likely > to be on a random system under test, and add it > to the enospc group. Looks good, Reviewed-by: Christoph Hellwig From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 03:37:25 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o159bO4H126036 for ; Fri, 5 Feb 2010 03:37:24 -0600 X-ASG-Debug-ID: 1265362715-66e601440000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8080A1830D36 for ; Fri, 5 Feb 2010 01:38:35 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id G2ZSClIsLqZba2im for ; Fri, 05 Feb 2010 01:38:35 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdKe3-00040a-7g; Fri, 05 Feb 2010 09:38:35 +0000 Date: Fri, 5 Feb 2010 04:38:35 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs mailing list X-ASG-Orig-Subj: Re: [PATCH 3/3] xfstests: rename _acl_requirements to _require_acls Subject: Re: [PATCH 3/3] xfstests: rename _acl_requirements to _require_acls Message-ID: <20100205093835.GC14508@infradead.org> References: <4B69EC23.8040207@sandeen.net> <4B69ED6D.6030902@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B69ED6D.6030902@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265362715 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 03, 2010 at 03:41:01PM -0600, Eric Sandeen wrote: > Most requirement tests are named _require_foo, so just > make this more consistent. > > Also remove a few redundant tests for /usr/bin/chacl > since _require_acls covers that. Looks good, Reviewed-by: Christoph Hellwig From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 03:56:46 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o159uiST127034 for ; Fri, 5 Feb 2010 03:56:45 -0600 X-ASG-Debug-ID: 1265363875-3e7e01680000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id AD00C1B224D for ; Fri, 5 Feb 2010 01:57:55 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id DVpoBOpPdQmnqH4W for ; Fri, 05 Feb 2010 01:57:55 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdKwl-00083S-Cj for xfs@oss.sgi.com; Fri, 05 Feb 2010 09:57:55 +0000 Date: Fri, 5 Feb 2010 04:57:55 -0500 From: Christoph Hellwig To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH] xfs: optimize log flushing in xfs_fsync Subject: [PATCH] xfs: optimize log flushing in xfs_fsync Message-ID: <20100205095755.GA30848@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265363875 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean If we have a pinned inode it must have a log item attached to it. Usually that log item will have ili_last_lsn already set, in which case we only need to flush the log up to that LSN instead of doing a full log force. This gives speedups of about 5% in some fsync heavy workloads. Signed-off-by: Christoph Hellwig Index: xfs/fs/xfs/xfs_vnodeops.c =================================================================== --- xfs.orig/fs/xfs/xfs_vnodeops.c 2010-02-04 17:38:33.679254119 +0100 +++ xfs/fs/xfs/xfs_vnodeops.c 2010-02-04 17:38:51.606006156 +0100 @@ -626,8 +626,14 @@ xfs_fsync( * force the log. */ if (xfs_ipincount(ip)) { - error = _xfs_log_force(ip->i_mount, XFS_LOG_SYNC, - &log_flushed); + if (ip->i_itemp->ili_last_lsn) { + error = _xfs_log_force_lsn(ip->i_mount, + ip->i_itemp->ili_last_lsn, + XFS_LOG_SYNC, &log_flushed); + } else { + error = _xfs_log_force(ip->i_mount, + XFS_LOG_SYNC, &log_flushed); + } } } else { /* From SRS0+SkGd+65+fromorbit.com=david@internode.on.net Fri Feb 5 04:43:09 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15Ah8Hh129712 for ; Fri, 5 Feb 2010 04:43:09 -0600 X-ASG-Debug-ID: 1265366656-388002390000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B8D1E137ED00 for ; Fri, 5 Feb 2010 02:44:17 -0800 (PST) Received: from mail.internode.on.net (bld-mail13.adl6.internode.on.net [150.101.137.98]) by cuda.sgi.com with ESMTP id yKYYcC3a8WHJ4cZJ for ; Fri, 05 Feb 2010 02:44:17 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12790495-1927428 for multiple; Fri, 05 Feb 2010 21:14:16 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdLfa-0004cB-NL; Fri, 05 Feb 2010 21:44:14 +1100 Date: Fri, 5 Feb 2010 21:44:14 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100205104414.GC11483@discord.disaster> References: <20100205092229.GA32454@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205092229.GA32454@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail13.adl6.internode.on.net[150.101.137.98] X-Barracuda-Start-Time: 1265366658 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21714 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 04:22:29AM -0500, Christoph Hellwig wrote: > When doing make install in xfsprogs I get a lot of spew like this: > > /usr/bin/make -C include install > make[1]: Entering directory `/root/xfsprogs-dev/include' > make[1]: Nothing to be done for `install'. > make[1]: Leaving directory `/root/xfsprogs-dev/include' > /usr/bin/make -C libxfs install > make[1]: Entering directory `/root/xfsprogs-dev/libxfs' > [DEP] > gcc -MM -I. -g -O2 -DNDEBUG -DVERSION=\"3.1.1\" -DLOCALEDIR=\"/usr/share/locale\" -DPACKAGE=\"xfsprogs\" -I../include -DENABLE_GETTEXT -D_GNU_SOURCE -D_XOPEN_SOURCE=500 -D_FILE_OFFSET_BITS=64 -funsigned-char -fno-strict-aliasing -Wall cache.c init.c kmem.c logitem.c rdwr.c trans.c util.c xfs_alloc.c xfs_ialloc.c xfs_inode.c xfs_btree.c xfs_alloc_btree.c xfs_ialloc_btree.c xfs_bmap_btree.c xfs_da_btree.c xfs_dir2.c xfs_dir2_leaf.c xfs_attr_leaf.c xfs_dir2_block.c xfs_dir2_node.c xfs_dir2_data.c xfs_dir2_sf.c xfs_bmap.c xfs_mount.c xfs_rtalloc.c xfs_trans.c xfs_attr.c linux.c | /bin/sed -e 's,^\([^:]*\)\.o,\1.lo,' > .dep > make[1]: Leaving directory `/root/xfsprogs-dev/libxfs' > > > So it seems like for some reason we do > > a) regenerate the dependencies in the install target (we already re-did > them once before as part of the all target implied by make install) The dependencies are always regenerated due to the default target requiring the depend target. Rebuilding the dependencies is the only way to catch changes between builds and so ensure the correct files are rebuilt. The install target is building the dependencies because it has a dependency on the default target. > b) for some reason the new silent make rules don't apply to this. That's something I can't answer off the top of my head. I'll have a look into it. Cheers, Dave. -- Dave Chinner david@fromorbit.com From SRS0+SkGd+65+fromorbit.com=david@internode.on.net Fri Feb 5 05:16:35 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15BGYaR131657 for ; Fri, 5 Feb 2010 05:16:35 -0600 X-ASG-Debug-ID: 1265368663-374302fb0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7783F1372204 for ; Fri, 5 Feb 2010 03:17:44 -0800 (PST) Received: from mail.internode.on.net (bld-mail13.adl6.internode.on.net [150.101.137.98]) by cuda.sgi.com with ESMTP id NJBENhS7DuaYY0Ur for ; Fri, 05 Feb 2010 03:17:44 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12792998-1927428 for multiple; Fri, 05 Feb 2010 21:47:43 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdMBx-0004e3-JI; Fri, 05 Feb 2010 22:17:41 +1100 Date: Fri, 5 Feb 2010 22:17:41 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100205111741.GD11483@discord.disaster> References: <20100205092229.GA32454@infradead.org> <20100205104414.GC11483@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205104414.GC11483@discord.disaster> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail13.adl6.internode.on.net[150.101.137.98] X-Barracuda-Start-Time: 1265368665 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0002 1.0000 -2.0196 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21715 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 09:44:14PM +1100, Dave Chinner wrote: > On Fri, Feb 05, 2010 at 04:22:29AM -0500, Christoph Hellwig wrote: > > When doing make install in xfsprogs I get a lot of spew like this: .... > > b) for some reason the new silent make rules don't apply to this. > > That's something I can't answer off the top of my head. I'll have a > look into it. Try the patch below. Cheers, Dave. -- Dave Chinner david@fromorbit.com xfsprogs: clean up make install build The install targets did not get the silent treatment like the normal build targets. Shut them up. Signed-off-by: Dave Chinner --- Makefile | 9 ++++++--- include/buildrules | 4 ---- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 62c4258..31be846 100644 --- a/Makefile +++ b/Makefile @@ -107,13 +107,16 @@ install-dev: default $(addsuffix -install-dev,$(SUBDIRS)) install-qa: install $(addsuffix -install-qa,$(SUBDIRS)) %-install: - $(MAKE) -C $* install + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install %-install-dev: - $(MAKE) -C $* install-dev + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install-dev %-install-qa: - $(MAKE) -C $* install-qa + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install-qa distclean: clean $(Q)rm -f $(DISTDIRT) diff --git a/include/buildrules b/include/buildrules index 1695e23..beb469b 100644 --- a/include/buildrules +++ b/include/buildrules @@ -101,7 +101,3 @@ ltdepend: $(CFILES) $(HFILES) depend: $(CFILES) $(HFILES) @echo " [DEP]" $(Q)$(MAKEDEP) $(CFILES) > .dep - - -# $(Q)$(MAKEDEP) $(CFILES) | $(SED) -e 's,^\([^:]*\)\.o,\1,' > .dep - From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 05:26:39 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15BQcfS132220 for ; Fri, 5 Feb 2010 05:26:39 -0600 X-ASG-Debug-ID: 1265369268-6737035b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 66A8E1CA65FD for ; Fri, 5 Feb 2010 03:27:49 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id FJGS6OV48fkTHGV9 for ; Fri, 05 Feb 2010 03:27:49 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdMLj-0002Rt-Db; Fri, 05 Feb 2010 11:27:47 +0000 Date: Fri, 5 Feb 2010 06:27:47 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100205112747.GA28701@infradead.org> References: <20100205092229.GA32454@infradead.org> <20100205104414.GC11483@discord.disaster> <20100205111741.GD11483@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205111741.GD11483@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265369269 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 10:17:41PM +1100, Dave Chinner wrote: > On Fri, Feb 05, 2010 at 09:44:14PM +1100, Dave Chinner wrote: > > On Fri, Feb 05, 2010 at 04:22:29AM -0500, Christoph Hellwig wrote: > > > When doing make install in xfsprogs I get a lot of spew like this: > .... > > > b) for some reason the new silent make rules don't apply to this. > > > > That's something I can't answer off the top of my head. I'll have a > > look into it. > > Try the patch below. That makes it a bit better, but it still prints the full INSTALL and LTINSTALL lines. From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 05:27:40 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15BRdRi132283 for ; Fri, 5 Feb 2010 05:27:40 -0600 X-ASG-Debug-ID: 1265369330-66d303540000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C42A81CA6608 for ; Fri, 5 Feb 2010 03:28:50 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id iS1k80OMuiVbn8HK for ; Fri, 05 Feb 2010 03:28:50 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdMMk-0002jv-ER; Fri, 05 Feb 2010 11:28:50 +0000 Date: Fri, 5 Feb 2010 06:28:50 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100205112850.GB28701@infradead.org> References: <20100205092229.GA32454@infradead.org> <20100205104414.GC11483@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205104414.GC11483@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265369330 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 09:44:14PM +1100, Dave Chinner wrote: > The dependencies are always regenerated due to the default target > requiring the depend target. Rebuilding the dependencies is the only > way to catch changes between builds and so ensure the correct files > are rebuilt. > > The install target is building the dependencies because it has a > dependency on the default target. Well, it's building the dependecies twice - once by invoking the default target, but they are also rebuilt again when the actuall install rules are called. The latter is pretty clearly superflous. From BATV+cb6c268b990492087475+2357+infradead.org+hch@bombadil.srs.infradead.org Fri Feb 5 05:34:42 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15BYfsZ132820 for ; Fri, 5 Feb 2010 05:34:41 -0600 X-ASG-Debug-ID: 1265369751-3e8502c50000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8C3A71B2557 for ; Fri, 5 Feb 2010 03:35:51 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id SEuZKHlXAe2yrfjm for ; Fri, 05 Feb 2010 03:35:51 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NdMTX-0005CY-1Z; Fri, 05 Feb 2010 11:35:51 +0000 Date: Fri, 5 Feb 2010 06:35:50 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Subject: Re: [PATCH for-2.6.33] xfs: flush all log buffers in xlog_dealloc_log Message-ID: <20100205113550.GA10926@infradead.org> References: <20100201220813.GA3519@infradead.org> <20100203105545.GA1047@infradead.org> <20100204081422.GF5332@discord.disaster> <20100204161050.GB11206@infradead.org> <20100205025811.GB11483@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205025811.GB11483@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265369752 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 01:58:11PM +1100, Dave Chinner wrote: > > and there's > > also a very similar one from Ed Cashin in > > mail archives from November. > > Yeah, that's clearly IO completion after the call to > xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call > which waits for all hashed buffers on the buftarg except for > those that are XBF_FS_MANAGED - the superblock. > > Looking at xfs_fs_put_super(), the superblock buffer is freed after > xfs_unmountfs() frees the log, so if the superblock was logged > and is undergoing IO then xfs_unmountfs_wait() would not have > blocked on it and it could leak past the "no more IO should > bein progress" barrier that xfs_unmountfs_wait() is supposed to > provide. > > Does this look like a plausible cause of the problem? We just did a xfs_unmountfs_writesb before the call to xfs_unmountfs_wait, so the superblock buffer should be a sync buffer at that point, so something more fishy is going on here. syncd is also stopped before that call, so that gets rid of another possible case of an async buffer pending. Independent of that I think xfs_wait_buftarg waiting for the SB buffer seems correct to me, but I fear something else is going on here. From SRS0+1LZ9+65+fromorbit.com=david@internode.on.net Fri Feb 5 05:47:51 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15Bloiu133447 for ; Fri, 5 Feb 2010 05:47:51 -0600 X-ASG-Debug-ID: 1265370539-3f3103550000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C41871B2222 for ; Fri, 5 Feb 2010 03:48:59 -0800 (PST) Received: from mail.internode.on.net (bld-mail18.adl2.internode.on.net [150.101.137.103]) by cuda.sgi.com with ESMTP id hR8SjiKUiSfj7eSX for ; Fri, 05 Feb 2010 03:48:59 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12677688-1927428 for multiple; Fri, 05 Feb 2010 22:18:58 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdMgC-0004fl-Mz; Fri, 05 Feb 2010 22:48:56 +1100 Date: Fri, 5 Feb 2010 22:48:56 +1100 From: Dave Chinner To: Christoph Hellwig Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100205114856.GE11483@discord.disaster> References: <20100205092229.GA32454@infradead.org> <20100205104414.GC11483@discord.disaster> <20100205112850.GB28701@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205112850.GB28701@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail18.adl2.internode.on.net[150.101.137.103] X-Barracuda-Start-Time: 1265370540 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21716 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 06:28:50AM -0500, Christoph Hellwig wrote: > On Fri, Feb 05, 2010 at 09:44:14PM +1100, Dave Chinner wrote: > > The dependencies are always regenerated due to the default target > > requiring the depend target. Rebuilding the dependencies is the only > > way to catch changes between builds and so ensure the correct files > > are rebuilt. > > > > The install target is building the dependencies because it has a > > dependency on the default target. > > Well, it's building the dependecies twice - once by invoking the > default target, but they are also rebuilt again when the actuall > install rules are called. The latter is pretty clearly superflous. Ah, there's a double depenency chain. The top level make file has: install: default Which causes "make install" to run the top level default target, which runs the default target in all the target subdirs. Then, in each subdir, the makefile has: install: default Which when then install target is actually run, does another dependency check because it's got a local dependency on the depend target via the default target. Replace the previous patch with the one below and try again. Now the "make install" will rebuild targets out of the local dependencies rather than a separate run of the top level default target (i.e. only traverse directories once). Cheers, Dave. -- Dave Chinner david@fromorbit.com xfsprogs: clean up make install build V2 The install targets did not get the silent treatment like the normal build targets. Shut them up. Also, remove the top level install target dependency on the default target. Each sub-directory already defines the correct dependencies for the install targets and so all the rebuilds can be done in one traversal of the subdirectories via the install rules. Signed-off-by: Dave Chinner --- Makefile | 13 ++++++++----- include/buildrules | 4 ---- man/Makefile | 4 ++-- mdrestore/Makefile | 2 +- 4 files changed, 11 insertions(+), 12 deletions(-) diff --git a/Makefile b/Makefile index 62c4258..1905261 100644 --- a/Makefile +++ b/Makefile @@ -98,22 +98,25 @@ include/platform_defs.h: include/builddefs $(MAKE) $(MAKEOPTS) $(AM_MAKEFLAGS) include/builddefs; \ fi -install: default $(addsuffix -install,$(SUBDIRS)) +install: $(addsuffix -install,$(SUBDIRS)) $(INSTALL) -m 755 -d $(PKG_DOC_DIR) $(INSTALL) -m 644 README $(PKG_DOC_DIR) -install-dev: default $(addsuffix -install-dev,$(SUBDIRS)) +install-dev: $(addsuffix -install-dev,$(SUBDIRS)) install-qa: install $(addsuffix -install-qa,$(SUBDIRS)) %-install: - $(MAKE) -C $* install + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install %-install-dev: - $(MAKE) -C $* install-dev + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install-dev %-install-qa: - $(MAKE) -C $* install-qa + @echo "Installing $@" + $(Q)$(MAKE) $(MAKEOPTS) -C $* install-qa distclean: clean $(Q)rm -f $(DISTDIRT) diff --git a/include/buildrules b/include/buildrules index 1695e23..beb469b 100644 --- a/include/buildrules +++ b/include/buildrules @@ -101,7 +101,3 @@ ltdepend: $(CFILES) $(HFILES) depend: $(CFILES) $(HFILES) @echo " [DEP]" $(Q)$(MAKEDEP) $(CFILES) > .dep - - -# $(Q)$(MAKEDEP) $(CFILES) | $(SED) -e 's,^\([^:]*\)\.o,\1,' > .dep - diff --git a/man/Makefile b/man/Makefile index 2b5e89c..863284c 100644 --- a/man/Makefile +++ b/man/Makefile @@ -14,9 +14,9 @@ install : $(addsuffix -install,$(SUBDIRS)) install-dev : $(addsuffix -install-dev,$(SUBDIRS)) %-install: - $(MAKE) -C $* install + $(Q)$(MAKE) $(MAKEOPTS) -C $* install %-install-dev: - $(MAKE) -C $* install-dev + $(Q)$(MAKE) $(MAKEOPTS) -C $* install-dev include $(BUILDRULES) diff --git a/mdrestore/Makefile b/mdrestore/Makefile index fd35d80..ca2d1a0 100644 --- a/mdrestore/Makefile +++ b/mdrestore/Makefile @@ -16,7 +16,7 @@ default: depend $(LTCOMMAND) include $(BUILDRULES) -install: +install: default $(INSTALL) -m 755 -d $(PKG_SBIN_DIR) $(LTINSTALL) -m 755 $(LTCOMMAND) $(PKG_SBIN_DIR) install-dev: From sandeen@sandeen.net Fri Feb 5 10:58:37 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_75,J_CHICKENPOX_93 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15GwbcF149741 for ; Fri, 5 Feb 2010 10:58:37 -0600 X-ASG-Debug-ID: 1265389186-651a00b70000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 84F381B362F for ; Fri, 5 Feb 2010 08:59:46 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id uMLCSLTs42ANFRsO for ; Fri, 05 Feb 2010 08:59:46 -0800 (PST) Received: from Liberator.local (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 1A1DB124CF18; Fri, 5 Feb 2010 10:59:46 -0600 (CST) Message-ID: <4B6C4E81.6060201@sandeen.net> Date: Fri, 05 Feb 2010 10:59:45 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss CC: Theodore Tso X-ASG-Orig-Subj: [PATCH] xfstests: fix up fs_perms test used by 126 Subject: [PATCH] xfstests: fix up fs_perms test used by 126 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265389187 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21735 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Test 126 was failing intermittently for Ted & I; it seems that this is because we were passing an unterminated string to fopen for the mode; I'm not certain why this made it fail, but it's pretty clearly not a good thing to do, and fixing it fixes the test. Rather than passing around characters, do things string-wise, since that is what is ultimately used in fopen(). Also make it at least possible to pass in the 2-character modes fopen can take (r+, w+, etc), I suppose testcases could be added for this later. Reported-by: Theodore Tso Signed-off-by: Eric Sandeen --- diff --git a/src/fs_perms.c b/src/fs_perms.c index 2c5e3fa..f34c4f4 100644 --- a/src/fs_perms.c +++ b/src/fs_perms.c @@ -42,7 +42,7 @@ int testsetup(mode_t mode, int cuserId, int cgroupId); int testfperm(int userId, int groupId, char* fperm); int main( int argc, char *argv[]) { - char fperm[1]; + char fperm[3]; int result, exresult=0, cuserId=0, cgroupId=0, userId=0, groupId=0; mode_t mode; @@ -53,7 +53,8 @@ int main( int argc, char *argv[]) { cgroupId = atoi(argv[3]); userId = atoi(argv[4]); groupId = atoi(argv[5]); - fperm[0] = *argv[6]; + strncpy(fperm, argv[6], 3); + fperm[2] = '\0'; exresult = atoi(argv[7]); break; default: @@ -64,7 +65,7 @@ int main( int argc, char *argv[]) { testsetup(mode,cuserId,cgroupId); result=testfperm(userId,groupId,fperm); system("rm test.file"); - printf("%c a %03o file owned by (%d/%d) as user/group(%d/%d) ",fperm[0],mode,cuserId,cgroupId,userId,groupId); + printf("%s a %03o file owned by (%d/%d) as user/group(%d/%d) ",fperm,mode,cuserId,cgroupId,userId,groupId); if (result == exresult) { printf("PASS\n"); exit(0); @@ -102,8 +103,7 @@ int testfperm(int userId, int groupId, char* fperm) { return(-1); } - switch(tolower(fperm[0])) { - case 'x': + if (!strcmp("x", fperm)) { PID = fork(); if (PID == 0) { execlp("./test.file","test.file",NULL); @@ -114,8 +114,7 @@ int testfperm(int userId, int groupId, char* fperm) { seteuid(0); setegid(0); return(nuthertmpi); - - default: + } else { if((testfile=fopen("test.file",fperm))){ fclose(testfile); seteuid(0); From aelder@sgi.com Fri Feb 5 13:05:21 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15J5KXS155937 for ; Fri, 5 Feb 2010 13:05:20 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay2.corp.sgi.com (Postfix) with ESMTP id 2A0B93040DB; Fri, 5 Feb 2010 11:06:28 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 13:06:21 -0600 Subject: Re: [PATCH 01/10] xfs: Make inode reclaim states explicit From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265153104-29680-2-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-2-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 13:06:20 -0600 Message-ID: <1265396780.2714.23.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Feb 2010 19:06:21.0157 (UTC) FILETIME=[4DA6E950:01CAA696] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > A.K.A.: don't rely on xfs_iflush() return value in reclaim > > We have gradually been moving checks out of the reclaim code because > they are duplicated in xfs_iflush(). We've had a history of problems > in this area, and many of them stem from the overloading of the > return values from xfs_iflush() and interaction with inode flush > locking to determine if the inode is safe to reclaim. > > With the desire to move to delayed write flushing of inodes and > non-blocking inode tree reclaim walks, the overloading of the > return value of xfs_iflush makes it very difficult to determine > the correct thing to do next. > > This patch explicitly re-adds the checks to the inode reclaim code, > removing the reliance on the return value of xfs_iflush() to > determine what to do next. It also means that we can clearly > document all the inode states that reclaim must handle and hence > we can easily see that we handled all the necessary cases. > > This also removes the need for the xfs_inode_clean() check in > xfs_iflush() as all callers now check this first (safely). I have a few comments, below. One is a real bug. At this point I'm trusting that your enumeration of all the possible inode states is correct. Other than what I indicate below, this change looks good. > Signed-off-by: Dave Chinner > Reviewed-by: Christoph Hellwig > --- > fs/xfs/linux-2.6/xfs_sync.c | 80 ++++++++++++++++++++++++++++++++---------- > fs/xfs/xfs_inode.c | 11 +----- > fs/xfs/xfs_inode.h | 1 + > 3 files changed, 63 insertions(+), 29 deletions(-) > > diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c > index c9b863e..98b8937 100644 > --- a/fs/xfs/linux-2.6/xfs_sync.c > +++ b/fs/xfs/linux-2.6/xfs_sync.c > @@ -706,12 +706,43 @@ __xfs_inode_clear_reclaim_tag( > XFS_INO_TO_AGINO(mp, ip->i_ino), XFS_ICI_RECLAIM_TAG); > } > > +/* > + * Inodes in different states need to be treated differently, and the return > + * value of xfs_iflush is not sufficient to get this right. The following table > + * lists the inode states and the reclaim actions necessary for non-blocking > + * reclaim: > + * > + * > + * inode state iflush ret required action > + * --------------- ---------- --------------- > + * bad - reclaim > + * shutdown EIO unpin and reclaim > + * clean, unpinned 0 reclaim > + * stale, unpinned 0 reclaim > + * clean, pinned(*) 0 unpin and reclaim > + * stale, pinned 0 unpin and reclaim > + * dirty, async 0 block on flush lock, reclaim > + * dirty, sync flush 0 block on flush lock, reclaim > + * > + * (*) dgc: I don't think the clean, pinned state is possible but it gets > + * handled anyway given the order of checks implemented. > + * > + * Hence the order of actions after gaining the locks should be: > + * bad => reclaim > + * shutdown => unpin and reclaim > + * pinned => unpin > + * stale => reclaim > + * clean => reclaim > + * dirty => flush, wait and reclaim > + */ > STATIC int > xfs_reclaim_inode( > struct xfs_inode *ip, > struct xfs_perag *pag, > int sync_mode) > { > + int error; > + > /* > * The radix tree lock here protects a thread in xfs_iget from racing > * with us starting reclaim on the inode. Once we have the > @@ -729,30 +760,41 @@ xfs_reclaim_inode( > spin_unlock(&ip->i_flags_lock); > write_unlock(&pag->pag_ici_lock); > > - /* > - * If the inode is still dirty, then flush it out. If the inode > - * is not in the AIL, then it will be OK to flush it delwri as > - * long as xfs_iflush() does not keep any references to the inode. > - * We leave that decision up to xfs_iflush() since it has the > - * knowledge of whether it's OK to simply do a delwri flush of > - * the inode or whether we need to wait until the inode is > - * pulled from the AIL. > - * We get the flush lock regardless, though, just to make sure > - * we don't free it while it is being flushed. > - */ > xfs_ilock(ip, XFS_ILOCK_EXCL); > xfs_iflock(ip); > > - /* > - * In the case of a forced shutdown we rely on xfs_iflush() to > - * wait for the inode to be unpinned before returning an error. > - */ > - if (!is_bad_inode(VFS_I(ip)) && xfs_iflush(ip, sync_mode) == 0) { > - /* synchronize with xfs_iflush_done */ > - xfs_iflock(ip); > - xfs_ifunlock(ip); > + if (is_bad_inode(VFS_I(ip))) > + goto reclaim; It looks to me like your code adds a call to xfs_ifunlock(ip) in the bad inode case that was not there before. My guess is that your way is correct, but I'd like to know whether you agree this differs. Is this intentional? Regardless, is the new xfs_ifunlock() call correct? > + if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { > + xfs_iunpin_wait(ip); > + goto reclaim; > + } > + if (xfs_ipincount(ip)) > + xfs_iunpin_wait(ip); I notice you avoid an unneeded xfs_iunpin_wait() call in the case where it's not pinned already. (Still there in xfs_iflush() when *not* non-blocking.) Effect is the same though, but a minor improvement. > + if (xfs_iflags_test(ip, XFS_ISTALE)) > + goto reclaim; > + if (xfs_inode_clean(ip)) > + goto reclaim; > + > + /* Now we have an inode that needs flushing */ > + error = xfs_iflush(ip, sync_mode); > + if (!error) { > + switch(sync_mode) { > + case XFS_IFLUSH_DELWRI_ELSE_ASYNC: > + case XFS_IFLUSH_DELWRI: > + case XFS_IFLUSH_ASYNC: > + case XFS_IFLUSH_DELWRI_ELSE_SYNC: > + case XFS_IFLUSH_SYNC: > + /* IO issued, synchronise with IO completion */ > + xfs_iflock(ip); You must not have run this with DEBUG enabled. You need a "break;" here. > + default: > + ASSERT(0); > + break; > + } > } > > +reclaim: > + xfs_ifunlock(ip); > xfs_iunlock(ip, XFS_ILOCK_EXCL); > xfs_ireclaim(ip); > return 0; > diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c > index d0d1b5a..8d0666d 100644 > --- a/fs/xfs/xfs_inode.c > +++ b/fs/xfs/xfs_inode.c > @@ -2493,7 +2493,7 @@ __xfs_iunpin_wait( > wait_event(ip->i_ipin_wait, (atomic_read(&ip->i_pincount) == 0)); > } > > -static inline void > +void > xfs_iunpin_wait( > xfs_inode_t *ip) > { > @@ -2849,15 +2849,6 @@ xfs_iflush( > mp = ip->i_mount; > > /* > - * If the inode isn't dirty, then just release the inode flush lock and > - * do nothing. > - */ > - if (xfs_inode_clean(ip)) { > - xfs_ifunlock(ip); > - return 0; > - } > - > - /* > * We can't flush the inode until it is unpinned, so wait for it if we > * are allowed to block. We know noone new can pin it, because we are > * holding the inode lock shared and you need to hold it exclusively to > diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h > index ec1f28c..8b618ea 100644 > --- a/fs/xfs/xfs_inode.h > +++ b/fs/xfs/xfs_inode.h > @@ -483,6 +483,7 @@ int xfs_iunlink(struct xfs_trans *, xfs_inode_t *); > void xfs_iext_realloc(xfs_inode_t *, int, int); > void xfs_ipin(xfs_inode_t *); > void xfs_iunpin(xfs_inode_t *); > +void xfs_iunpin_wait(xfs_inode_t *); > int xfs_iflush(xfs_inode_t *, uint); > void xfs_ichgtime(xfs_inode_t *, int); > void xfs_lock_inodes(xfs_inode_t **, int, uint); From sandeen@sandeen.net Fri Feb 5 15:13:48 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15LDlAT161896 for ; Fri, 5 Feb 2010 15:13:47 -0600 X-ASG-Debug-ID: 1265404497-133402180000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 870891C8D607 for ; Fri, 5 Feb 2010 13:14:57 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id 0VBlrhgJjtOsyToy for ; Fri, 05 Feb 2010 13:14:57 -0800 (PST) Received: from Liberator.local (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 50679124CF07; Fri, 5 Feb 2010 15:14:57 -0600 (CST) Message-ID: <4B6C8A50.6090108@sandeen.net> Date: Fri, 05 Feb 2010 15:14:56 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Dave Chinner CC: xfs mailing list X-ASG-Orig-Subj: Re: xfsprogs: to -DDEBUG or not to -DDEBUG? Subject: Re: xfsprogs: to -DDEBUG or not to -DDEBUG? References: <4B6AFB83.1070309@sandeen.net> <20100205023134.GA11483@discord.disaster> In-Reply-To: <20100205023134.GA11483@discord.disaster> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265404498 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21750 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Dave Chinner wrote: > On Thu, Feb 04, 2010 at 10:53:23AM -0600, Eric Sandeen wrote: >> Fedora, long ago, disabled debug in the xfsprogs specfile: >> >> http://cvs.fedoraproject.org/viewvc/F-12/xfsprogs/xfsprogs.spec?r1=1.5&r2=1.6 >> >> * Wed Apr 20 2005 Dave Jones >> - Disable debug. (#151438) >> >> per this bug: https://bugzilla.redhat.com/show_bug.cgi?id=151438 >> >> referencing this email thread: >> >> http://oss.sgi.com/archives/linux-xfs/2005-03/msg00038.html >> which no longer exists (grrrr) but is probably now here: >> >> http://oss.sgi.com/archives/xfs/2005-03/msg00416.html >> >> Fedora still builds with -DNDEBUG, but the upstream tarball has >> it on by default. I have seen several occasions where other >> distros had failing xfs_repairs which were "fixed" by disabling >> debug. I'd like to make a decision on whether DEBUG should be >> on or off by default for upstream releases. Any thoughts? > > Personally I'd prefer the repair process to stop if it comes across > inconsistencies it can't handle. Ignoring them is as likely to "fix" > the crash as it is to corrupt the filesystem more. > > On top of that we get bug reports when those problems are hit so > we can look into what caused the problem. We don't really have a > repair test suite that covers all the possible corruptions that > can occur, so we are kind of reliant on users reporting conditions > we've never had to handle before so we can fix them... > > Cheers, > > Dave. Hrm. Well, that makes sense. OTOH, Debian turns it off too: $ grep DEBUG debian/rules options = export DEBUG=-DNDEBUG DISTRIBUTION=debian \ and for some reason when I take this out of the Fedora specfile, it fails to build because: xfs_bmap.c:23: warning: 'xfs_bmap_check_leaf_extents' used but never defined and indeed there is a forward declaration: #ifdef DEBUG STATIC void xfs_bmap_check_leaf_extents(xfs_btree_cur_t *cur, xfs_inode_t *ip, int whichfork); #endif but no definition that I can find .... Something must be odd w/ the way this flag works though because a simple "make" in the git tree, w/o any -DNDEBUG, works fine for me ... Meanwhile I'm leaving DEBUG=-DNDEBUG on in Fedora just for now :) -Eric From aelder@sgi.com Fri Feb 5 15:41:09 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15Lf9sP164056 for ; Fri, 5 Feb 2010 15:41:09 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay1.corp.sgi.com (Postfix) with ESMTP id A4CEB8F80BF; Fri, 5 Feb 2010 13:42:17 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 15:38:45 -0600 Subject: Re: [PATCH 02/10] xfs: Use delayed write for inodes rather than async V2 From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265153104-29680-3-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-3-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 15:38:44 -0600 Message-ID: <1265405924.2714.117.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Feb 2010 21:38:45.0497 (UTC) FILETIME=[981A7690:01CAA6AB] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > We currently do background inode flush asynchronously, resulting in > inodes being written in whatever order the background writeback > issues them. Not only that, there are also blocking and non-blocking > asynchronous inode flushes, depending on where the flush comes from. > > This patch completely removes asynchronous inode writeback. It > removes all the strange writeback modes and replaces them with > either a synchronous flush or a non-blocking delayed write flush. > That is, inode flushes will only issue IO directly if they are > synchronous, and background flushing may do nothing if the operation > would block (e.g. on a pinned inode or buffer lock). > > Delayed write flushes will now result in the inode buffer sitting in > the delwri queue of the buffer cache to be flushed by either an AIL > push or by the xfsbufd timing out the buffer. This will allow > accumulation of dirty inode buffers in memory and allow optimisation > of inode cluster writeback at the xfsbufd level where we have much > greater queue depths than the block layer elevators. We will also > get adjacent inode cluster buffer IO merging for free when a later > patch in the series allows sorting of the delayed write buffers > before dispatch. > > This effectively means that any inode that is written back by > background writeback will be seen as flush locked during AIL > pushing, and will result in the buffers being pushed from there. > This writeback path is currently non-optimal, but the next patch > in the series will fix that problem. > > A side effect of this delayed write mechanism is that background > inode reclaim will no longer directly flush inodes, nor can it wait > on the flush lock. The result is that inode reclaim must leave the > inode in the reclaimable state until it is clean. Hence attempts to > reclaim a dirty inode in the background will simply skip the inode > until it is clean and this allows other mechanisms (i.e. xfsbufd) to > do more optimal writeback of the dirty buffers. As a result, the > inode reclaim code has been rewritten so that it no longer relies on > the ambiguous return values of xfs_iflush() to determine whether it > is safe to reclaim an inode. > > Portions of this patch are derived from patches by Christoph > Hellwig. > > Version 2: > - cleanup reclaim code as suggested by Christoph > - log background reclaim inode flush errors > - just pass sync flags to xfs_iflush I now see that the missing "break;" in the previous patch doesn't matter as long as this patch is also applied... This change looks right to me, and it's pretty cool to see where you're headed with it. I wasn't quite following a few weeks back when you and Christoph were discussing this. I have a few comments on the comments, below. Maybe you could either confirm or correct what I say below. > Signed-off-by: Dave Chinner Reviewed-by: Alex Elder > --- > fs/xfs/linux-2.6/xfs_super.c | 4 +- > fs/xfs/linux-2.6/xfs_sync.c | 104 ++++++++++++++++++++++++++++++----------- > fs/xfs/xfs_inode.c | 75 ++---------------------------- > fs/xfs/xfs_inode.h | 10 ---- > fs/xfs/xfs_inode_item.c | 10 +++- > fs/xfs/xfs_mount.c | 13 +++++- > 6 files changed, 102 insertions(+), 114 deletions(-) > . . . > diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c > index 98b8937..a9f6d20 100644 > --- a/fs/xfs/linux-2.6/xfs_sync.c > +++ b/fs/xfs/linux-2.6/xfs_sync.c . . . > @@ -719,21 +720,42 @@ __xfs_inode_clear_reclaim_tag( Regardless of the state, if the inode has a flush underway we requeue. > * shutdown EIO unpin and reclaim > * clean, unpinned 0 reclaim > * stale, unpinned 0 reclaim > - * clean, pinned(*) 0 unpin and reclaim > - * stale, pinned 0 unpin and reclaim > - * dirty, async 0 block on flush lock, reclaim > - * dirty, sync flush 0 block on flush lock, reclaim > + * clean, pinned(*) 0 requeue > + * stale, pinned EAGAIN requeue Actually, if it's pinned (clean or stale) then we either unpin and reclaim (if synchronous) or requeue (otherwise), i.e.: * clean, pinned, async(*) N/A requeue * stale, pinned, async(*) N/A requeue * clean, pinned, sync flush 0 unpin and reclaim * stale, pinned, sync flush EAGAIN unpin and reclaim > + * dirty, delwri ok 0 requeue > + * dirty, delwri blocked EAGAIN requeue > + * dirty, sync flush 0 reclaim > * > * (*) dgc: I don't think the clean, pinned state is possible but it gets > * handled anyway given the order of checks implemented. > * > + * As can be seen from the table, the return value of xfs_iflush() is not > + * sufficient to correctly decide the reclaim action here. The checks in > + * xfs_iflush() might look like duplicates, but they are not. > + * > + * Also, because we get the flush lock first, we know that any inode that has > + * been flushed delwri has had the flush completed by the time we check that > + * the inode is clean. The clean inode check needs to be done before flushing > + * the inode delwri otherwise we would loop forever requeuing clean inodes as > + * we cannot tell apart a successful delwri flush and a clean inode from the > + * return value of xfs_iflush(). > + * > + * Note that because the inode is flushed delayed write by background > + * writeback, the flush lock may already be held here and waiting on it can > + * result in very long latencies. Hence for sync reclaims, where we wait on the > + * flush lock, the caller should push out delayed write inodes first before > + * trying to reclaim them to minimise the amount of time spent waiting. For > + * background relaim, we just requeue the inode for the next pass. > + * > * Hence the order of actions after gaining the locks should be: > * bad => reclaim > * shutdown => unpin and reclaim > - * pinned => unpin > + * pinned, delwri => requeue > + * pinned, sync => unpin > * stale => reclaim > * clean => reclaim > - * dirty => flush, wait and reclaim > + * dirty, delwri => flush and requeue > + * dirty, sync => flush, wait and reclaim > */ > STATIC int > xfs_reclaim_inode( . . . From aelder@sgi.com Fri Feb 5 16:52:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15MqVYo167912 for ; Fri, 5 Feb 2010 16:52:31 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay3.corp.sgi.com (Postfix) with ESMTP id 67EE7AC016; Fri, 5 Feb 2010 14:53:39 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 16:51:29 -0600 Subject: Re: [PATCH 03/10] xfs: Don't issue buffer IO direct from AIL push V2 From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265153104-29680-4-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-4-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 16:51:29 -0600 Message-ID: <1265410289.2714.153.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Feb 2010 22:51:29.0978 (UTC) FILETIME=[C18969A0:01CAA6B5] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > All buffers logged into the AIL are marked as delayed write. > When the AIL needs to push the buffer out, it issues an async write of the > buffer. This means that IO patterns are dependent on the order of > buffers in the AIL. > > Instead of flushing the buffer, promote the buffer in the delayed > write list so that the next time the xfsbufd is run the buffer will > be flushed by the xfsbufd. Return the state to the xfsaild that the > buffer was promoted so that the xfsaild knows that it needs to cause > the xfsbufd to run to flush the buffers that were promoted. > > Using the xfsbufd for issuing the IO allows us to dispatch all > buffer IO from the one queue. This means that we can make much more > enlightened decisions on what order to flush buffers to disk as > we don't have multiple places issuing IO. Optimisations to xfsbufd > will be in a future patch. > > Version 2 > - kill XFS_ITEM_FLUSHING as it is now unused. Looks good. > Signed-off-by: Dave Chinner > Reviewed-by: Christoph Hellwig Reviewed-by: Alex Elder > --- > fs/xfs/linux-2.6/xfs_buf.c | 29 ++++++++++++ > fs/xfs/linux-2.6/xfs_buf.h | 2 + > fs/xfs/linux-2.6/xfs_trace.h | 1 + > fs/xfs/quota/xfs_dquot_item.c | 85 +++++------------------------------ > fs/xfs/quota/xfs_dquot_item.h | 4 -- > fs/xfs/xfs_buf_item.c | 64 +++++++++++++++------------ > fs/xfs/xfs_inode_item.c | 98 ++++++---------------------------------- > fs/xfs/xfs_inode_item.h | 6 --- > fs/xfs/xfs_trans.h | 3 +- > fs/xfs/xfs_trans_ail.c | 13 +++--- > 10 files changed, 102 insertions(+), 203 deletions(-) > > diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c > index 44e20e5..b306265 100644 > --- a/fs/xfs/linux-2.6/xfs_buf.c > +++ b/fs/xfs/linux-2.6/xfs_buf.c > @@ -1778,6 +1778,35 @@ xfs_buf_delwri_dequeue( > trace_xfs_buf_delwri_dequeue(bp, _RET_IP_); > } > > +/* > + * If a delwri buffer needs to be pushed before it has aged out, then promote > + * it to the head of the delwri queue so that it will be flushed on the next > + * xfsbufd run. We do this by resetting the queuetime of the buffer to be older > + * than the age currently needed to flush the buffer. Hence the next time the > + * xfsbufd sees it is guaranteed to be considered old enough to flush. > + */ > +void > +xfs_buf_delwri_promote( > + struct xfs_buf *bp) > +{ > + struct xfs_buftarg *btp = bp->b_target; > + long age = xfs_buf_age_centisecs * msecs_to_jiffies(10) + 1; > + > + ASSERT(bp->b_flags & XBF_DELWRI); > + ASSERT(bp->b_flags & _XBF_DELWRI_Q); > + > + /* > + * Check the buffer age before locking the delayed write queue as we > + * don't need to promote buffers that are already past the flush age. > + */ > + if (bp->b_queuetime < jiffies - age) > + return; > + bp->b_queuetime = jiffies - age; > + spin_lock(&btp->bt_delwrite_lock); > + list_move(&bp->b_list, &btp->bt_delwrite_queue); > + spin_unlock(&btp->bt_delwrite_lock); > +} > + > STATIC void > xfs_buf_runall_queues( > struct workqueue_struct *queue) > diff --git a/fs/xfs/linux-2.6/xfs_buf.h b/fs/xfs/linux-2.6/xfs_buf.h > index ea8c198..be45e8c 100644 > --- a/fs/xfs/linux-2.6/xfs_buf.h > +++ b/fs/xfs/linux-2.6/xfs_buf.h > @@ -266,6 +266,7 @@ extern int xfs_buf_ispin(xfs_buf_t *); > > /* Delayed Write Buffer Routines */ > extern void xfs_buf_delwri_dequeue(xfs_buf_t *); > +extern void xfs_buf_delwri_promote(xfs_buf_t *); > > /* Buffer Daemon Setup Routines */ > extern int xfs_buf_init(void); > @@ -395,6 +396,7 @@ extern void xfs_free_buftarg(struct xfs_mount *, struct xfs_buftarg *); > extern void xfs_wait_buftarg(xfs_buftarg_t *); > extern int xfs_setsize_buftarg(xfs_buftarg_t *, unsigned int, unsigned int); > extern int xfs_flush_buftarg(xfs_buftarg_t *, int); > + > #ifdef CONFIG_KDB_MODULES > extern struct list_head *xfs_get_buftarg_list(void); > #endif > diff --git a/fs/xfs/linux-2.6/xfs_trace.h b/fs/xfs/linux-2.6/xfs_trace.h > index 1bb09e7..a4574dc 100644 > --- a/fs/xfs/linux-2.6/xfs_trace.h > +++ b/fs/xfs/linux-2.6/xfs_trace.h > @@ -483,6 +483,7 @@ DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock); > DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock_stale); > DEFINE_BUF_ITEM_EVENT(xfs_buf_item_committed); > DEFINE_BUF_ITEM_EVENT(xfs_buf_item_push); > +DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pushbuf); > DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf); > DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf_recur); > DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb); > diff --git a/fs/xfs/quota/xfs_dquot_item.c b/fs/xfs/quota/xfs_dquot_item.c > index 1b56437..dda0fb0 100644 > --- a/fs/xfs/quota/xfs_dquot_item.c > +++ b/fs/xfs/quota/xfs_dquot_item.c > @@ -212,66 +212,31 @@ xfs_qm_dquot_logitem_pushbuf( > xfs_dquot_t *dqp; > xfs_mount_t *mp; > xfs_buf_t *bp; > - uint dopush; > > dqp = qip->qli_dquot; > ASSERT(XFS_DQ_IS_LOCKED(dqp)); > > /* > - * The qli_pushbuf_flag keeps others from > - * trying to duplicate our effort. > - */ > - ASSERT(qip->qli_pushbuf_flag != 0); > - ASSERT(qip->qli_push_owner == current_pid()); > - > - /* > * If flushlock isn't locked anymore, chances are that the > * inode flush completed and the inode was taken off the AIL. > * So, just get out. > */ > if (completion_done(&dqp->q_flush) || > ((qip->qli_item.li_flags & XFS_LI_IN_AIL) == 0)) { > - qip->qli_pushbuf_flag = 0; > xfs_dqunlock(dqp); > return; > } > mp = dqp->q_mount; > bp = xfs_incore(mp->m_ddev_targp, qip->qli_format.qlf_blkno, > XFS_QI_DQCHUNKLEN(mp), XBF_TRYLOCK); > - if (bp != NULL) { > - if (XFS_BUF_ISDELAYWRITE(bp)) { > - dopush = ((qip->qli_item.li_flags & XFS_LI_IN_AIL) && > - !completion_done(&dqp->q_flush)); > - qip->qli_pushbuf_flag = 0; > - xfs_dqunlock(dqp); > - > - if (XFS_BUF_ISPINNED(bp)) > - xfs_log_force(mp, 0); > - > - if (dopush) { > - int error; > -#ifdef XFSRACEDEBUG > - delay_for_intr(); > - delay(300); > -#endif > - error = xfs_bawrite(mp, bp); > - if (error) > - xfs_fs_cmn_err(CE_WARN, mp, > - "xfs_qm_dquot_logitem_pushbuf: pushbuf error %d on qip %p, bp %p", > - error, qip, bp); > - } else { > - xfs_buf_relse(bp); > - } > - } else { > - qip->qli_pushbuf_flag = 0; > - xfs_dqunlock(dqp); > - xfs_buf_relse(bp); > - } > + xfs_dqunlock(dqp); > + if (!bp) > return; > - } > + if (XFS_BUF_ISDELAYWRITE(bp)) > + xfs_buf_delwri_promote(bp); > + xfs_buf_relse(bp); > + return; > > - qip->qli_pushbuf_flag = 0; > - xfs_dqunlock(dqp); > } > > /* > @@ -289,50 +254,24 @@ xfs_qm_dquot_logitem_trylock( > xfs_dq_logitem_t *qip) > { > xfs_dquot_t *dqp; > - uint retval; > > dqp = qip->qli_dquot; > if (atomic_read(&dqp->q_pincount) > 0) > - return (XFS_ITEM_PINNED); > + return XFS_ITEM_PINNED; > > if (! xfs_qm_dqlock_nowait(dqp)) > - return (XFS_ITEM_LOCKED); > + return XFS_ITEM_LOCKED; > > - retval = XFS_ITEM_SUCCESS; > if (!xfs_dqflock_nowait(dqp)) { > /* > - * The dquot is already being flushed. It may have been > - * flushed delayed write, however, and we don't want to > - * get stuck waiting for that to complete. So, we want to check > - * to see if we can lock the dquot's buffer without sleeping. > - * If we can and it is marked for delayed write, then we > - * hold it and send it out from the push routine. We don't > - * want to do that now since we might sleep in the device > - * strategy routine. We also don't want to grab the buffer lock > - * here because we'd like not to call into the buffer cache > - * while holding the AIL lock. > - * Make sure to only return PUSHBUF if we set pushbuf_flag > - * ourselves. If someone else is doing it then we don't > - * want to go to the push routine and duplicate their efforts. > + * dquot has already been flushed to the backing buffer, > + * leave it locked, pushbuf routine will unlock it. > */ > - if (qip->qli_pushbuf_flag == 0) { > - qip->qli_pushbuf_flag = 1; > - ASSERT(qip->qli_format.qlf_blkno == dqp->q_blkno); > -#ifdef DEBUG > - qip->qli_push_owner = current_pid(); > -#endif > - /* > - * The dquot is left locked. > - */ > - retval = XFS_ITEM_PUSHBUF; > - } else { > - retval = XFS_ITEM_FLUSHING; > - xfs_dqunlock_nonotify(dqp); > - } > + return XFS_ITEM_PUSHBUF; > } > > ASSERT(qip->qli_item.li_flags & XFS_LI_IN_AIL); > - return (retval); > + return XFS_ITEM_SUCCESS; > } > > > diff --git a/fs/xfs/quota/xfs_dquot_item.h b/fs/xfs/quota/xfs_dquot_item.h > index 5a63253..5acae2a 100644 > --- a/fs/xfs/quota/xfs_dquot_item.h > +++ b/fs/xfs/quota/xfs_dquot_item.h > @@ -27,10 +27,6 @@ typedef struct xfs_dq_logitem { > xfs_log_item_t qli_item; /* common portion */ > struct xfs_dquot *qli_dquot; /* dquot ptr */ > xfs_lsn_t qli_flush_lsn; /* lsn at last flush */ > - unsigned short qli_pushbuf_flag; /* 1 bit used in push_ail */ > -#ifdef DEBUG > - uint64_t qli_push_owner; > -#endif > xfs_dq_logformat_t qli_format; /* logged structure */ > } xfs_dq_logitem_t; > > diff --git a/fs/xfs/xfs_buf_item.c b/fs/xfs/xfs_buf_item.c > index e0a1158..f3c49e6 100644 > --- a/fs/xfs/xfs_buf_item.c > +++ b/fs/xfs/xfs_buf_item.c > @@ -467,8 +467,10 @@ xfs_buf_item_unpin_remove( > /* > * This is called to attempt to lock the buffer associated with this > * buf log item. Don't sleep on the buffer lock. If we can't get > - * the lock right away, return 0. If we can get the lock, pull the > - * buffer from the free list, mark it busy, and return 1. > + * the lock right away, return 0. If we can get the lock, take a > + * reference to the buffer. If this is a delayed write buffer that > + * needs AIL help to be written back, invoke the pushbuf routine > + * rather than the normal success path. > */ > STATIC uint > xfs_buf_item_trylock( > @@ -477,24 +479,18 @@ xfs_buf_item_trylock( > xfs_buf_t *bp; > > bp = bip->bli_buf; > - > - if (XFS_BUF_ISPINNED(bp)) { > + if (XFS_BUF_ISPINNED(bp)) > return XFS_ITEM_PINNED; > - } > - > - if (!XFS_BUF_CPSEMA(bp)) { > + if (!XFS_BUF_CPSEMA(bp)) > return XFS_ITEM_LOCKED; > - } > > - /* > - * Remove the buffer from the free list. Only do this > - * if it's on the free list. Private buffers like the > - * superblock buffer are not. > - */ > + /* take a reference to the buffer. */ > XFS_BUF_HOLD(bp); > > ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); > trace_xfs_buf_item_trylock(bip); > + if (XFS_BUF_ISDELAYWRITE(bp)) > + return XFS_ITEM_PUSHBUF; > return XFS_ITEM_SUCCESS; > } > > @@ -626,11 +622,9 @@ xfs_buf_item_committed( > } > > /* > - * This is called to asynchronously write the buffer associated with this > - * buf log item out to disk. The buffer will already have been locked by > - * a successful call to xfs_buf_item_trylock(). If the buffer still has > - * B_DELWRI set, then get it going out to disk with a call to bawrite(). > - * If not, then just release the buffer. > + * The buffer is locked, but is not a delayed write buffer. This happens > + * if we race with IO completion and hence we don't want to try to write it > + * again. Just release the buffer. > */ > STATIC void > xfs_buf_item_push( > @@ -642,17 +636,29 @@ xfs_buf_item_push( > trace_xfs_buf_item_push(bip); > > bp = bip->bli_buf; > + ASSERT(!XFS_BUF_ISDELAYWRITE(bp)); > + xfs_buf_relse(bp); > +} > > - if (XFS_BUF_ISDELAYWRITE(bp)) { > - int error; > - error = xfs_bawrite(bip->bli_item.li_mountp, bp); > - if (error) > - xfs_fs_cmn_err(CE_WARN, bip->bli_item.li_mountp, > - "xfs_buf_item_push: pushbuf error %d on bip %p, bp %p", > - error, bip, bp); > - } else { > - xfs_buf_relse(bp); > - } > +/* > + * The buffer is locked and is a delayed write buffer. Promote the buffer > + * in the delayed write queue as the caller knows that they must invoke > + * the xfsbufd to get this buffer written. We have to unlock the buffer > + * to allow the xfsbufd to write it, too. > + */ > +STATIC void > +xfs_buf_item_pushbuf( > + xfs_buf_log_item_t *bip) > +{ > + xfs_buf_t *bp; > + > + ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); > + trace_xfs_buf_item_pushbuf(bip); > + > + bp = bip->bli_buf; > + ASSERT(XFS_BUF_ISDELAYWRITE(bp)); > + xfs_buf_delwri_promote(bp); > + xfs_buf_relse(bp); > } > > /* ARGSUSED */ > @@ -677,7 +683,7 @@ static struct xfs_item_ops xfs_buf_item_ops = { > .iop_committed = (xfs_lsn_t(*)(xfs_log_item_t*, xfs_lsn_t)) > xfs_buf_item_committed, > .iop_push = (void(*)(xfs_log_item_t*))xfs_buf_item_push, > - .iop_pushbuf = NULL, > + .iop_pushbuf = (void(*)(xfs_log_item_t*))xfs_buf_item_pushbuf, > .iop_committing = (void(*)(xfs_log_item_t*, xfs_lsn_t)) > xfs_buf_item_committing > }; > diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c > index 207553e..d4dc063 100644 > --- a/fs/xfs/xfs_inode_item.c > +++ b/fs/xfs/xfs_inode_item.c > @@ -602,33 +602,20 @@ xfs_inode_item_trylock( > > if (!xfs_iflock_nowait(ip)) { > /* > - * If someone else isn't already trying to push the inode > - * buffer, we get to do it. > + * inode has already been flushed to the backing buffer, > + * leave it locked in shared mode, pushbuf routine will > + * unlock it. > */ > - if (iip->ili_pushbuf_flag == 0) { > - iip->ili_pushbuf_flag = 1; > -#ifdef DEBUG > - iip->ili_push_owner = current_pid(); > -#endif > - /* > - * Inode is left locked in shared mode. > - * Pushbuf routine gets to unlock it. > - */ > - return XFS_ITEM_PUSHBUF; > - } else { > - /* > - * We hold the AIL lock, so we must specify the > - * NONOTIFY flag so that we won't double trip. > - */ > - xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); > - return XFS_ITEM_FLUSHING; > - } > - /* NOTREACHED */ > + return XFS_ITEM_PUSHBUF; > } > > /* Stale items should force out the iclog */ > if (ip->i_flags & XFS_ISTALE) { > xfs_ifunlock(ip); > + /* > + * we hold the AIL lock - notify the unlock routine of this > + * so it doesn't try to get the lock again. > + */ > xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); > return XFS_ITEM_PINNED; > } > @@ -746,11 +733,8 @@ xfs_inode_item_committed( > * This gets called by xfs_trans_push_ail(), when IOP_TRYLOCK > * failed to get the inode flush lock but did get the inode locked SHARED. > * Here we're trying to see if the inode buffer is incore, and if so whether it's > - * marked delayed write. If that's the case, we'll initiate a bawrite on that > - * buffer to expedite the process. > - * > - * We aren't holding the AIL lock (or the flush lock) when this gets called, > - * so it is inherently race-y. > + * marked delayed write. If that's the case, we'll promote it and that will > + * allow the caller to write the buffer by triggering the xfsbufd to run. > */ > STATIC void > xfs_inode_item_pushbuf( > @@ -759,26 +743,16 @@ xfs_inode_item_pushbuf( > xfs_inode_t *ip; > xfs_mount_t *mp; > xfs_buf_t *bp; > - uint dopush; > > ip = iip->ili_inode; > - > ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED)); > > /* > - * The ili_pushbuf_flag keeps others from > - * trying to duplicate our effort. > - */ > - ASSERT(iip->ili_pushbuf_flag != 0); > - ASSERT(iip->ili_push_owner == current_pid()); > - > - /* > * If a flush is not in progress anymore, chances are that the > * inode was taken off the AIL. So, just get out. > */ > if (completion_done(&ip->i_flush) || > ((iip->ili_item.li_flags & XFS_LI_IN_AIL) == 0)) { > - iip->ili_pushbuf_flag = 0; > xfs_iunlock(ip, XFS_ILOCK_SHARED); > return; > } > @@ -787,53 +761,12 @@ xfs_inode_item_pushbuf( > bp = xfs_incore(mp->m_ddev_targp, iip->ili_format.ilf_blkno, > iip->ili_format.ilf_len, XBF_TRYLOCK); > > - if (bp != NULL) { > - if (XFS_BUF_ISDELAYWRITE(bp)) { > - /* > - * We were racing with iflush because we don't hold > - * the AIL lock or the flush lock. However, at this point, > - * we have the buffer, and we know that it's dirty. > - * So, it's possible that iflush raced with us, and > - * this item is already taken off the AIL. > - * If not, we can flush it async. > - */ > - dopush = ((iip->ili_item.li_flags & XFS_LI_IN_AIL) && > - !completion_done(&ip->i_flush)); > - iip->ili_pushbuf_flag = 0; > - xfs_iunlock(ip, XFS_ILOCK_SHARED); > - > - trace_xfs_inode_item_push(bp, _RET_IP_); > - > - if (XFS_BUF_ISPINNED(bp)) > - xfs_log_force(mp, 0); > - > - if (dopush) { > - int error; > - error = xfs_bawrite(mp, bp); > - if (error) > - xfs_fs_cmn_err(CE_WARN, mp, > - "xfs_inode_item_pushbuf: pushbuf error %d on iip %p, bp %p", > - error, iip, bp); > - } else { > - xfs_buf_relse(bp); > - } > - } else { > - iip->ili_pushbuf_flag = 0; > - xfs_iunlock(ip, XFS_ILOCK_SHARED); > - xfs_buf_relse(bp); > - } > - return; > - } > - /* > - * We have to be careful about resetting pushbuf flag too early (above). > - * Even though in theory we can do it as soon as we have the buflock, > - * we don't want others to be doing work needlessly. They'll come to > - * this function thinking that pushing the buffer is their > - * responsibility only to find that the buffer is still locked by > - * another doing the same thing > - */ > - iip->ili_pushbuf_flag = 0; > xfs_iunlock(ip, XFS_ILOCK_SHARED); > + if (!bp) > + return; > + if (XFS_BUF_ISDELAYWRITE(bp)) > + xfs_buf_delwri_promote(bp); > + xfs_buf_relse(bp); > return; > } > > @@ -937,7 +870,6 @@ xfs_inode_item_init( > /* > We have zeroed memory. No need ... > iip->ili_extents_buf = NULL; > - iip->ili_pushbuf_flag = 0; > */ > > iip->ili_format.ilf_type = XFS_LI_INODE; > diff --git a/fs/xfs/xfs_inode_item.h b/fs/xfs/xfs_inode_item.h > index cc8df1a..9a46795 100644 > --- a/fs/xfs/xfs_inode_item.h > +++ b/fs/xfs/xfs_inode_item.h > @@ -144,12 +144,6 @@ typedef struct xfs_inode_log_item { > data exts */ > struct xfs_bmbt_rec *ili_aextents_buf; /* array of logged > attr exts */ > - unsigned int ili_pushbuf_flag; /* one bit used in push_ail */ > - > -#ifdef DEBUG > - uint64_t ili_push_owner; /* one who sets pushbuf_flag > - above gets to push the buf */ > -#endif > #ifdef XFS_TRANS_DEBUG > int ili_root_size; > char *ili_orig_root; > diff --git a/fs/xfs/xfs_trans.h b/fs/xfs/xfs_trans.h > index ca64f33..c93e3a1 100644 > --- a/fs/xfs/xfs_trans.h > +++ b/fs/xfs/xfs_trans.h > @@ -861,8 +861,7 @@ typedef struct xfs_item_ops { > #define XFS_ITEM_SUCCESS 0 > #define XFS_ITEM_PINNED 1 > #define XFS_ITEM_LOCKED 2 > -#define XFS_ITEM_FLUSHING 3 > -#define XFS_ITEM_PUSHBUF 4 > +#define XFS_ITEM_PUSHBUF 3 > > /* > * This structure is used to maintain a list of block ranges that have been > diff --git a/fs/xfs/xfs_trans_ail.c b/fs/xfs/xfs_trans_ail.c > index d7b1af8..e799824 100644 > --- a/fs/xfs/xfs_trans_ail.c > +++ b/fs/xfs/xfs_trans_ail.c > @@ -253,6 +253,7 @@ xfsaild_push( > int flush_log, count, stuck; > xfs_mount_t *mp = ailp->xa_mount; > struct xfs_ail_cursor *cur = &ailp->xa_cursors; > + int push_xfsbufd = 0; > > spin_lock(&ailp->xa_lock); > xfs_trans_ail_cursor_init(ailp, cur); > @@ -308,6 +309,7 @@ xfsaild_push( > XFS_STATS_INC(xs_push_ail_pushbuf); > IOP_PUSHBUF(lip); > last_pushed_lsn = lsn; > + push_xfsbufd = 1; > break; > > case XFS_ITEM_PINNED: > @@ -322,12 +324,6 @@ xfsaild_push( > stuck++; > break; > > - case XFS_ITEM_FLUSHING: > - XFS_STATS_INC(xs_push_ail_flushing); > - last_pushed_lsn = lsn; > - stuck++; > - break; > - > default: > ASSERT(0); > break; > @@ -374,6 +370,11 @@ xfsaild_push( > xfs_log_force(mp, 0); > } > > + if (push_xfsbufd) { > + /* we've got delayed write buffers to flush */ > + wake_up_process(mp->m_ddev_targp->bt_task); > + } > + > if (!count) { > /* We're past our target or empty, so idle */ > last_pushed_lsn = 0; From sandeen@sandeen.net Fri Feb 5 16:58:46 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15MwjJn168246 for ; Fri, 5 Feb 2010 16:58:45 -0600 X-ASG-Debug-ID: 1265410794-2d2700350000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E84361382293 for ; Fri, 5 Feb 2010 14:59:54 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id wVYW4gmAk2NO1Ux7 for ; Fri, 05 Feb 2010 14:59:54 -0800 (PST) Received: from Liberator.local (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 2120F97FB47 for ; Fri, 5 Feb 2010 16:59:54 -0600 (CST) Message-ID: <4B6CA2E9.7020803@sandeen.net> Date: Fri, 05 Feb 2010 16:59:53 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs mailing list X-ASG-Orig-Subj: PATCH V2] more reserved blocks fixups Subject: PATCH V2] more reserved blocks fixups References: <4B60C8EE.5080700@sandeen.net> In-Reply-To: <4B60C8EE.5080700@sandeen.net> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265410795 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21757 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This mangles the reserved blocks counts a little more. 1) add a helper function for the default reserved count 2) add helper functions to save/restore counts on ro/rw 3) save/restore reserved blocks on freeze/thaw 4) disallow changing reserved count while readonly Signed-off-by: Eric Sandeen --- V2: changed field name to match Dave's changes diff --git a/fs/xfs/linux-2.6/xfs_ioctl.c b/fs/xfs/linux-2.6/xfs_ioctl.c index a034cf6..883929b 100644 --- a/fs/xfs/linux-2.6/xfs_ioctl.c +++ b/fs/xfs/linux-2.6/xfs_ioctl.c @@ -1431,6 +1431,9 @@ xfs_file_ioctl( if (!capable(CAP_SYS_ADMIN)) return -EPERM; + if (mp->m_flags & XFS_MOUNT_RDONLY) + return -XFS_ERROR(EROFS); + if (copy_from_user(&inout, arg, sizeof(inout))) return -XFS_ERROR(EFAULT); diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 8884f26..97c0f5a 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -1257,6 +1257,29 @@ xfs_fs_statfs( return 0; } +STATIC void +xfs_save_resvblks(struct xfs_mount *mp) +{ + __uint64_t resblks = 0; + + mp->m_resblks_save = mp->m_resblks; + xfs_reserve_blocks(mp, &resblks, NULL); +} + +STATIC void +xfs_restore_resvblks(struct xfs_mount *mp) +{ + __uint64_t resblks; + + if (mp->m_resblks_save) { + resblks = mp->m_resblks_save; + mp->m_resblks_save = 0; + } else + resblks = xfs_default_resblks(mp); + + xfs_reserve_blocks(mp, &resblks, NULL); +} + STATIC int xfs_fs_remount( struct super_block *sb, @@ -1319,8 +1342,6 @@ xfs_fs_remount( /* ro -> rw */ if ((mp->m_flags & XFS_MOUNT_RDONLY) && !(*flags & MS_RDONLY)) { - __uint64_t resblks; - mp->m_flags &= ~XFS_MOUNT_RDONLY; if (mp->m_flags & XFS_MOUNT_BARRIER) xfs_mountfs_check_barriers(mp); @@ -1343,15 +1364,7 @@ xfs_fs_remount( * Fill out the reserve pool if it is empty. Use the stashed * value if it is non-zero, otherwise go with the default. */ - if (mp->m_resblks_save) { - resblks = mp->m_resblks_save; - mp->m_resblks_save = 0; - } else { - resblks = mp->m_sb.sb_dblocks; - do_div(resblks, 20); - resblks = min_t(__uint64_t, resblks, 1024); - } - xfs_reserve_blocks(mp, &resblks, NULL); + xfs_restore_resvblks(mp); } /* rw -> ro */ @@ -1364,11 +1377,9 @@ xfs_fs_remount( * so that if we get remounted rw, we can return it to the same * size. */ - __uint64_t resblks = 0; xfs_quiesce_data(mp); - mp->m_resblks_save = mp->m_resblks; - xfs_reserve_blocks(mp, &resblks, NULL); + xfs_save_resvblks(mp); xfs_quiesce_attr(mp); mp->m_flags |= XFS_MOUNT_RDONLY; } @@ -1387,11 +1398,22 @@ xfs_fs_freeze( { struct xfs_mount *mp = XFS_M(sb); + xfs_save_resvblks(mp); xfs_quiesce_attr(mp); return -xfs_fs_log_dummy(mp); } STATIC int +xfs_fs_unfreeze( + struct super_block *sb) +{ + struct xfs_mount *mp = XFS_M(sb); + + xfs_restore_resvblks(mp); + return 0; +} + +STATIC int xfs_fs_show_options( struct seq_file *m, struct vfsmount *mnt) @@ -1613,6 +1635,7 @@ static const struct super_operations xfs_super_operations = { .put_super = xfs_fs_put_super, .sync_fs = xfs_fs_sync_fs, .freeze_fs = xfs_fs_freeze, + .unfreeze_fs = xfs_fs_unfreeze, .statfs = xfs_fs_statfs, .remount_fs = xfs_fs_remount, .show_options = xfs_fs_show_options, diff --git a/fs/xfs/xfs_mount.c b/fs/xfs/xfs_mount.c index eb403b4..48e1ee7 100644 --- a/fs/xfs/xfs_mount.c +++ b/fs/xfs/xfs_mount.c @@ -1008,6 +1008,22 @@ xfs_mount_reset_sbqflags( return xfs_trans_commit(tp, 0); } +__uint64_t +xfs_default_resblks(xfs_mount_t *mp) +{ + __uint64_t resblks; + + /* + * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. + * This may drive us straight to ENOSPC on mount, but that implies + * we were already there on the last unmount. Warn if this occurs. + */ + resblks = mp->m_sb.sb_dblocks; + do_div(resblks, 20); + resblks = min_t(__uint64_t, resblks, 1024); + return resblks; +} + /* * This function does the following on an initial mount of a file system: * - reads the superblock from disk and init the mount struct @@ -1318,18 +1334,14 @@ xfs_mountfs( * when at ENOSPC. This is needed for operations like create with * attr, unwritten extent conversion at ENOSPC, etc. Data allocations * are not allowed to use this reserved space. - * - * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. - * This may drive us straight to ENOSPC on mount, but that implies - * we were already there on the last unmount. Warn if this occurs. */ - resblks = mp->m_sb.sb_dblocks; - do_div(resblks, 20); - resblks = min_t(__uint64_t, resblks, 1024); - error = xfs_reserve_blocks(mp, &resblks, NULL); - if (error) - cmn_err(CE_WARN, "XFS: Unable to allocate reserve blocks. " - "Continuing without a reserve pool."); + if (!(mp->m_flags & XFS_MOUNT_RDONLY)) { + resblks = xfs_default_resblks(mp); + error = xfs_reserve_blocks(mp, &resblks, NULL); + if (error) + cmn_err(CE_WARN, "XFS: Unable to allocate reserve " + "blocks. Continuing without a reserve pool."); + } return 0; diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h index 996273d..c8f880d 100644 --- a/fs/xfs/xfs_mount.h +++ b/fs/xfs/xfs_mount.h @@ -429,6 +429,7 @@ typedef struct xfs_mod_sb { } xfs_mod_sb_t; extern int xfs_log_sbcount(xfs_mount_t *, uint); +extern __uint64_t xfs_default_resblks(xfs_mount_t *mp); extern int xfs_mountfs(xfs_mount_t *mp); extern void xfs_unmountfs(xfs_mount_t *); From aelder@sgi.com Fri Feb 5 17:52:34 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_66 autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15NqYld172299 for ; Fri, 5 Feb 2010 17:52:34 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay1.corp.sgi.com (Postfix) with ESMTP id F1CBD8F80C0; Fri, 5 Feb 2010 15:53:42 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 17:53:34 -0600 Subject: Re: [PATCH 04/10] xfs: Sort delayed write buffers before dispatch From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265153104-29680-5-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-5-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 17:53:34 -0600 Message-ID: <1265414014.2714.154.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: quoted-printable X-OriginalArrivalTime: 05 Feb 2010 23:53:34.0857 (UTC) FILETIME=[6DBCD790:01CAA6BE] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > Currently when the xfsbufd writes delayed write buffers, it pushes > them to disk in the order they come off the delayed write list. If > there are lots of buffers =D1=95pread widely over the disk, this results > in overwhelming the elevator sort queues in the block layer and we > end up losing the posibility of merging adjacent buffers to minimise > the number of IOs. >=20 > Use the new generic list_sort function to sort the delwri dispatch > queue before issue to ensure that the buffers are pushed in the most > friendly order possible to the lower layers. Looks good. > Signed-off-by: Dave Chinner > Reviewed-by: Christoph Hellwig Reviewed-by: Alex Elder > --- > fs/xfs/linux-2.6/xfs_buf.c | 87 ++++++++++++++++++++++++++++++--------= ------ > 1 files changed, 60 insertions(+), 27 deletions(-) >=20 > diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c > index b306265..4556a4c 100644 > --- a/fs/xfs/linux-2.6/xfs_buf.c > +++ b/fs/xfs/linux-2.6/xfs_buf.c > @@ -33,6 +33,7 @@ > #include > #include > #include > +#include > =20 > #include "xfs_sb.h" > #include "xfs_inum.h" > @@ -1877,14 +1878,42 @@ xfs_buf_delwri_split( > =20 > } > =20 > +/* > + * Compare function is more complex than it needs to be because > + * the return value is only 32 bits and we are doing comparisons > + * on 64 bit values > + */ > +static int > +xfs_buf_cmp( > + void *priv, > + struct list_head *a, > + struct list_head *b) > +{ > + struct xfs_buf *ap =3D container_of(a, struct xfs_buf, b_list); > + struct xfs_buf *bp =3D container_of(b, struct xfs_buf, b_list); > + xfs_daddr_t diff; > + > + diff =3D ap->b_bn - bp->b_bn; > + if (diff < 0) > + return -1; > + if (diff > 0) > + return 1; > + return 0; > +} > + > +void > +xfs_buf_delwri_sort( > + xfs_buftarg_t *target, > + struct list_head *list) > +{ > + list_sort(NULL, list, xfs_buf_cmp); > +} > + > STATIC int > xfsbufd( > void *data) > { > - struct list_head tmp; > - xfs_buftarg_t *target =3D (xfs_buftarg_t *)data; > - int count; > - xfs_buf_t *bp; > + xfs_buftarg_t *target =3D (xfs_buftarg_t *)data; > =20 > current->flags |=3D PF_MEMALLOC; > =20 > @@ -1893,6 +1922,8 @@ xfsbufd( > do { > long age =3D xfs_buf_age_centisecs * msecs_to_jiffies(10); > long tout =3D xfs_buf_timer_centisecs * msecs_to_jiffies(10); > + int count =3D 0; > + struct list_head tmp; > =20 > if (unlikely(freezing(current))) { > set_bit(XBT_FORCE_SLEEP, &target->bt_flags); > @@ -1907,11 +1938,10 @@ xfsbufd( > schedule_timeout_interruptible(tout); > =20 > xfs_buf_delwri_split(target, &tmp, age); > - count =3D 0; > + list_sort(NULL, &tmp, xfs_buf_cmp); > while (!list_empty(&tmp)) { > - bp =3D list_entry(tmp.next, xfs_buf_t, b_list); > - ASSERT(target =3D=3D bp->b_target); > - > + struct xfs_buf *bp; > + bp =3D list_first_entry(&tmp, struct xfs_buf, b_list); > list_del_init(&bp->b_list); > xfs_buf_iostrategy(bp); > count++; > @@ -1937,42 +1967,45 @@ xfs_flush_buftarg( > xfs_buftarg_t *target, > int wait) > { > - struct list_head tmp; > - xfs_buf_t *bp, *n; > + xfs_buf_t *bp; > int pincount =3D 0; > + LIST_HEAD(tmp_list); > + LIST_HEAD(wait_list); > =20 > xfs_buf_runall_queues(xfsconvertd_workqueue); > xfs_buf_runall_queues(xfsdatad_workqueue); > xfs_buf_runall_queues(xfslogd_workqueue); > =20 > set_bit(XBT_FORCE_FLUSH, &target->bt_flags); > - pincount =3D xfs_buf_delwri_split(target, &tmp, 0); > + pincount =3D xfs_buf_delwri_split(target, &tmp_list, 0); > =20 > /* > - * Dropped the delayed write list lock, now walk the temporary list > + * Dropped the delayed write list lock, now walk the temporary list. > + * All I/O is issued async and then if we need to wait for completion > + * we do that after issuing all the IO. > */ > - list_for_each_entry_safe(bp, n, &tmp, b_list) { > + list_sort(NULL, &tmp_list, xfs_buf_cmp); > + while (!list_empty(&tmp_list)) { > + bp =3D list_first_entry(&tmp_list, struct xfs_buf, b_list); > ASSERT(target =3D=3D bp->b_target); > - if (wait) > + list_del_init(&bp->b_list); > + if (wait) { > bp->b_flags &=3D ~XBF_ASYNC; > - else > - list_del_init(&bp->b_list); > - > + list_add(&bp->b_list, &wait_list); > + } > xfs_buf_iostrategy(bp); > } > =20 > - if (wait) > + if (wait) { > + /* Expedite and wait for IO to complete. */ > blk_run_address_space(target->bt_mapping); > + while (!list_empty(&wait_list)) { > + bp =3D list_first_entry(&wait_list, struct xfs_buf, b_list); > =20 > - /* > - * Remaining list items must be flushed before returning > - */ > - while (!list_empty(&tmp)) { > - bp =3D list_entry(tmp.next, xfs_buf_t, b_list); > - > - list_del_init(&bp->b_list); > - xfs_iowait(bp); > - xfs_buf_relse(bp); > + list_del_init(&bp->b_list); > + xfs_iowait(bp); > + xfs_buf_relse(bp); > + } > } > =20 > return pincount; From aelder@sgi.com Fri Feb 5 17:56:18 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o15NuHWf172455 for ; Fri, 5 Feb 2010 17:56:17 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay1.corp.sgi.com (Postfix) with ESMTP id 17D378F80C1; Fri, 5 Feb 2010 15:57:29 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 17:55:56 -0600 Subject: Re: [PATCH 05/10] xfs: Use delay write promotion for dquot flushing From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265153104-29680-6-git-send-email-david@fromorbit.com> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-6-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 17:55:55 -0600 Message-ID: <1265414155.2714.155.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 05 Feb 2010 23:55:56.0077 (UTC) FILETIME=[C1E94DD0:01CAA6BE] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > xfs_qm_dqflock_pushbuf_wait() does a very similar trick to item > pushing used to do to flush out delayed write dquot buffers. Change > it to use the new promotion method rather than an async flush. > > Also, xfs_qm_dqflock_pushbuf_wait() can return without the flush lock > held, yet the callers make the assumption that after this call the > flush lock is held. Always return with the flush lock held. Looks good. > Signed-off-by: Dave Chinner > Reviewed-by: Christoph Hellwig Reviewed-by: Alex Elder > --- > fs/xfs/quota/xfs_dquot.c | 25 ++++++++++--------------- > 1 files changed, 10 insertions(+), 15 deletions(-) > > diff --git a/fs/xfs/quota/xfs_dquot.c b/fs/xfs/quota/xfs_dquot.c > index f9baeed..1620a56 100644 > --- a/fs/xfs/quota/xfs_dquot.c > +++ b/fs/xfs/quota/xfs_dquot.c > @@ -1528,21 +1528,16 @@ xfs_qm_dqflock_pushbuf_wait( > */ > bp = xfs_incore(dqp->q_mount->m_ddev_targp, dqp->q_blkno, > XFS_QI_DQCHUNKLEN(dqp->q_mount), XBF_TRYLOCK); > - if (bp != NULL) { > - if (XFS_BUF_ISDELAYWRITE(bp)) { > - int error; > - > - if (XFS_BUF_ISPINNED(bp)) > - xfs_log_force(dqp->q_mount, 0); > - error = xfs_bawrite(dqp->q_mount, bp); > - if (error) > - xfs_fs_cmn_err(CE_WARN, dqp->q_mount, > - "xfs_qm_dqflock_pushbuf_wait: " > - "pushbuf error %d on dqp %p, bp %p", > - error, dqp, bp); > - } else { > - xfs_buf_relse(bp); > - } > + if (!bp) > + goto out_lock; > + > + if (XFS_BUF_ISDELAYWRITE(bp)) { > + if (XFS_BUF_ISPINNED(bp)) > + xfs_log_force(dqp->q_mount, 0); > + xfs_buf_delwri_promote(bp); > + wake_up_process(bp->b_target->bt_task); > } > + xfs_buf_relse(bp); > +out_lock: > xfs_dqflock(dqp); > } From SRS0+69A6+66+fromorbit.com=david@internode.on.net Fri Feb 5 18:06:47 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1606lrs173131 for ; Fri, 5 Feb 2010 18:06:47 -0600 X-ASG-Debug-ID: 1265414875-39f801490000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C55E41B4C37 for ; Fri, 5 Feb 2010 16:07:56 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id u2wMpEjSyPOE0suS for ; Fri, 05 Feb 2010 16:07:56 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12597344-1927428 for multiple; Sat, 06 Feb 2010 10:37:54 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NdYDI-0005MW-S2; Sat, 06 Feb 2010 11:07:52 +1100 Date: Sat, 6 Feb 2010 11:07:52 +1100 From: Dave Chinner To: Alex Elder Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH 01/10] xfs: Make inode reclaim states explicit Subject: Re: [PATCH 01/10] xfs: Make inode reclaim states explicit Message-ID: <20100206000752.GF11483@discord.disaster> References: <1265153104-29680-1-git-send-email-david@fromorbit.com> <1265153104-29680-2-git-send-email-david@fromorbit.com> <1265396780.2714.23.camel@doink1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1265396780.2714.23.camel@doink1> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265414877 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21762 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 01:06:20PM -0600, Alex Elder wrote: > On Wed, 2010-02-03 at 10:24 +1100, Dave Chinner wrote: > > A.K.A.: don't rely on xfs_iflush() return value in reclaim > > > > We have gradually been moving checks out of the reclaim code because > > they are duplicated in xfs_iflush(). We've had a history of problems > > in this area, and many of them stem from the overloading of the > > return values from xfs_iflush() and interaction with inode flush > > locking to determine if the inode is safe to reclaim. > > > > With the desire to move to delayed write flushing of inodes and > > non-blocking inode tree reclaim walks, the overloading of the > > return value of xfs_iflush makes it very difficult to determine > > the correct thing to do next. > > > > This patch explicitly re-adds the checks to the inode reclaim code, > > removing the reliance on the return value of xfs_iflush() to > > determine what to do next. It also means that we can clearly > > document all the inode states that reclaim must handle and hence > > we can easily see that we handled all the necessary cases. > > > > This also removes the need for the xfs_inode_clean() check in > > xfs_iflush() as all callers now check this first (safely). > > I have a few comments, below. One is a real bug. At this > point I'm trusting that your enumeration of all the possible > inode states is correct. Other than what I indicate below, > this change looks good. ... > It looks to me like your code adds a call to xfs_ifunlock(ip) > in the bad inode case that was not there before. My guess is > that your way is correct, but I'd like to know whether you > agree this differs. Is this intentional? Regardless, is > the new xfs_ifunlock() call correct? Yes, it is correct - the code before was broken. > > + if (xfs_iflags_test(ip, XFS_ISTALE)) > > + goto reclaim; > > + if (xfs_inode_clean(ip)) > > + goto reclaim; > > + > > + /* Now we have an inode that needs flushing */ > > + error = xfs_iflush(ip, sync_mode); > > + if (!error) { > > + switch(sync_mode) { > > + case XFS_IFLUSH_DELWRI_ELSE_ASYNC: > > + case XFS_IFLUSH_DELWRI: > > + case XFS_IFLUSH_ASYNC: > > + case XFS_IFLUSH_DELWRI_ELSE_SYNC: > > + case XFS_IFLUSH_SYNC: > > + /* IO issued, synchronise with IO completion */ > > + xfs_iflock(ip); > > You must not have run this with DEBUG enabled. > You need a "break;" here. I always run with DEBUG enabled, but I probably didn't test this patch by iteself in the last iteration because i don't have unlimited test cycles. This code gets removed by the next patch, so the fact that it was broken would not have been picked up by testing. I'll update it and re-push it to the for-2.6.34 branch. Cheers, -- Dave Chinner david@fromorbit.com From aelder@sgi.com Fri Feb 5 19:40:56 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o161euPR178328 for ; Fri, 5 Feb 2010 19:40:56 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay1.corp.sgi.com (Postfix) with ESMTP id 6AFDA8F80BE; Fri, 5 Feb 2010 17:42:03 -0800 (PST) Received: from [128.162.232.188] ([128.162.232.188]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 5 Feb 2010 19:41:15 -0600 Subject: Re: PATCH V2] more reserved blocks fixups From: Alex Elder Reply-To: aelder@sgi.com To: Eric Sandeen Cc: xfs mailing list In-Reply-To: <4B6CA2E9.7020803@sandeen.net> References: <4B60C8EE.5080700@sandeen.net> <4B6CA2E9.7020803@sandeen.net> Content-Type: text/plain; charset="UTF-8" Date: Fri, 05 Feb 2010 19:41:15 -0600 Message-ID: <1265420475.2714.162.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 06 Feb 2010 01:41:15.0914 (UTC) FILETIME=[78D3E6A0:01CAA6CD] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, 2010-02-05 at 16:59 -0600, Eric Sandeen wrote: > This mangles the reserved blocks counts a little more. > > 1) add a helper function for the default reserved count > 2) add helper functions to save/restore counts on ro/rw > 3) save/restore reserved blocks on freeze/thaw > 4) disallow changing reserved count while readonly Looks good. Thanks for updating it. > Signed-off-by: Eric Sandeen Reviewed-by: Alex Elder > --- > > V2: changed field name to match Dave's changes > > diff --git a/fs/xfs/linux-2.6/xfs_ioctl.c b/fs/xfs/linux-2.6/xfs_ioctl.c > index a034cf6..883929b 100644 > --- a/fs/xfs/linux-2.6/xfs_ioctl.c > +++ b/fs/xfs/linux-2.6/xfs_ioctl.c > @@ -1431,6 +1431,9 @@ xfs_file_ioctl( > if (!capable(CAP_SYS_ADMIN)) > return -EPERM; > > + if (mp->m_flags & XFS_MOUNT_RDONLY) > + return -XFS_ERROR(EROFS); > + > if (copy_from_user(&inout, arg, sizeof(inout))) > return -XFS_ERROR(EFAULT); > > diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c > index 8884f26..97c0f5a 100644 > --- a/fs/xfs/linux-2.6/xfs_super.c > +++ b/fs/xfs/linux-2.6/xfs_super.c > @@ -1257,6 +1257,29 @@ xfs_fs_statfs( > return 0; > } > > +STATIC void > +xfs_save_resvblks(struct xfs_mount *mp) > +{ > + __uint64_t resblks = 0; > + > + mp->m_resblks_save = mp->m_resblks; > + xfs_reserve_blocks(mp, &resblks, NULL); > +} > + > +STATIC void > +xfs_restore_resvblks(struct xfs_mount *mp) > +{ > + __uint64_t resblks; > + > + if (mp->m_resblks_save) { > + resblks = mp->m_resblks_save; > + mp->m_resblks_save = 0; > + } else > + resblks = xfs_default_resblks(mp); > + > + xfs_reserve_blocks(mp, &resblks, NULL); > +} > + > STATIC int > xfs_fs_remount( > struct super_block *sb, > @@ -1319,8 +1342,6 @@ xfs_fs_remount( > > /* ro -> rw */ > if ((mp->m_flags & XFS_MOUNT_RDONLY) && !(*flags & MS_RDONLY)) { > - __uint64_t resblks; > - > mp->m_flags &= ~XFS_MOUNT_RDONLY; > if (mp->m_flags & XFS_MOUNT_BARRIER) > xfs_mountfs_check_barriers(mp); > @@ -1343,15 +1364,7 @@ xfs_fs_remount( > * Fill out the reserve pool if it is empty. Use the stashed > * value if it is non-zero, otherwise go with the default. > */ > - if (mp->m_resblks_save) { > - resblks = mp->m_resblks_save; > - mp->m_resblks_save = 0; > - } else { > - resblks = mp->m_sb.sb_dblocks; > - do_div(resblks, 20); > - resblks = min_t(__uint64_t, resblks, 1024); > - } > - xfs_reserve_blocks(mp, &resblks, NULL); > + xfs_restore_resvblks(mp); > } > > /* rw -> ro */ > @@ -1364,11 +1377,9 @@ xfs_fs_remount( > * so that if we get remounted rw, we can return it to the same > * size. > */ > - __uint64_t resblks = 0; > > xfs_quiesce_data(mp); > - mp->m_resblks_save = mp->m_resblks; > - xfs_reserve_blocks(mp, &resblks, NULL); > + xfs_save_resvblks(mp); > xfs_quiesce_attr(mp); > mp->m_flags |= XFS_MOUNT_RDONLY; > } > @@ -1387,11 +1398,22 @@ xfs_fs_freeze( > { > struct xfs_mount *mp = XFS_M(sb); > > + xfs_save_resvblks(mp); > xfs_quiesce_attr(mp); > return -xfs_fs_log_dummy(mp); > } > > STATIC int > +xfs_fs_unfreeze( > + struct super_block *sb) > +{ > + struct xfs_mount *mp = XFS_M(sb); > + > + xfs_restore_resvblks(mp); > + return 0; > +} > + > +STATIC int > xfs_fs_show_options( > struct seq_file *m, > struct vfsmount *mnt) > @@ -1613,6 +1635,7 @@ static const struct super_operations xfs_super_operations = { > .put_super = xfs_fs_put_super, > .sync_fs = xfs_fs_sync_fs, > .freeze_fs = xfs_fs_freeze, > + .unfreeze_fs = xfs_fs_unfreeze, > .statfs = xfs_fs_statfs, > .remount_fs = xfs_fs_remount, > .show_options = xfs_fs_show_options, > diff --git a/fs/xfs/xfs_mount.c b/fs/xfs/xfs_mount.c > index eb403b4..48e1ee7 100644 > --- a/fs/xfs/xfs_mount.c > +++ b/fs/xfs/xfs_mount.c > @@ -1008,6 +1008,22 @@ xfs_mount_reset_sbqflags( > return xfs_trans_commit(tp, 0); > } > > +__uint64_t > +xfs_default_resblks(xfs_mount_t *mp) > +{ > + __uint64_t resblks; > + > + /* > + * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. > + * This may drive us straight to ENOSPC on mount, but that implies > + * we were already there on the last unmount. Warn if this occurs. > + */ > + resblks = mp->m_sb.sb_dblocks; > + do_div(resblks, 20); > + resblks = min_t(__uint64_t, resblks, 1024); > + return resblks; > +} > + > /* > * This function does the following on an initial mount of a file system: > * - reads the superblock from disk and init the mount struct > @@ -1318,18 +1334,14 @@ xfs_mountfs( > * when at ENOSPC. This is needed for operations like create with > * attr, unwritten extent conversion at ENOSPC, etc. Data allocations > * are not allowed to use this reserved space. > - * > - * We default to 5% or 1024 fsbs of space reserved, whichever is smaller. > - * This may drive us straight to ENOSPC on mount, but that implies > - * we were already there on the last unmount. Warn if this occurs. > */ > - resblks = mp->m_sb.sb_dblocks; > - do_div(resblks, 20); > - resblks = min_t(__uint64_t, resblks, 1024); > - error = xfs_reserve_blocks(mp, &resblks, NULL); > - if (error) > - cmn_err(CE_WARN, "XFS: Unable to allocate reserve blocks. " > - "Continuing without a reserve pool."); > + if (!(mp->m_flags & XFS_MOUNT_RDONLY)) { > + resblks = xfs_default_resblks(mp); > + error = xfs_reserve_blocks(mp, &resblks, NULL); > + if (error) > + cmn_err(CE_WARN, "XFS: Unable to allocate reserve " > + "blocks. Continuing without a reserve pool."); > + } > > return 0; > > diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h > index 996273d..c8f880d 100644 > --- a/fs/xfs/xfs_mount.h > +++ b/fs/xfs/xfs_mount.h > @@ -429,6 +429,7 @@ typedef struct xfs_mod_sb { > } xfs_mod_sb_t; > > extern int xfs_log_sbcount(xfs_mount_t *, uint); > +extern __uint64_t xfs_default_resblks(xfs_mount_t *mp); > extern int xfs_mountfs(xfs_mount_t *mp); > > extern void xfs_unmountfs(xfs_mount_t *); > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From julia@diku.dk Sat Feb 6 02:44:09 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o168i8tg204495 for ; Sat, 6 Feb 2010 02:44:09 -0600 X-ASG-Debug-ID: 1265445918-5440013f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mgw2.diku.dk (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 351E41CA7BD5; Sat, 6 Feb 2010 00:45:18 -0800 (PST) Received: from mgw2.diku.dk (mgw2.diku.dk [130.225.96.92]) by cuda.sgi.com with ESMTP id 9JUXhSuG2GLuNFqk; Sat, 06 Feb 2010 00:45:18 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by mgw2.diku.dk (Postfix) with ESMTP id 74B3C19BBF8; Sat, 6 Feb 2010 09:45:16 +0100 (CET) Received: from mgw2.diku.dk ([127.0.0.1]) by localhost (mgw2.diku.dk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 06564-17; Sat, 6 Feb 2010 09:45:15 +0100 (CET) Received: from nhugin.diku.dk (nhugin.diku.dk [130.225.96.140]) by mgw2.diku.dk (Postfix) with ESMTP id 53FEF19BBA2; Sat, 6 Feb 2010 09:45:15 +0100 (CET) Received: from ask.diku.dk (ask.diku.dk [130.225.96.225]) by nhugin.diku.dk (Postfix) with ESMTP id 7F8916DF8B9; Sat, 6 Feb 2010 09:40:12 +0100 (CET) Received: by ask.diku.dk (Postfix, from userid 3767) id 3CE58200AA; Sat, 6 Feb 2010 09:45:15 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by ask.diku.dk (Postfix) with ESMTP id 326A0200A9; Sat, 6 Feb 2010 09:45:15 +0100 (CET) Date: Sat, 6 Feb 2010 09:45:15 +0100 (CET) From: Julia Lawall To: Alex Elder , xfs-masters@oss.sgi.com, xfs@oss.sgi.com, linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org X-ASG-Orig-Subj: [PATCH 11/11] fs/xfs: Correct NULL test Subject: [PATCH 11/11] fs/xfs: Correct NULL test Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: amavisd-new at diku.dk X-Barracuda-Connect: mgw2.diku.dk[130.225.96.92] X-Barracuda-Start-Time: 1265445919 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean From: Julia Lawall Test the value that was just allocated rather than the previously tested one. A simplified version of the semantic match that finds this problem is as follows: (http://coccinelle.lip6.fr/) // @r@ expression *x; expression e; identifier l; @@ if (x == NULL || ...) { ... when forall return ...; } ... when != goto l; when != x = e when != &x *x == NULL // Signed-off-by: Julia Lawall --- fs/xfs/quota/xfs_qm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/xfs/quota/xfs_qm.c b/fs/xfs/quota/xfs_qm.c index 11cfd82..4318cd5 100644 --- a/fs/xfs/quota/xfs_qm.c +++ b/fs/xfs/quota/xfs_qm.c @@ -123,7 +123,7 @@ xfs_Gqm_init(void) goto out; gdqhash = kmem_zalloc_large(hsize); - if (!udqhash) + if (!gdqhash) goto out_free_udqhash; hsize /= sizeof(xfs_dqhash_t); From t.landschoff@gmx.net Sat Feb 6 14:36:41 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o16KaeQ1243429 for ; Sat, 6 Feb 2010 14:36:41 -0600 X-ASG-Debug-ID: 1265488669-105f022a0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.gmx.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with SMTP id D54841383C07 for ; Sat, 6 Feb 2010 12:37:50 -0800 (PST) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by cuda.sgi.com with SMTP id 0wkvDxkFew9OBkpe for ; Sat, 06 Feb 2010 12:37:50 -0800 (PST) Received: (qmail invoked by alias); 06 Feb 2010 20:37:49 -0000 Received: from p5DDAAA53.dip.t-dialin.net (EHLO [192.168.2.115]) [93.218.170.83] by mail.gmx.net (mp012) with SMTP; 06 Feb 2010 21:37:49 +0100 X-Authenticated: #597135 X-Provags-ID: V01U2FsdGVkX1/cBz0YYLfcN45ST6MZJaXd4okGUPQAtMc01smLrd JW+ecaEdxcBmOf X-ASG-Orig-Subj: Abysmal performance with XFS on LVM on md (raid1) Subject: Abysmal performance with XFS on LVM on md (raid1) From: Torsten Landschoff To: xfs@oss.sgi.com Cc: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Date: Sat, 06 Feb 2010 21:37:43 +0100 Message-ID: <1265488663.2739.24.camel@sharokan.intern> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 X-FuHaFi: 0.51000000000000001 X-Barracuda-Connect: mail.gmx.net[213.165.64.20] X-Barracuda-Start-Time: 1265488671 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.02 X-Barracuda-Spam-Status: No, SCORE=-1.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M, BSF_RULE_7582B X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21839 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M 0.50 BSF_RULE_7582B Custom Rule 7582B X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hi List, A few days before I went to do a few test builds of wxWidgets on my home system which I recently upgraded. I was quite surprised when it performed much worse than my old notebook. Turns out that this is related to barriers support: $ time tar xzf wxwidgets2.8_2.8.10.1.orig.tar.gz real 1m10.899s user 0m2.996s sys 0m2.088s $ time rm -rf wxwidgets2.8-2.8.10.1.orig real 3m9.067s user 0m0.036s sys 0m1.352s $ sudo mount -o remount,nobarrier /dev/vgsys/home /home $ time tar xzf wxwidgets2.8_2.8.10.1.orig.tar.gz real 0m1.640s user 0m1.264s sys 0m1.200s $ time rm -rf wxwidgets2.8-2.8.10.1.orig real 0m2.783s user 0m0.024s sys 0m0.796s I knew that barriers could cause some slowdowns but this is above my worst estimations. Truth to be told, I did not notice that barriers are enabled in my setup now and found the commit that affected xfs performance the hard way (using git bisect). Here is the bisect log: git bisect start # bad: [b4bdd73ce865213a5653dc424873e8da37e858cc] Linux 2.6.32.7 git bisect bad b4bdd73ce865213a5653dc424873e8da37e858cc # good: [07a2039b8eb0af4ff464efd3dfd95de5c02648c6] Linux 2.6.30 git bisect good 07a2039b8eb0af4ff464efd3dfd95de5c02648c6 # bad: [19720737187aaee006afb20e63be5e9eddc505a8] sky2: hold RTNL when doing suspend/shutdown operations git bisect bad 19720737187aaee006afb20e63be5e9eddc505a8 # good: [49809d6a511960e5ccfb85b780894f45ac119065] V4L/DVB (11970): gspca - ov519: Add support for the ov518 bridge. git bisect good 49809d6a511960e5ccfb85b780894f45ac119065 # bad: [5a4f13fad1ab5bd08dea78fc55321e429d83cddf] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/ide-2.6 git bisect bad 5a4f13fad1ab5bd08dea78fc55321e429d83cddf # good: [e6423407d01168f7760cdee7270d9f51d1240301] Merge branch 'for-2.6.31' of git://git.kernel.org/pub/scm/linux/kernel/git/bart/ide-2.6 git bisect good e6423407d01168f7760cdee7270d9f51d1240301 # good: [a200ad22bb15fe01cf222fa631687876baad5e01] Blackfin: update anomaly lists git bisect good a200ad22bb15fe01cf222fa631687876baad5e01 # skip: [0c26d7cc31cd81a82be3b9d7687217d49fe9c47e] Merge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6 git bisect skip 0c26d7cc31cd81a82be3b9d7687217d49fe9c47e # bad: [71f9dacd2e4d233029e9e956ca3f79531f411827] inet: Call skb_orphan before tproxy activates git bisect bad 71f9dacd2e4d233029e9e956ca3f79531f411827 # good: [01c031945f2755c7afaaf456088543312f2b72ea] cleanup __writeback_single_inode git bisect good 01c031945f2755c7afaaf456088543312f2b72ea # good: [b81c087f6deb049023e41ce00717202a953f3939] uwb: allow WLP to be used with IPv6. git bisect good b81c087f6deb049023e41ce00717202a953f3939 # good: [1962f39abbb2d5643a7d59169422661a2d58793d] ocfs2: Update atime in splice read if necessary. git bisect good 1962f39abbb2d5643a7d59169422661a2d58793d # good: [caf420c68afe01acd7c458ce40b85b3db5330ff5] ACPI: pci_root: use driver data rather than list lookup git bisect good caf420c68afe01acd7c458ce40b85b3db5330ff5 # skip: [ba52270d18fb17ce2cf176b35419dab1e43fe4a3] SLUB: Don't pass __GFP_FAIL for the initial allocation git bisect skip ba52270d18fb17ce2cf176b35419dab1e43fe4a3 # good: [871043bc463e7d191e7b5b00436a8852921dd833] hp-wmi: Add support for reporting tablet state git bisect good 871043bc463e7d191e7b5b00436a8852921dd833 # good: [d7880f10c5d42ba182a97c1fd41d41d0b8837097] thinkpad-acpi: forbid the use of HBRV on Lenovo ThinkPads git bisect good d7880f10c5d42ba182a97c1fd41d41d0b8837097 # good: [0c526d96a5bd86c70507b7d9372e6a26a1e3ea43] ACPI: clean up whitespace in drivers/acpi/scan.c git bisect good 0c526d96a5bd86c70507b7d9372e6a26a1e3ea43 # good: [281eede0328c84a8f20e0e85b807d5b51c3de4f2] switch reiserfs to inode->i_acl git bisect good 281eede0328c84a8f20e0e85b807d5b51c3de4f2 # skip: [9937ac0cc087b03d6d73f46a5d6b38c43626e60e] MAINTAINERS: Change mailing list info for CRIS git bisect skip 9937ac0cc087b03d6d73f46a5d6b38c43626e60e # good: [0b47b5703b1cc6c3aa89663ac70e28dadedf6ccc] w1: ds2760: add support for EEPROM read and write git bisect good 0b47b5703b1cc6c3aa89663ac70e28dadedf6ccc # good: [8d8890b7751387f58ce0a6428773de2fbc0fd596] netfilter: nf_conntrack: fix conntrack lookup race git bisect good 8d8890b7751387f58ce0a6428773de2fbc0fd596 # skip: [c82e6d450fda56cb2d4f68534173d3cd11b32f9f] Merge branch 'upstream' of git://ftp.linux-mips.org/pub/scm/upstream-linus git bisect skip c82e6d450fda56cb2d4f68534173d3cd11b32f9f # bad: [1f2ccd00f224a4e2d6d26f590f3e6851f3deef99] ipv6: Use rcu_barrier() on module unload. git bisect bad 1f2ccd00f224a4e2d6d26f590f3e6851f3deef99 # bad: [a1faa69810b2af562b70b2a71c116c7d03575dd3] ipv6: avoid wraparound for expired preferred lifetime git bisect bad a1faa69810b2af562b70b2a71c116c7d03575dd3 # good: [8cbeb67ad50f7d68e5e83be2cb2284de8f9c03b5] dm: avoid unsupported spanning of md stripe boundaries git bisect good 8cbeb67ad50f7d68e5e83be2cb2284de8f9c03b5 # good: [b6280b47a7a42970d098a3059f4ebe7e55e90d8d] ipv4 routing: Ensure that route cache entries are usable and reclaimable with caching is off git bisect good b6280b47a7a42970d098a3059f4ebe7e55e90d8d # good: [c4658b4e777bebf69884f4884a9bfb2f84dd71d9] Intel-IOMMU, intr-remap: set the whole 128bits of irte when modify/free it git bisect good c4658b4e777bebf69884f4884a9bfb2f84dd71d9 # good: [48fe112744d1ff2e899a6491633ac58a3229aabf] ACPI: ac: use .notify method instead of installing handler directly git bisect good 48fe112744d1ff2e899a6491633ac58a3229aabf # skip: [0705495d9010048e293013d9d129cf723363a0a8] ACPI: pci_root: remove unused dev/fn information git bisect skip 0705495d9010048e293013d9d129cf723363a0a8 # bad: [60935eb21d3c5bac79618000f38f92c249d153c4] dm ioctl: support cookies for udev git bisect bad 60935eb21d3c5bac79618000f38f92c249d153c4 # bad: [374bf7e7f6cc38b0483351a2029a97910eadde1b] dm: stripe support flush git bisect bad 374bf7e7f6cc38b0483351a2029a97910eadde1b # good: [5aa2781d964e9835c441932110484bc454b5c207] dm: store only first barrier error git bisect good 5aa2781d964e9835c441932110484bc454b5c207 # good: [9015df24a8008d7bea2bd3df881783ebe0dcb9af] dm: initialise tio in alloc_tio git bisect good 9015df24a8008d7bea2bd3df881783ebe0dcb9af # bad: [433bcac5645508b71eab2710b6817c3ef937eba8] dm: linear support flush git bisect bad 433bcac5645508b71eab2710b6817c3ef937eba8 I am actually glad that barrier support is now working, but this performance drop is just more than I had expected. Especially, since a single barrier would suffice here - write out all the file content and ensure the meta data is only written after the first is completed. Any hints how to improve on this? Greetings and thanks in advance, Torsten From patrick@news-service.com Mon Feb 8 04:15:11 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.7 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18AF9FZ123766 for ; Mon, 8 Feb 2010 04:15:10 -0600 X-ASG-Debug-ID: 1265624180-51c703050000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from pu01.news-service.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E39A21CADBD3 for ; Mon, 8 Feb 2010 02:16:20 -0800 (PST) Received: from pu01.news-service.com (ns1.news-service.com [195.114.240.3]) by cuda.sgi.com with ESMTP id YJpdCase28lZYDp5 for ; Mon, 08 Feb 2010 02:16:20 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by pu01.news-service.com (Postfix) with ESMTP id A8B3F13E25; Mon, 8 Feb 2010 11:16:19 +0100 (CET) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Debian amavisd-new at pu01.news-service.com Received: from pu01.news-service.com ([127.0.0.1]) by localhost (pu01.nse [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Nhjuh7ZfPvdu; Mon, 8 Feb 2010 11:16:17 +0100 (CET) Received: from [172.25.0.47] (nse01.nse [172.25.0.47]) by pu01.news-service.com (Postfix) with ESMTP id 73EB513E23; Mon, 8 Feb 2010 11:16:17 +0100 (CET) Message-ID: <4B6FE47A.7070208@news-service.com> Date: Mon, 08 Feb 2010 11:16:26 +0100 From: Patrick Schreurs User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Dave Chinner CC: Christoph Hellwig , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) References: <89c4f90c0910270341r7833f490g60810f2817eb0950@mail.gmail.com> <89c4f90c0910280519k759230c1r7b1586932ac792f7@mail.gmail.com> <20091030101601.GA11142@infradead.org> <4AF0422D.1070104@news-service.com> <20091114162126.GB17658@infradead.org> <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> In-Reply-To: <4B6706CE.1020207@news-service.com> Content-Type: multipart/mixed; boundary="------------020806050006070405000203" X-Barracuda-Connect: ns1.news-service.com[195.114.240.3] X-Barracuda-Start-Time: 1265624181 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21978 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean This is a multi-part message in MIME format. --------------020806050006070405000203 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit We had another crash on the same server last weekend. Looks the same to me. Thanks for looking into this. -Patrick On 1-2-2010 17:52, Patrick Schreurs wrote: > Hello Dave, > > I'm afraid we had another crash a few days ago. Have a look at the > screenshot. Can you make anything out of it? This server is running > 2.6.32.3 with your inode-reclaim patch applied and XFS_DEBUG still enabled. > > Thanks for looking into this. > > -Patrick > > On 15-1-2010 12:01, Patrick Schreurs wrote: >> Hi Dave, >> >> I think it's save to consider this issue fixed. We have currently 9 >> servers operational with these patches and they have been stable so far. >> For a 100% certainty we'll have to test/wait a little bit longer, but >> considering the frequency of crashes we saw earlier i think it's save to >> come to a conclusion. >> >> I hope these patches will be included in 2.6.33 and will be back ported >> to at least 2.6.32. >> >> Many thanks to Dave and to Christoph for fixing this apparently rare and >> seldom triggered condition. >> >> -Patrick >> >> Dave Chinner wrote: >>> Hi Patrick, >>> >>> I've attached two compendium patches that will hopefully fix >>> the inode reclaim problems you've been seeing - one is for 2.6.31, >>> the other is for 2.6.32. I've cc'd this to the XFS list Ñ•o that >>> anyone else who has been seeing crashes, assert failures and >>> general nastiness around inode reclaim can test them as well. >>> >>> These are not final patches - there's a few changes that Christoph >>> has picked up on during review - so there'll be another round of >>> patches before checkins and -stable backports can be requested. >>> >>> I'm hoping that these patches fix your problem, because with them >>> I can't make my machines fall over anymore.... >>> >>> Cheers, >>> >>> Dave. >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> xfs mailing list >>> xfs@oss.sgi.com >>> http://oss.sgi.com/mailman/listinfo/xfs >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs > > > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs --------------020806050006070405000203-- From BATV+6409d81330fd2592ba7a+2360+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 8 04:28:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18ASUAo124504 for ; Mon, 8 Feb 2010 04:28:32 -0600 X-ASG-Debug-ID: 1265624983-0fa700050000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 53F431CADCAA for ; Mon, 8 Feb 2010 02:29:43 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id ncwwOLB85m030Gzo for ; Mon, 08 Feb 2010 02:29:43 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeQsA-0008I8-QJ; Mon, 08 Feb 2010 10:29:42 +0000 Date: Mon, 8 Feb 2010 05:29:42 -0500 From: Christoph Hellwig To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: make install in the brave new build system world Subject: Re: make install in the brave new build system world Message-ID: <20100208102942.GA25395@infradead.org> References: <20100205092229.GA32454@infradead.org> <20100205104414.GC11483@discord.disaster> <20100205112850.GB28701@infradead.org> <20100205114856.GE11483@discord.disaster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100205114856.GE11483@discord.disaster> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265624983 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 10:48:56PM +1100, Dave Chinner wrote: > Ah, there's a double depenency chain. > > The top level make file has: > > install: default > > Which causes "make install" to run the top level default target, > which runs the default target in all the target subdirs. > > Then, in each subdir, the makefile has: > > install: default > > Which when then install target is actually run, does another > dependency check because it's got a local dependency on the depend > target via the default target. > > Replace the previous patch with the one below and try again. > Now the "make install" will rebuild targets out of the local > dependencies rather than a separate run of the top level default > target (i.e. only traverse directories once). Thanks, this looks much better. We still don't implement the quite behaviour for most install targets, but that should be relatively simple to fix. Maybe we can even replace all those handcode install rules with generic one - I'll look into that once I get a bit time. So Reviewed-by: Christoph Hellwig for this patch for now. From ccoager@gmail.com Mon Feb 8 08:39:38 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18EdcZO138762 for ; Mon, 8 Feb 2010 08:39:38 -0600 X-ASG-Debug-ID: 1265640050-0d7200ab0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from cdptpa-omtalb.mail.rr.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id CC9611B92BA for ; Mon, 8 Feb 2010 06:40:50 -0800 (PST) Received: from cdptpa-omtalb.mail.rr.com (cdptpa-omtalb.mail.rr.com [75.180.132.121]) by cuda.sgi.com with ESMTP id purgZyHZ1iOKCD2g for ; Mon, 08 Feb 2010 06:40:50 -0800 (PST) X-Authority-Analysis: v=1.0 c=1 a=11pvPlEd5bsG_rgWPRIA:9 a=nlBqkPCv-f2NXtOSUh9Wdb2vAcEA:4 X-Cloudmark-Score: 0 X-Originating-IP: 72.224.58.127 Received: from [72.224.58.127] ([72.224.58.127:52906] helo=erebus.underworld.local) by cdptpa-oedge01.mail.rr.com (envelope-from ) (ecelerity 2.2.2.39 r()) with ESMTP id 6D/F4-28731-272207B4; Mon, 08 Feb 2010 14:40:50 +0000 Received: from localhost (localhost [127.0.0.1]) by erebus.underworld.local (Postfix) with ESMTP id 522AD1FD58 for ; Mon, 8 Feb 2010 09:40:48 -0500 (EST) Received: from erebus.underworld.local ([127.0.0.1]) by localhost (erebus.underworld.local [127.0.0.1]) (amavisd-maia, port 10024) with ESMTP id 12634-04 for ; Mon, 8 Feb 2010 09:40:38 -0500 (EST) Received: from erebus.underworld.local (erebus.underworld.local [192.168.0.4]) by erebus.underworld.local (Postfix) with ESMTPS id B161D1FD43 for ; Mon, 8 Feb 2010 09:40:37 -0500 (EST) Date: Mon, 8 Feb 2010 09:40:35 -0500 From: Cory Coager To: xfs@oss.sgi.com X-ASG-Orig-Subj: format and xfs_info don't match Subject: format and xfs_info don't match Message-ID: <20100208144035.GA12922@erebus.underworld.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: Fachschaft Informatik X-OS: Linux erebus 2.6.28-hardened-r9 x86_64 User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Maia Mailguard 1.0.2a X-Barracuda-Connect: cdptpa-omtalb.mail.rr.com[75.180.132.121] X-Barracuda-Start-Time: 1265640050 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.21994 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean For some reason the format attributes don't match the mounted attributes. If you look below, lazy-count and attr aren't getting set. Why is this happening? # mkfs.xfs -f -i size=1024,attr=2 -l version=2,size=128m,lazy-count=1 /dev/mapper/360a9800050334c59543455596d625a77 meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 agcount=32, agsize=8388744 blks = sectsz=512 attr=2 data = bsize=4096 blocks=268439808, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 # mount -t xfs -o noatime,nodiratime,nobarrier /dev/mapper/360a9800050334c59543455596d625a77 /mnt/new-homes/ # xfs_info /mnt/new-homes/ meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 agcount=32, agsize=8388744 blks = sectsz=512 attr=0 data = bsize=4096 blocks=268439808, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 From eparis@parisplace.org Mon Feb 8 12:00:36 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18I0ZEn150687 for ; Mon, 8 Feb 2010 12:00:35 -0600 X-ASG-Debug-ID: 1265652107-1b9202330000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail-vw0-f53.google.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 46E3C1CB1484 for ; Mon, 8 Feb 2010 10:01:47 -0800 (PST) Received: from mail-vw0-f53.google.com (mail-vw0-f53.google.com [209.85.212.53]) by cuda.sgi.com with ESMTP id LfUuInnnulTtWEvc for ; Mon, 08 Feb 2010 10:01:47 -0800 (PST) Received: by vws10 with SMTP id 10so1569798vws.26 for ; Mon, 08 Feb 2010 10:01:47 -0800 (PST) MIME-Version: 1.0 Received: by 10.220.122.24 with SMTP id j24mr11053629vcr.88.1265652106462; Mon, 08 Feb 2010 10:01:46 -0800 (PST) In-Reply-To: <20100125042925.GM25842@discord.disaster> References: <20100125150456.c796f0b0.sfr@canb.auug.org.au> <20100125042925.GM25842@discord.disaster> Date: Mon, 8 Feb 2010 13:01:46 -0500 Message-ID: <7e0fb38c1002081001y111bd0a2p40a8a896b0668d13@mail.gmail.com> X-ASG-Orig-Subj: Re: linux-next: vfs/fsnotify trees build warning Subject: Re: linux-next: vfs/fsnotify trees build warning From: Eric Paris To: Dave Chinner Cc: Stephen Rothwell , Eric Paris , Al Viro , linux-next@vger.kernel.org, linux-kernel@vger.kernel.org, xfs@oss.sgi.com Content-Type: text/plain; charset=ISO-8859-1 X-Barracuda-Connect: mail-vw0-f53.google.com[209.85.212.53] X-Barracuda-Start-Time: 1265652108 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22008 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sun, Jan 24, 2010 at 11:29 PM, Dave Chinner wrote: > On Mon, Jan 25, 2010 at 03:04:56PM +1100, Stephen Rothwell wrote: >> Hi Eric, Al, >> >> Today's linux-next build (x86_64 allmodconfig) produced these warnings: I just switched fsnotify to use const unsigned char * so hopefully they will shut up tomorrow.... -Eric From sandeen@sandeen.net Mon Feb 8 13:17:22 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18JHLlp154663 for ; Mon, 8 Feb 2010 13:17:21 -0600 X-ASG-Debug-ID: 1265656713-374502900000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 08EAF138886F for ; Mon, 8 Feb 2010 11:18:33 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id H0MyYQpQGDHOJQKB for ; Mon, 08 Feb 2010 11:18:33 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id D051812AEC1E; Mon, 8 Feb 2010 13:18:32 -0600 (CST) Message-ID: <4B706388.7080309@sandeen.net> Date: Mon, 08 Feb 2010 13:18:32 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Cory Coager CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: format and xfs_info don't match Subject: Re: format and xfs_info don't match References: <20100208144035.GA12922@erebus.underworld.local> In-Reply-To: <20100208144035.GA12922@erebus.underworld.local> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265656714 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22013 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Cory Coager wrote: > For some reason the format attributes don't match the mounted > attributes. If you look below, lazy-count and attr aren't getting set. > Why is this happening? which xfsprogs version? # /sbin/mkfs.xfs -f -i size=1024,attr=2 -l version=2,size=128m,lazy-count=1 -bsize=4096 -dsize=268439808b,file,name=testfile meta-data=testfile isize=1024 agcount=4, agsize=67109952 blks = sectsz=512 attr=2 data = bsize=4096 blocks=268439808, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 # mount -o loop testfile mnt/ # xfs_info mnt/ meta-data=/dev/loop0 isize=1024 agcount=4, agsize=67109952 blks = sectsz=512 attr=2 data = bsize=4096 blocks=268439808, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal bsize=4096 blocks=32768, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 # mkfs.xfs -V mkfs.xfs version 3.0.1 -Eric > # mkfs.xfs -f -i size=1024,attr=2 -l version=2,size=128m,lazy-count=1 /dev/mapper/360a9800050334c59543455596d625a77 > meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 > agcount=32, agsize=8388744 blks > = sectsz=512 attr=2 > data = bsize=4096 blocks=268439808, imaxpct=25 > = sunit=0 swidth=0 blks, unwritten=1 > naming =version 2 bsize=4096 > log =internal log bsize=4096 blocks=32768, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > > # mount -t xfs -o noatime,nodiratime,nobarrier > /dev/mapper/360a9800050334c59543455596d625a77 /mnt/new-homes/ > > # xfs_info /mnt/new-homes/ > meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 > agcount=32, agsize=8388744 blks > = sectsz=512 attr=0 > data = bsize=4096 blocks=268439808, imaxpct=25 > = sunit=0 swidth=0 blks, unwritten=1 > naming =version 2 bsize=4096 > log =internal bsize=4096 blocks=32768, version=2 > = sectsz=512 sunit=0 blks, lazy-count=0 > realtime =none extsz=4096 blocks=0, rtextents=0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > From BATV+6409d81330fd2592ba7a+2360+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 8 13:36:03 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18Ja1FJ155626 for ; Mon, 8 Feb 2010 13:36:02 -0600 X-ASG-Debug-ID: 1265657834-1c1000460000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9412E1CB1932 for ; Mon, 8 Feb 2010 11:37:14 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id KR08t0C6XKyARgeN for ; Mon, 08 Feb 2010 11:37:14 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeZPm-0002n2-EN; Mon, 08 Feb 2010 19:36:58 +0000 Date: Mon, 8 Feb 2010 14:36:58 -0500 From: Christoph Hellwig To: Eric Paris Cc: Dave Chinner , Stephen Rothwell , linux-kernel@vger.kernel.org, Eric Paris , xfs@oss.sgi.com, linux-next@vger.kernel.org, Al Viro X-ASG-Orig-Subj: Re: linux-next: vfs/fsnotify trees build warning Subject: Re: linux-next: vfs/fsnotify trees build warning Message-ID: <20100208193658.GA9527@infradead.org> References: <20100125150456.c796f0b0.sfr@canb.auug.org.au> <20100125042925.GM25842@discord.disaster> <7e0fb38c1002081001y111bd0a2p40a8a896b0668d13@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7e0fb38c1002081001y111bd0a2p40a8a896b0668d13@mail.gmail.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265657834 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Feb 08, 2010 at 01:01:46PM -0500, Eric Paris wrote: > On Sun, Jan 24, 2010 at 11:29 PM, Dave Chinner wrote: > > On Mon, Jan 25, 2010 at 03:04:56PM +1100, Stephen Rothwell wrote: > >> Hi Eric, Al, > >> > >> Today's linux-next build (x86_64 allmodconfig) produced these warnings: > > I just switched fsnotify to use const unsigned char * so hopefully > they will shut up tomorrow.... The xfs tree is also about to get a patch to turn off -Wpointer-sign again. If it was up to me we should turn it on for the whole build, we'll just need good way to deal with static initializers and strlen, but some simple wrappers might do it for those. From BATV+6409d81330fd2592ba7a+2360+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 8 13:36:16 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18JaG59155680 for ; Mon, 8 Feb 2010 13:36:16 -0600 X-ASG-Debug-ID: 1265657848-0a20005e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 7149F1BA617; Mon, 8 Feb 2010 11:37:28 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id MoML3tJiCyUiLoFT; Mon, 08 Feb 2010 11:37:28 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeZQE-0002tA-Hx; Mon, 08 Feb 2010 19:37:26 +0000 Date: Mon, 8 Feb 2010 14:37:26 -0500 From: Christoph Hellwig To: Julia Lawall Cc: Alex Elder , xfs-masters@oss.sgi.com, xfs@oss.sgi.com, linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org X-ASG-Orig-Subj: Re: [PATCH 11/11] fs/xfs: Correct NULL test Subject: Re: [PATCH 11/11] fs/xfs: Correct NULL test Message-ID: <20100208193726.GB9527@infradead.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265657849 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Sat, Feb 06, 2010 at 09:45:15AM +0100, Julia Lawall wrote: > diff --git a/fs/xfs/quota/xfs_qm.c b/fs/xfs/quota/xfs_qm.c > index 11cfd82..4318cd5 100644 > --- a/fs/xfs/quota/xfs_qm.c > +++ b/fs/xfs/quota/xfs_qm.c > @@ -123,7 +123,7 @@ xfs_Gqm_init(void) > goto out; > > gdqhash = kmem_zalloc_large(hsize); > - if (!udqhash) > + if (!gdqhash) > goto out_free_udqhash; Thanks, this looks correct, Reviewed-by: Christoph Hellwig From BATV+6409d81330fd2592ba7a+2360+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 8 13:39:47 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_75 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18Jdkp0155840 for ; Mon, 8 Feb 2010 13:39:47 -0600 X-ASG-Debug-ID: 1265658059-0d4700550000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 76A271BA84B for ; Mon, 8 Feb 2010 11:40:59 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id H0t8Q6mX9vj2YEDc for ; Mon, 08 Feb 2010 11:40:59 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeZTf-0003hs-0L; Mon, 08 Feb 2010 19:40:59 +0000 Date: Mon, 8 Feb 2010 14:40:58 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs-oss , Theodore Tso X-ASG-Orig-Subj: Re: [PATCH] xfstests: fix up fs_perms test used by 126 Subject: Re: [PATCH] xfstests: fix up fs_perms test used by 126 Message-ID: <20100208194058.GC9527@infradead.org> References: <4B6C4E81.6060201@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B6C4E81.6060201@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265658059 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Fri, Feb 05, 2010 at 10:59:45AM -0600, Eric Sandeen wrote: > @@ -53,7 +53,8 @@ int main( int argc, char *argv[]) { > cgroupId = atoi(argv[3]); > userId = atoi(argv[4]); > groupId = atoi(argv[5]); > - fperm[0] = *argv[6]; > + strncpy(fperm, argv[6], 3); > + fperm[2] = '\0'; This still looks rather weird to me. What's the reason for copying the string into a fixed length buffer? Why not leave fperm as a pointer to the original argument? The rest of the patch looks fine, but a clean up pass on the whole file wouldn't hurt either, it's a grotty mess.. From BATV+6409d81330fd2592ba7a+2360+infradead.org+hch@bombadil.srs.infradead.org Mon Feb 8 13:41:15 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18JfF31155939 for ; Mon, 8 Feb 2010 13:41:15 -0600 X-ASG-Debug-ID: 1265658147-630800a00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E9E9D1388AA2 for ; Mon, 8 Feb 2010 11:42:27 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id Uu0wuNsPLFp9FWca for ; Mon, 08 Feb 2010 11:42:27 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeZV4-0003tb-1U; Mon, 08 Feb 2010 19:42:26 +0000 Date: Mon, 8 Feb 2010 14:42:26 -0500 From: Christoph Hellwig To: Patrick Schreurs Cc: Dave Chinner , Christoph Hellwig , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Message-ID: <20100208194226.GD9527@infradead.org> References: <4AF0422D.1070104@news-service.com> <20091114162126.GB17658@infradead.org> <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B6706CE.1020207@news-service.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265658147 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Mon, Feb 01, 2010 at 05:52:30PM +0100, Patrick Schreurs wrote: > Hello Dave, > > I'm afraid we had another crash a few days ago. Have a look at the > screenshot. Can you make anything out of it? This server is running > 2.6.32.3 with your inode-reclaim patch applied and XFS_DEBUG still > enabled. Just wondering, which set of patches is this exactly? From sandeen@sandeen.net Mon Feb 8 13:46:05 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_75 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18Jk4XN156158 for ; Mon, 8 Feb 2010 13:46:04 -0600 X-ASG-Debug-ID: 1265658436-1c0e00c00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 513CF1CB1F60 for ; Mon, 8 Feb 2010 11:47:17 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id GuW9fyaRImZsJRMb for ; Mon, 08 Feb 2010 11:47:17 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id BF6DE12AEBE1; Mon, 8 Feb 2010 13:47:16 -0600 (CST) Message-ID: <4B706A44.4000804@sandeen.net> Date: Mon, 08 Feb 2010 13:47:16 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: xfs-oss , Theodore Tso X-ASG-Orig-Subj: Re: [PATCH] xfstests: fix up fs_perms test used by 126 Subject: Re: [PATCH] xfstests: fix up fs_perms test used by 126 References: <4B6C4E81.6060201@sandeen.net> <20100208194058.GC9527@infradead.org> In-Reply-To: <20100208194058.GC9527@infradead.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265658437 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22014 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Christoph Hellwig wrote: > On Fri, Feb 05, 2010 at 10:59:45AM -0600, Eric Sandeen wrote: >> @@ -53,7 +53,8 @@ int main( int argc, char *argv[]) { >> cgroupId = atoi(argv[3]); >> userId = atoi(argv[4]); >> groupId = atoi(argv[5]); >> - fperm[0] = *argv[6]; >> + strncpy(fperm, argv[6], 3); >> + fperm[2] = '\0'; > > This still looks rather weird to me. What's the reason for copying > the string into a fixed length buffer? Why not leave fperm as a pointer > to the original argument? eh that's probably better, I guess I was just thinking copy based on how it was before. (which copied the char, right, it didn't assign a pointer, unless I'm short on coffee today...) OTOH fopen only takes 2 chars anyway. But probably no reason to truncate what was given, just fail if it's something that's wrong... -Eric > The rest of the patch looks fine, but a clean up pass on the whole > file wouldn't hurt either, it's a grotty mess.. > From aelder@oss.sgi.com Mon Feb 8 14:43:09 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_72 autolearn=no version=3.3.0-rupdated Received: from oss.sgi.com (localhost [127.0.0.1]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18Kh9fH159780 for ; Mon, 8 Feb 2010 14:43:09 -0600 Received: (from aelder@localhost) by oss.sgi.com (8.14.3/8.14.3/Submit) id o18Kh9Wd159746; Mon, 8 Feb 2010 14:43:09 -0600 Date: Mon, 8 Feb 2010 14:43:09 -0600 Message-Id: <201002082043.o18Kh9Wd159746@oss.sgi.com> From: xfs@oss.sgi.com To: xfs@oss.sgi.com Subject: [XFS updates] XFS development tree branch, master, updated. v2.6.33-rc4-46-g388f1f0 X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 9b00f30762fe9f914eb6e03057a616ed63a4e8ca X-Git-Newrev: 388f1f0c346b533b06d8bc792f7204ebc3e4b7da This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "XFS development tree". The branch, master has been updated 388f1f0 xfs: turn off sign warnings cbe132a xfs: don't hold onto reserved blocks on remount,ro from 9b00f30762fe9f914eb6e03057a616ed63a4e8ca (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 388f1f0c346b533b06d8bc792f7204ebc3e4b7da Author: Dave Chinner Date: Tue Jan 26 15:10:15 2010 +1100 xfs: turn off sign warnings Because they cause warnings in static inline functions conditionally compiled into XFS from the VFS (e.g. fsnotify). Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig commit cbe132a8bdcff0f9afd9060948fb50597c7400b8 Author: Dave Chinner Date: Tue Jan 26 15:08:49 2010 +1100 xfs: don't hold onto reserved blocks on remount,ro If we hold onto reserved blocks when doing a remount,ro we end up writing the blocks used count to disk that includes the reserved blocks. Reserved blocks are not actually used, so this results in the values in the superblock being incorrect. Hence if we run xfs_check or xfs_repair -n while the filesystem is mounted remount,ro we end up with an inconsistent filesystem being reported. Also, running xfs_copy on the remount,ro filesystem will result in an inconsistent image being generated. To fix this, unreserve the blocks when doing the remount,ro, and reserved them again on remount,rw. This way a remount,ro filesystem will appear consistent on disk to all utilities. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig ----------------------------------------------------------------------- Summary of changes: fs/xfs/Makefile | 2 +- fs/xfs/linux-2.6/xfs_super.c | 28 ++++++++++++++++++++++++++++ fs/xfs/xfs_mount.h | 1 + 3 files changed, 30 insertions(+), 1 deletions(-) hooks/post-receive -- XFS development tree From stevecs@chaven.com Mon Feb 8 17:44:12 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.9 required=5.0 tests=AWL,BAYES_20,FH_DATE_PAST_20XX, HTML_MESSAGE,J_CHICKENPOX_43 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o18NiBcf170140 for ; Mon, 8 Feb 2010 17:44:12 -0600 X-ASG-Debug-ID: 1265672723-211101e00000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from omr10.networksolutionsemail.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 40F481BB5D0 for ; Mon, 8 Feb 2010 15:45:23 -0800 (PST) Received: from omr10.networksolutionsemail.com (omr10.networksolutionsemail.com [205.178.146.60]) by cuda.sgi.com with ESMTP id IqtmNexhuo6u5D9e for ; Mon, 08 Feb 2010 15:45:23 -0800 (PST) Received: from mail.networksolutionsemail.com (mail.networksolutionsemail.com [205.178.146.50]) by omr10.networksolutionsemail.com (8.13.6/8.13.6) with SMTP id o18NjNIU017440 for ; Mon, 8 Feb 2010 18:45:23 -0500 Received: (qmail 9029 invoked by uid 78); 8 Feb 2010 23:45:23 -0000 Received: from unknown (HELO ?127.0.0.1?) (stevecs@chaven.com@75.26.225.171) by ns-omr3.lb.hosting.dc2.netsol.com with SMTP; 8 Feb 2010 23:45:23 -0000 Message-ID: <4B70A213.1020002@chaven.com> Date: Mon, 08 Feb 2010 17:45:23 -0600 From: Steve Costaras User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: format and xfs_info don't match Subject: Re: format and xfs_info don't match References: <20100208144035.GA12922@erebus.underworld.local> In-Reply-To: <20100208144035.GA12922@erebus.underworld.local> Content-Type: multipart/alternative; boundary="------------080700030001050308060402" X-Barracuda-Connect: omr10.networksolutionsemail.com[205.178.146.60] X-Barracuda-Start-Time: 1265672724 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=HTML_MESSAGE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22030 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 HTML_MESSAGE BODY: HTML included in message X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is a multi-part message in MIME format. --------------080700030001050308060402 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I've seen this as well under ubuntu 8.04.3LTS (xfsprogs for that distro is 2.9.4). I grabbed 3.1.0 and use that now which agree. Steve On 02/08/2010 08:40, Cory Coager wrote: > For some reason the format attributes don't match the mounted > attributes. If you look below, lazy-count and attr aren't getting set. > Why is this happening? > > # mkfs.xfs -f -i size=1024,attr=2 -l version=2,size=128m,lazy-count=1 /dev/mapper/360a9800050334c59543455596d625a77 > meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 > agcount=32, agsize=8388744 blks > = sectsz=512 attr=2 > data = bsize=4096 blocks=268439808, imaxpct=25 > = sunit=0 swidth=0 blks, unwritten=1 > naming =version 2 bsize=4096 > log =internal log bsize=4096 blocks=32768, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > > # mount -t xfs -o noatime,nodiratime,nobarrier > /dev/mapper/360a9800050334c59543455596d625a77 /mnt/new-homes/ > > # xfs_info /mnt/new-homes/ > meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024 > agcount=32, agsize=8388744 blks > = sectsz=512 attr=0 > data = bsize=4096 blocks=268439808, imaxpct=25 > = sunit=0 swidth=0 blks, unwritten=1 > naming =version 2 bsize=4096 > log =internal bsize=4096 blocks=32768, version=2 > = sectsz=512 sunit=0 blks, lazy-count=0 > realtime =none extsz=4096 blocks=0, rtextents=0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > > > --------------080700030001050308060402 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
I've seen this as well under ubuntu 8.04.3LTS (xfsprogs for that distro is 2.9.4).   I grabbed 3.1.0 and use that now which agree.

Steve


On 02/08/2010 08:40, Cory Coager wrote: 
For some reason the format attributes don't match the mounted 
attributes.  If you look below, lazy-count and attr aren't getting set.  
Why is this happening?

# mkfs.xfs -f -i size=1024,attr=2 -l version=2,size=128m,lazy-count=1 /dev/mapper/360a9800050334c59543455596d625a77
meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024   
agcount=32, agsize=8388744 blks
        =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=268439808, imaxpct=25
        =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal log           bsize=4096   blocks=32768, version=2
        =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

# mount -t xfs -o noatime,nodiratime,nobarrier 
/dev/mapper/360a9800050334c59543455596d625a77 /mnt/new-homes/

# xfs_info /mnt/new-homes/
meta-data=/dev/mapper/360a9800050334c59543455596d625a77 isize=1024   
agcount=32, agsize=8388744 blks
        =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=268439808, imaxpct=25
        =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal               bsize=4096   blocks=32768, version=2
        =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
--------------080700030001050308060402-- From aelder@oss.sgi.com Mon Feb 8 18:56:35 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from oss.sgi.com (localhost [127.0.0.1]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o190uZxf174108 for ; Mon, 8 Feb 2010 18:56:35 -0600 Received: (from aelder@localhost) by oss.sgi.com (8.14.3/8.14.3/Submit) id o190uXHC174076; Mon, 8 Feb 2010 18:56:33 -0600 Date: Mon, 8 Feb 2010 18:56:33 -0600 Message-Id: <201002090056.o190uXHC174076@oss.sgi.com> From: xfs@oss.sgi.com To: xfs@oss.sgi.com Subject: [XFS updates] XFS development tree branch, master, updated. v2.6.33-rc4-47-gd5db0f9 X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 388f1f0c346b533b06d8bc792f7204ebc3e4b7da X-Git-Newrev: d5db0f97fbbeff11c88dec1aaf1536a975afbaeb This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "XFS development tree". The branch, master has been updated d5db0f9 xfs: more reserved blocks fixups from 388f1f0c346b533b06d8bc792f7204ebc3e4b7da (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit d5db0f97fbbeff11c88dec1aaf1536a975afbaeb Author: Eric Sandeen Date: Fri Feb 5 22:59:53 2010 +0000 xfs: more reserved blocks fixups This mangles the reserved blocks counts a little more. 1) add a helper function for the default reserved count 2) add helper functions to save/restore counts on ro/rw 3) save/restore reserved blocks on freeze/thaw 4) disallow changing reserved count while readonly V2: changed field name to match Dave's changes Signed-off-by: Eric Sandeen Signed-off-by: Alex Elder ----------------------------------------------------------------------- Summary of changes: fs/xfs/linux-2.6/xfs_ioctl.c | 3 ++ fs/xfs/linux-2.6/xfs_super.c | 51 ++++++++++++++++++++++++++++++----------- fs/xfs/xfs_mount.c | 34 +++++++++++++++++++--------- fs/xfs/xfs_mount.h | 1 + 4 files changed, 64 insertions(+), 25 deletions(-) hooks/post-receive -- XFS development tree From SRS0+WGMz+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:38 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tbBr182487 for ; Mon, 8 Feb 2010 21:55:38 -0600 X-ASG-Debug-ID: 1265687808-287d003e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 9BC031BBE08 for ; Mon, 8 Feb 2010 19:56:48 -0800 (PST) Received: from mail.internode.on.net (bld-mail14.adl6.internode.on.net [150.101.137.99]) by cuda.sgi.com with ESMTP id VG0UqxVfbKfaousX for ; Mon, 08 Feb 2010 19:56:48 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 13236044-1927428 for ; Tue, 09 Feb 2010 14:26:47 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDS-0000O6-7P for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:46 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002SQ-H5 for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 5/9] xfs: Use delay write promotion for dquot flushing Subject: [PATCH 5/9] xfs: Use delay write promotion for dquot flushing Date: Tue, 9 Feb 2010 14:56:38 +1100 Message-Id: <1265687802-23043-6-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail14.adl6.internode.on.net[150.101.137.99] X-Barracuda-Start-Time: 1265687810 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean xfs_qm_dqflock_pushbuf_wait() does a very similar trick to item pushing used to do to flush out delayed write dquot buffers. Change it to use the new promotion method rather than an async flush. Also, xfs_qm_dqflock_pushbuf_wait() can return without the flush lock held, yet the callers make the assumption that after this call the flush lock is held. Always return with the flush lock held. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/quota/xfs_dquot.c | 25 ++++++++++--------------- 1 files changed, 10 insertions(+), 15 deletions(-) diff --git a/fs/xfs/quota/xfs_dquot.c b/fs/xfs/quota/xfs_dquot.c index f9baeed..1620a56 100644 --- a/fs/xfs/quota/xfs_dquot.c +++ b/fs/xfs/quota/xfs_dquot.c @@ -1528,21 +1528,16 @@ xfs_qm_dqflock_pushbuf_wait( */ bp = xfs_incore(dqp->q_mount->m_ddev_targp, dqp->q_blkno, XFS_QI_DQCHUNKLEN(dqp->q_mount), XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - int error; - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(dqp->q_mount, 0); - error = xfs_bawrite(dqp->q_mount, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, dqp->q_mount, - "xfs_qm_dqflock_pushbuf_wait: " - "pushbuf error %d on dqp %p, bp %p", - error, dqp, bp); - } else { - xfs_buf_relse(bp); - } + if (!bp) + goto out_lock; + + if (XFS_BUF_ISDELAYWRITE(bp)) { + if (XFS_BUF_ISPINNED(bp)) + xfs_log_force(dqp->q_mount, 0); + xfs_buf_delwri_promote(bp); + wake_up_process(bp->b_target->bt_task); } + xfs_buf_relse(bp); +out_lock: xfs_dqflock(dqp); } -- 1.6.5 From SRS0+8Wp3+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:38 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tbMJ182485 for ; Mon, 8 Feb 2010 21:55:37 -0600 X-ASG-Debug-ID: 1265687808-14ef03060000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6248A1CB5F52 for ; Mon, 8 Feb 2010 19:56:48 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id Td93y6E8GVye24vs for ; Mon, 08 Feb 2010 19:56:48 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12912035-1927428 for ; Tue, 09 Feb 2010 14:26:47 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDS-0000O0-0t for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:46 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002SC-9K for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 1/9] xfs: Make inode reclaim states explicit Subject: [PATCH 1/9] xfs: Make inode reclaim states explicit Date: Tue, 9 Feb 2010 14:56:34 +1100 Message-Id: <1265687802-23043-2-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265687810 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean A.K.A.: don't rely on xfs_iflush() return value in reclaim We have gradually been moving checks out of the reclaim code because they are duplicated in xfs_iflush(). We've had a history of problems in this area, and many of them stem from the overloading of the return values from xfs_iflush() and interaction with inode flush locking to determine if the inode is safe to reclaim. With the desire to move to delayed write flushing of inodes and non-blocking inode tree reclaim walks, the overloading of the return value of xfs_iflush makes it very difficult to determine the correct thing to do next. This patch explicitly re-adds the checks to the inode reclaim code, removing the reliance on the return value of xfs_iflush() to determine what to do next. It also means that we can clearly document all the inode states that reclaim must handle and hence we can easily see that we handled all the necessary cases. This also removes the need for the xfs_inode_clean() check in xfs_iflush() as all callers now check this first (safely). Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_sync.c | 81 +++++++++++++++++++++++++++++++++---------- fs/xfs/xfs_inode.c | 11 +----- fs/xfs/xfs_inode.h | 1 + 3 files changed, 64 insertions(+), 29 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c index c9b863e..525260c 100644 --- a/fs/xfs/linux-2.6/xfs_sync.c +++ b/fs/xfs/linux-2.6/xfs_sync.c @@ -706,12 +706,43 @@ __xfs_inode_clear_reclaim_tag( XFS_INO_TO_AGINO(mp, ip->i_ino), XFS_ICI_RECLAIM_TAG); } +/* + * Inodes in different states need to be treated differently, and the return + * value of xfs_iflush is not sufficient to get this right. The following table + * lists the inode states and the reclaim actions necessary for non-blocking + * reclaim: + * + * + * inode state iflush ret required action + * --------------- ---------- --------------- + * bad - reclaim + * shutdown EIO unpin and reclaim + * clean, unpinned 0 reclaim + * stale, unpinned 0 reclaim + * clean, pinned(*) 0 unpin and reclaim + * stale, pinned 0 unpin and reclaim + * dirty, async 0 block on flush lock, reclaim + * dirty, sync flush 0 block on flush lock, reclaim + * + * (*) dgc: I don't think the clean, pinned state is possible but it gets + * handled anyway given the order of checks implemented. + * + * Hence the order of actions after gaining the locks should be: + * bad => reclaim + * shutdown => unpin and reclaim + * pinned => unpin + * stale => reclaim + * clean => reclaim + * dirty => flush, wait and reclaim + */ STATIC int xfs_reclaim_inode( struct xfs_inode *ip, struct xfs_perag *pag, int sync_mode) { + int error; + /* * The radix tree lock here protects a thread in xfs_iget from racing * with us starting reclaim on the inode. Once we have the @@ -729,30 +760,42 @@ xfs_reclaim_inode( spin_unlock(&ip->i_flags_lock); write_unlock(&pag->pag_ici_lock); - /* - * If the inode is still dirty, then flush it out. If the inode - * is not in the AIL, then it will be OK to flush it delwri as - * long as xfs_iflush() does not keep any references to the inode. - * We leave that decision up to xfs_iflush() since it has the - * knowledge of whether it's OK to simply do a delwri flush of - * the inode or whether we need to wait until the inode is - * pulled from the AIL. - * We get the flush lock regardless, though, just to make sure - * we don't free it while it is being flushed. - */ xfs_ilock(ip, XFS_ILOCK_EXCL); xfs_iflock(ip); - /* - * In the case of a forced shutdown we rely on xfs_iflush() to - * wait for the inode to be unpinned before returning an error. - */ - if (!is_bad_inode(VFS_I(ip)) && xfs_iflush(ip, sync_mode) == 0) { - /* synchronize with xfs_iflush_done */ - xfs_iflock(ip); - xfs_ifunlock(ip); + if (is_bad_inode(VFS_I(ip))) + goto reclaim; + if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { + xfs_iunpin_wait(ip); + goto reclaim; + } + if (xfs_ipincount(ip)) + xfs_iunpin_wait(ip); + if (xfs_iflags_test(ip, XFS_ISTALE)) + goto reclaim; + if (xfs_inode_clean(ip)) + goto reclaim; + + /* Now we have an inode that needs flushing */ + error = xfs_iflush(ip, sync_mode); + if (!error) { + switch(sync_mode) { + case XFS_IFLUSH_DELWRI_ELSE_ASYNC: + case XFS_IFLUSH_DELWRI: + case XFS_IFLUSH_ASYNC: + case XFS_IFLUSH_DELWRI_ELSE_SYNC: + case XFS_IFLUSH_SYNC: + /* IO issued, synchronise with IO completion */ + xfs_iflock(ip); + break; + default: + ASSERT(0); + break; + } } +reclaim: + xfs_ifunlock(ip); xfs_iunlock(ip, XFS_ILOCK_EXCL); xfs_ireclaim(ip); return 0; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index d0d1b5a..8d0666d 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2493,7 +2493,7 @@ __xfs_iunpin_wait( wait_event(ip->i_ipin_wait, (atomic_read(&ip->i_pincount) == 0)); } -static inline void +void xfs_iunpin_wait( xfs_inode_t *ip) { @@ -2849,15 +2849,6 @@ xfs_iflush( mp = ip->i_mount; /* - * If the inode isn't dirty, then just release the inode flush lock and - * do nothing. - */ - if (xfs_inode_clean(ip)) { - xfs_ifunlock(ip); - return 0; - } - - /* * We can't flush the inode until it is unpinned, so wait for it if we * are allowed to block. We know noone new can pin it, because we are * holding the inode lock shared and you need to hold it exclusively to diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index ec1f28c..8b618ea 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -483,6 +483,7 @@ int xfs_iunlink(struct xfs_trans *, xfs_inode_t *); void xfs_iext_realloc(xfs_inode_t *, int, int); void xfs_ipin(xfs_inode_t *); void xfs_iunpin(xfs_inode_t *); +void xfs_iunpin_wait(xfs_inode_t *); int xfs_iflush(xfs_inode_t *, uint); void xfs_ichgtime(xfs_inode_t *, int); void xfs_lock_inodes(xfs_inode_t **, int, uint); -- 1.6.5 From SRS0+8Wp3+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:38 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_CHICKENPOX_66 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tcH6182489 for ; Mon, 8 Feb 2010 21:55:38 -0600 X-ASG-Debug-ID: 1265687808-608001690000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EEF01138A717 for ; Mon, 8 Feb 2010 19:56:49 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id WkMEdYDE1bcw4UOG for ; Mon, 08 Feb 2010 19:56:49 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12912039-1927428 for ; Tue, 09 Feb 2010 14:26:48 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDS-0000O3-5b for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:46 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002SL-FI for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 4/9] xfs: Sort delayed write buffers before dispatch Subject: [PATCH 4/9] xfs: Sort delayed write buffers before dispatch Date: Tue, 9 Feb 2010 14:56:37 +1100 Message-Id: <1265687802-23043-5-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265687810 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22047 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Currently when the xfsbufd writes delayed write buffers, it pushes them to disk in the order they come off the delayed write list. If there are lots of buffers Ñ•pread widely over the disk, this results in overwhelming the elevator sort queues in the block layer and we end up losing the posibility of merging adjacent buffers to minimise the number of IOs. Use the new generic list_sort function to sort the delwri dispatch queue before issue to ensure that the buffers are pushed in the most friendly order possible to the lower layers. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_buf.c | 87 ++++++++++++++++++++++++++++++-------------- 1 files changed, 60 insertions(+), 27 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index b306265..4556a4c 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "xfs_sb.h" #include "xfs_inum.h" @@ -1877,14 +1878,42 @@ xfs_buf_delwri_split( } +/* + * Compare function is more complex than it needs to be because + * the return value is only 32 bits and we are doing comparisons + * on 64 bit values + */ +static int +xfs_buf_cmp( + void *priv, + struct list_head *a, + struct list_head *b) +{ + struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list); + struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list); + xfs_daddr_t diff; + + diff = ap->b_bn - bp->b_bn; + if (diff < 0) + return -1; + if (diff > 0) + return 1; + return 0; +} + +void +xfs_buf_delwri_sort( + xfs_buftarg_t *target, + struct list_head *list) +{ + list_sort(NULL, list, xfs_buf_cmp); +} + STATIC int xfsbufd( void *data) { - struct list_head tmp; - xfs_buftarg_t *target = (xfs_buftarg_t *)data; - int count; - xfs_buf_t *bp; + xfs_buftarg_t *target = (xfs_buftarg_t *)data; current->flags |= PF_MEMALLOC; @@ -1893,6 +1922,8 @@ xfsbufd( do { long age = xfs_buf_age_centisecs * msecs_to_jiffies(10); long tout = xfs_buf_timer_centisecs * msecs_to_jiffies(10); + int count = 0; + struct list_head tmp; if (unlikely(freezing(current))) { set_bit(XBT_FORCE_SLEEP, &target->bt_flags); @@ -1907,11 +1938,10 @@ xfsbufd( schedule_timeout_interruptible(tout); xfs_buf_delwri_split(target, &tmp, age); - count = 0; + list_sort(NULL, &tmp, xfs_buf_cmp); while (!list_empty(&tmp)) { - bp = list_entry(tmp.next, xfs_buf_t, b_list); - ASSERT(target == bp->b_target); - + struct xfs_buf *bp; + bp = list_first_entry(&tmp, struct xfs_buf, b_list); list_del_init(&bp->b_list); xfs_buf_iostrategy(bp); count++; @@ -1937,42 +1967,45 @@ xfs_flush_buftarg( xfs_buftarg_t *target, int wait) { - struct list_head tmp; - xfs_buf_t *bp, *n; + xfs_buf_t *bp; int pincount = 0; + LIST_HEAD(tmp_list); + LIST_HEAD(wait_list); xfs_buf_runall_queues(xfsconvertd_workqueue); xfs_buf_runall_queues(xfsdatad_workqueue); xfs_buf_runall_queues(xfslogd_workqueue); set_bit(XBT_FORCE_FLUSH, &target->bt_flags); - pincount = xfs_buf_delwri_split(target, &tmp, 0); + pincount = xfs_buf_delwri_split(target, &tmp_list, 0); /* - * Dropped the delayed write list lock, now walk the temporary list + * Dropped the delayed write list lock, now walk the temporary list. + * All I/O is issued async and then if we need to wait for completion + * we do that after issuing all the IO. */ - list_for_each_entry_safe(bp, n, &tmp, b_list) { + list_sort(NULL, &tmp_list, xfs_buf_cmp); + while (!list_empty(&tmp_list)) { + bp = list_first_entry(&tmp_list, struct xfs_buf, b_list); ASSERT(target == bp->b_target); - if (wait) + list_del_init(&bp->b_list); + if (wait) { bp->b_flags &= ~XBF_ASYNC; - else - list_del_init(&bp->b_list); - + list_add(&bp->b_list, &wait_list); + } xfs_buf_iostrategy(bp); } - if (wait) + if (wait) { + /* Expedite and wait for IO to complete. */ blk_run_address_space(target->bt_mapping); + while (!list_empty(&wait_list)) { + bp = list_first_entry(&wait_list, struct xfs_buf, b_list); - /* - * Remaining list items must be flushed before returning - */ - while (!list_empty(&tmp)) { - bp = list_entry(tmp.next, xfs_buf_t, b_list); - - list_del_init(&bp->b_list); - xfs_iowait(bp); - xfs_buf_relse(bp); + list_del_init(&bp->b_list); + xfs_iowait(bp); + xfs_buf_relse(bp); + } } return pincount; -- 1.6.5 From SRS0+Ag99+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:43 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193thNK182530 for ; Mon, 8 Feb 2010 21:55:43 -0600 X-ASG-Debug-ID: 1265687813-2f1001670000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 85B851CB5F56 for ; Mon, 8 Feb 2010 19:56:54 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id 1l8JYxSBGdWAbbVR for ; Mon, 08 Feb 2010 19:56:54 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 13074385-1927428 for ; Tue, 09 Feb 2010 14:26:52 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDX-0000O2-4f for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:51 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002SI-DT for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 3/9] xfs: Don't issue buffer IO direct from AIL push V2 Subject: [PATCH 3/9] xfs: Don't issue buffer IO direct from AIL push V2 Date: Tue, 9 Feb 2010 14:56:36 +1100 Message-Id: <1265687802-23043-4-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1265687815 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean All buffers logged into the AIL are marked as delayed write. When the AIL needs to push the buffer out, it issues an async write of the buffer. This means that IO patterns are dependent on the order of buffers in the AIL. Instead of flushing the buffer, promote the buffer in the delayed write list so that the next time the xfsbufd is run the buffer will be flushed by the xfsbufd. Return the state to the xfsaild that the buffer was promoted so that the xfsaild knows that it needs to cause the xfsbufd to run to flush the buffers that were promoted. Using the xfsbufd for issuing the IO allows us to dispatch all buffer IO from the one queue. This means that we can make much more enlightened decisions on what order to flush buffers to disk as we don't have multiple places issuing IO. Optimisations to xfsbufd will be in a future patch. Version 2 - kill XFS_ITEM_FLUSHING as it is now unused. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_buf.c | 29 ++++++++++++ fs/xfs/linux-2.6/xfs_buf.h | 2 + fs/xfs/linux-2.6/xfs_trace.h | 1 + fs/xfs/quota/xfs_dquot_item.c | 85 +++++------------------------------ fs/xfs/quota/xfs_dquot_item.h | 4 -- fs/xfs/xfs_buf_item.c | 64 +++++++++++++++------------ fs/xfs/xfs_inode_item.c | 98 ++++++---------------------------------- fs/xfs/xfs_inode_item.h | 6 --- fs/xfs/xfs_trans.h | 3 +- fs/xfs/xfs_trans_ail.c | 13 +++--- 10 files changed, 102 insertions(+), 203 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index 44e20e5..b306265 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -1778,6 +1778,35 @@ xfs_buf_delwri_dequeue( trace_xfs_buf_delwri_dequeue(bp, _RET_IP_); } +/* + * If a delwri buffer needs to be pushed before it has aged out, then promote + * it to the head of the delwri queue so that it will be flushed on the next + * xfsbufd run. We do this by resetting the queuetime of the buffer to be older + * than the age currently needed to flush the buffer. Hence the next time the + * xfsbufd sees it is guaranteed to be considered old enough to flush. + */ +void +xfs_buf_delwri_promote( + struct xfs_buf *bp) +{ + struct xfs_buftarg *btp = bp->b_target; + long age = xfs_buf_age_centisecs * msecs_to_jiffies(10) + 1; + + ASSERT(bp->b_flags & XBF_DELWRI); + ASSERT(bp->b_flags & _XBF_DELWRI_Q); + + /* + * Check the buffer age before locking the delayed write queue as we + * don't need to promote buffers that are already past the flush age. + */ + if (bp->b_queuetime < jiffies - age) + return; + bp->b_queuetime = jiffies - age; + spin_lock(&btp->bt_delwrite_lock); + list_move(&bp->b_list, &btp->bt_delwrite_queue); + spin_unlock(&btp->bt_delwrite_lock); +} + STATIC void xfs_buf_runall_queues( struct workqueue_struct *queue) diff --git a/fs/xfs/linux-2.6/xfs_buf.h b/fs/xfs/linux-2.6/xfs_buf.h index ea8c198..be45e8c 100644 --- a/fs/xfs/linux-2.6/xfs_buf.h +++ b/fs/xfs/linux-2.6/xfs_buf.h @@ -266,6 +266,7 @@ extern int xfs_buf_ispin(xfs_buf_t *); /* Delayed Write Buffer Routines */ extern void xfs_buf_delwri_dequeue(xfs_buf_t *); +extern void xfs_buf_delwri_promote(xfs_buf_t *); /* Buffer Daemon Setup Routines */ extern int xfs_buf_init(void); @@ -395,6 +396,7 @@ extern void xfs_free_buftarg(struct xfs_mount *, struct xfs_buftarg *); extern void xfs_wait_buftarg(xfs_buftarg_t *); extern int xfs_setsize_buftarg(xfs_buftarg_t *, unsigned int, unsigned int); extern int xfs_flush_buftarg(xfs_buftarg_t *, int); + #ifdef CONFIG_KDB_MODULES extern struct list_head *xfs_get_buftarg_list(void); #endif diff --git a/fs/xfs/linux-2.6/xfs_trace.h b/fs/xfs/linux-2.6/xfs_trace.h index 1bb09e7..a4574dc 100644 --- a/fs/xfs/linux-2.6/xfs_trace.h +++ b/fs/xfs/linux-2.6/xfs_trace.h @@ -483,6 +483,7 @@ DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock_stale); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_committed); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_push); +DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pushbuf); DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf); DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf_recur); DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb); diff --git a/fs/xfs/quota/xfs_dquot_item.c b/fs/xfs/quota/xfs_dquot_item.c index 1b56437..dda0fb0 100644 --- a/fs/xfs/quota/xfs_dquot_item.c +++ b/fs/xfs/quota/xfs_dquot_item.c @@ -212,66 +212,31 @@ xfs_qm_dquot_logitem_pushbuf( xfs_dquot_t *dqp; xfs_mount_t *mp; xfs_buf_t *bp; - uint dopush; dqp = qip->qli_dquot; ASSERT(XFS_DQ_IS_LOCKED(dqp)); /* - * The qli_pushbuf_flag keeps others from - * trying to duplicate our effort. - */ - ASSERT(qip->qli_pushbuf_flag != 0); - ASSERT(qip->qli_push_owner == current_pid()); - - /* * If flushlock isn't locked anymore, chances are that the * inode flush completed and the inode was taken off the AIL. * So, just get out. */ if (completion_done(&dqp->q_flush) || ((qip->qli_item.li_flags & XFS_LI_IN_AIL) == 0)) { - qip->qli_pushbuf_flag = 0; xfs_dqunlock(dqp); return; } mp = dqp->q_mount; bp = xfs_incore(mp->m_ddev_targp, qip->qli_format.qlf_blkno, XFS_QI_DQCHUNKLEN(mp), XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - dopush = ((qip->qli_item.li_flags & XFS_LI_IN_AIL) && - !completion_done(&dqp->q_flush)); - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(mp, 0); - - if (dopush) { - int error; -#ifdef XFSRACEDEBUG - delay_for_intr(); - delay(300); -#endif - error = xfs_bawrite(mp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, mp, - "xfs_qm_dquot_logitem_pushbuf: pushbuf error %d on qip %p, bp %p", - error, qip, bp); - } else { - xfs_buf_relse(bp); - } - } else { - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); - xfs_buf_relse(bp); - } + xfs_dqunlock(dqp); + if (!bp) return; - } + if (XFS_BUF_ISDELAYWRITE(bp)) + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); + return; - qip->qli_pushbuf_flag = 0; - xfs_dqunlock(dqp); } /* @@ -289,50 +254,24 @@ xfs_qm_dquot_logitem_trylock( xfs_dq_logitem_t *qip) { xfs_dquot_t *dqp; - uint retval; dqp = qip->qli_dquot; if (atomic_read(&dqp->q_pincount) > 0) - return (XFS_ITEM_PINNED); + return XFS_ITEM_PINNED; if (! xfs_qm_dqlock_nowait(dqp)) - return (XFS_ITEM_LOCKED); + return XFS_ITEM_LOCKED; - retval = XFS_ITEM_SUCCESS; if (!xfs_dqflock_nowait(dqp)) { /* - * The dquot is already being flushed. It may have been - * flushed delayed write, however, and we don't want to - * get stuck waiting for that to complete. So, we want to check - * to see if we can lock the dquot's buffer without sleeping. - * If we can and it is marked for delayed write, then we - * hold it and send it out from the push routine. We don't - * want to do that now since we might sleep in the device - * strategy routine. We also don't want to grab the buffer lock - * here because we'd like not to call into the buffer cache - * while holding the AIL lock. - * Make sure to only return PUSHBUF if we set pushbuf_flag - * ourselves. If someone else is doing it then we don't - * want to go to the push routine and duplicate their efforts. + * dquot has already been flushed to the backing buffer, + * leave it locked, pushbuf routine will unlock it. */ - if (qip->qli_pushbuf_flag == 0) { - qip->qli_pushbuf_flag = 1; - ASSERT(qip->qli_format.qlf_blkno == dqp->q_blkno); -#ifdef DEBUG - qip->qli_push_owner = current_pid(); -#endif - /* - * The dquot is left locked. - */ - retval = XFS_ITEM_PUSHBUF; - } else { - retval = XFS_ITEM_FLUSHING; - xfs_dqunlock_nonotify(dqp); - } + return XFS_ITEM_PUSHBUF; } ASSERT(qip->qli_item.li_flags & XFS_LI_IN_AIL); - return (retval); + return XFS_ITEM_SUCCESS; } diff --git a/fs/xfs/quota/xfs_dquot_item.h b/fs/xfs/quota/xfs_dquot_item.h index 5a63253..5acae2a 100644 --- a/fs/xfs/quota/xfs_dquot_item.h +++ b/fs/xfs/quota/xfs_dquot_item.h @@ -27,10 +27,6 @@ typedef struct xfs_dq_logitem { xfs_log_item_t qli_item; /* common portion */ struct xfs_dquot *qli_dquot; /* dquot ptr */ xfs_lsn_t qli_flush_lsn; /* lsn at last flush */ - unsigned short qli_pushbuf_flag; /* 1 bit used in push_ail */ -#ifdef DEBUG - uint64_t qli_push_owner; -#endif xfs_dq_logformat_t qli_format; /* logged structure */ } xfs_dq_logitem_t; diff --git a/fs/xfs/xfs_buf_item.c b/fs/xfs/xfs_buf_item.c index e0a1158..f3c49e6 100644 --- a/fs/xfs/xfs_buf_item.c +++ b/fs/xfs/xfs_buf_item.c @@ -467,8 +467,10 @@ xfs_buf_item_unpin_remove( /* * This is called to attempt to lock the buffer associated with this * buf log item. Don't sleep on the buffer lock. If we can't get - * the lock right away, return 0. If we can get the lock, pull the - * buffer from the free list, mark it busy, and return 1. + * the lock right away, return 0. If we can get the lock, take a + * reference to the buffer. If this is a delayed write buffer that + * needs AIL help to be written back, invoke the pushbuf routine + * rather than the normal success path. */ STATIC uint xfs_buf_item_trylock( @@ -477,24 +479,18 @@ xfs_buf_item_trylock( xfs_buf_t *bp; bp = bip->bli_buf; - - if (XFS_BUF_ISPINNED(bp)) { + if (XFS_BUF_ISPINNED(bp)) return XFS_ITEM_PINNED; - } - - if (!XFS_BUF_CPSEMA(bp)) { + if (!XFS_BUF_CPSEMA(bp)) return XFS_ITEM_LOCKED; - } - /* - * Remove the buffer from the free list. Only do this - * if it's on the free list. Private buffers like the - * superblock buffer are not. - */ + /* take a reference to the buffer. */ XFS_BUF_HOLD(bp); ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); trace_xfs_buf_item_trylock(bip); + if (XFS_BUF_ISDELAYWRITE(bp)) + return XFS_ITEM_PUSHBUF; return XFS_ITEM_SUCCESS; } @@ -626,11 +622,9 @@ xfs_buf_item_committed( } /* - * This is called to asynchronously write the buffer associated with this - * buf log item out to disk. The buffer will already have been locked by - * a successful call to xfs_buf_item_trylock(). If the buffer still has - * B_DELWRI set, then get it going out to disk with a call to bawrite(). - * If not, then just release the buffer. + * The buffer is locked, but is not a delayed write buffer. This happens + * if we race with IO completion and hence we don't want to try to write it + * again. Just release the buffer. */ STATIC void xfs_buf_item_push( @@ -642,17 +636,29 @@ xfs_buf_item_push( trace_xfs_buf_item_push(bip); bp = bip->bli_buf; + ASSERT(!XFS_BUF_ISDELAYWRITE(bp)); + xfs_buf_relse(bp); +} - if (XFS_BUF_ISDELAYWRITE(bp)) { - int error; - error = xfs_bawrite(bip->bli_item.li_mountp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, bip->bli_item.li_mountp, - "xfs_buf_item_push: pushbuf error %d on bip %p, bp %p", - error, bip, bp); - } else { - xfs_buf_relse(bp); - } +/* + * The buffer is locked and is a delayed write buffer. Promote the buffer + * in the delayed write queue as the caller knows that they must invoke + * the xfsbufd to get this buffer written. We have to unlock the buffer + * to allow the xfsbufd to write it, too. + */ +STATIC void +xfs_buf_item_pushbuf( + xfs_buf_log_item_t *bip) +{ + xfs_buf_t *bp; + + ASSERT(!(bip->bli_flags & XFS_BLI_STALE)); + trace_xfs_buf_item_pushbuf(bip); + + bp = bip->bli_buf; + ASSERT(XFS_BUF_ISDELAYWRITE(bp)); + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); } /* ARGSUSED */ @@ -677,7 +683,7 @@ static struct xfs_item_ops xfs_buf_item_ops = { .iop_committed = (xfs_lsn_t(*)(xfs_log_item_t*, xfs_lsn_t)) xfs_buf_item_committed, .iop_push = (void(*)(xfs_log_item_t*))xfs_buf_item_push, - .iop_pushbuf = NULL, + .iop_pushbuf = (void(*)(xfs_log_item_t*))xfs_buf_item_pushbuf, .iop_committing = (void(*)(xfs_log_item_t*, xfs_lsn_t)) xfs_buf_item_committing }; diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index 207553e..d4dc063 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -602,33 +602,20 @@ xfs_inode_item_trylock( if (!xfs_iflock_nowait(ip)) { /* - * If someone else isn't already trying to push the inode - * buffer, we get to do it. + * inode has already been flushed to the backing buffer, + * leave it locked in shared mode, pushbuf routine will + * unlock it. */ - if (iip->ili_pushbuf_flag == 0) { - iip->ili_pushbuf_flag = 1; -#ifdef DEBUG - iip->ili_push_owner = current_pid(); -#endif - /* - * Inode is left locked in shared mode. - * Pushbuf routine gets to unlock it. - */ - return XFS_ITEM_PUSHBUF; - } else { - /* - * We hold the AIL lock, so we must specify the - * NONOTIFY flag so that we won't double trip. - */ - xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); - return XFS_ITEM_FLUSHING; - } - /* NOTREACHED */ + return XFS_ITEM_PUSHBUF; } /* Stale items should force out the iclog */ if (ip->i_flags & XFS_ISTALE) { xfs_ifunlock(ip); + /* + * we hold the AIL lock - notify the unlock routine of this + * so it doesn't try to get the lock again. + */ xfs_iunlock(ip, XFS_ILOCK_SHARED|XFS_IUNLOCK_NONOTIFY); return XFS_ITEM_PINNED; } @@ -746,11 +733,8 @@ xfs_inode_item_committed( * This gets called by xfs_trans_push_ail(), when IOP_TRYLOCK * failed to get the inode flush lock but did get the inode locked SHARED. * Here we're trying to see if the inode buffer is incore, and if so whether it's - * marked delayed write. If that's the case, we'll initiate a bawrite on that - * buffer to expedite the process. - * - * We aren't holding the AIL lock (or the flush lock) when this gets called, - * so it is inherently race-y. + * marked delayed write. If that's the case, we'll promote it and that will + * allow the caller to write the buffer by triggering the xfsbufd to run. */ STATIC void xfs_inode_item_pushbuf( @@ -759,26 +743,16 @@ xfs_inode_item_pushbuf( xfs_inode_t *ip; xfs_mount_t *mp; xfs_buf_t *bp; - uint dopush; ip = iip->ili_inode; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED)); /* - * The ili_pushbuf_flag keeps others from - * trying to duplicate our effort. - */ - ASSERT(iip->ili_pushbuf_flag != 0); - ASSERT(iip->ili_push_owner == current_pid()); - - /* * If a flush is not in progress anymore, chances are that the * inode was taken off the AIL. So, just get out. */ if (completion_done(&ip->i_flush) || ((iip->ili_item.li_flags & XFS_LI_IN_AIL) == 0)) { - iip->ili_pushbuf_flag = 0; xfs_iunlock(ip, XFS_ILOCK_SHARED); return; } @@ -787,53 +761,12 @@ xfs_inode_item_pushbuf( bp = xfs_incore(mp->m_ddev_targp, iip->ili_format.ilf_blkno, iip->ili_format.ilf_len, XBF_TRYLOCK); - if (bp != NULL) { - if (XFS_BUF_ISDELAYWRITE(bp)) { - /* - * We were racing with iflush because we don't hold - * the AIL lock or the flush lock. However, at this point, - * we have the buffer, and we know that it's dirty. - * So, it's possible that iflush raced with us, and - * this item is already taken off the AIL. - * If not, we can flush it async. - */ - dopush = ((iip->ili_item.li_flags & XFS_LI_IN_AIL) && - !completion_done(&ip->i_flush)); - iip->ili_pushbuf_flag = 0; - xfs_iunlock(ip, XFS_ILOCK_SHARED); - - trace_xfs_inode_item_push(bp, _RET_IP_); - - if (XFS_BUF_ISPINNED(bp)) - xfs_log_force(mp, 0); - - if (dopush) { - int error; - error = xfs_bawrite(mp, bp); - if (error) - xfs_fs_cmn_err(CE_WARN, mp, - "xfs_inode_item_pushbuf: pushbuf error %d on iip %p, bp %p", - error, iip, bp); - } else { - xfs_buf_relse(bp); - } - } else { - iip->ili_pushbuf_flag = 0; - xfs_iunlock(ip, XFS_ILOCK_SHARED); - xfs_buf_relse(bp); - } - return; - } - /* - * We have to be careful about resetting pushbuf flag too early (above). - * Even though in theory we can do it as soon as we have the buflock, - * we don't want others to be doing work needlessly. They'll come to - * this function thinking that pushing the buffer is their - * responsibility only to find that the buffer is still locked by - * another doing the same thing - */ - iip->ili_pushbuf_flag = 0; xfs_iunlock(ip, XFS_ILOCK_SHARED); + if (!bp) + return; + if (XFS_BUF_ISDELAYWRITE(bp)) + xfs_buf_delwri_promote(bp); + xfs_buf_relse(bp); return; } @@ -937,7 +870,6 @@ xfs_inode_item_init( /* We have zeroed memory. No need ... iip->ili_extents_buf = NULL; - iip->ili_pushbuf_flag = 0; */ iip->ili_format.ilf_type = XFS_LI_INODE; diff --git a/fs/xfs/xfs_inode_item.h b/fs/xfs/xfs_inode_item.h index cc8df1a..9a46795 100644 --- a/fs/xfs/xfs_inode_item.h +++ b/fs/xfs/xfs_inode_item.h @@ -144,12 +144,6 @@ typedef struct xfs_inode_log_item { data exts */ struct xfs_bmbt_rec *ili_aextents_buf; /* array of logged attr exts */ - unsigned int ili_pushbuf_flag; /* one bit used in push_ail */ - -#ifdef DEBUG - uint64_t ili_push_owner; /* one who sets pushbuf_flag - above gets to push the buf */ -#endif #ifdef XFS_TRANS_DEBUG int ili_root_size; char *ili_orig_root; diff --git a/fs/xfs/xfs_trans.h b/fs/xfs/xfs_trans.h index ca64f33..c93e3a1 100644 --- a/fs/xfs/xfs_trans.h +++ b/fs/xfs/xfs_trans.h @@ -861,8 +861,7 @@ typedef struct xfs_item_ops { #define XFS_ITEM_SUCCESS 0 #define XFS_ITEM_PINNED 1 #define XFS_ITEM_LOCKED 2 -#define XFS_ITEM_FLUSHING 3 -#define XFS_ITEM_PUSHBUF 4 +#define XFS_ITEM_PUSHBUF 3 /* * This structure is used to maintain a list of block ranges that have been diff --git a/fs/xfs/xfs_trans_ail.c b/fs/xfs/xfs_trans_ail.c index d7b1af8..e799824 100644 --- a/fs/xfs/xfs_trans_ail.c +++ b/fs/xfs/xfs_trans_ail.c @@ -253,6 +253,7 @@ xfsaild_push( int flush_log, count, stuck; xfs_mount_t *mp = ailp->xa_mount; struct xfs_ail_cursor *cur = &ailp->xa_cursors; + int push_xfsbufd = 0; spin_lock(&ailp->xa_lock); xfs_trans_ail_cursor_init(ailp, cur); @@ -308,6 +309,7 @@ xfsaild_push( XFS_STATS_INC(xs_push_ail_pushbuf); IOP_PUSHBUF(lip); last_pushed_lsn = lsn; + push_xfsbufd = 1; break; case XFS_ITEM_PINNED: @@ -322,12 +324,6 @@ xfsaild_push( stuck++; break; - case XFS_ITEM_FLUSHING: - XFS_STATS_INC(xs_push_ail_flushing); - last_pushed_lsn = lsn; - stuck++; - break; - default: ASSERT(0); break; @@ -374,6 +370,11 @@ xfsaild_push( xfs_log_force(mp, 0); } + if (push_xfsbufd) { + /* we've got delayed write buffers to flush */ + wake_up_process(mp->m_ddev_targp->bt_task); + } + if (!count) { /* We're past our target or empty, so idle */ last_pushed_lsn = 0; -- 1.6.5 From SRS0+Ag99+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:48 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tlmq182551 for ; Mon, 8 Feb 2010 21:55:47 -0600 X-ASG-Debug-ID: 1265687818-608c01670000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 33C77138A720 for ; Mon, 8 Feb 2010 19:56:58 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id tJy2r5H0OjyOXaht for ; Mon, 08 Feb 2010 19:56:58 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 13074392-1927428 for ; Tue, 09 Feb 2010 14:26:57 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDb-0000Nz-Tk for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:55 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002S9-7Q for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 0/9] Delayed write metadata writeback V5 Subject: [PATCH 0/9] Delayed write metadata writeback V5 Date: Tue, 9 Feb 2010 14:56:33 +1100 Message-Id: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1265687820 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22047 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean While I started with killing async inode writeback, the series has grown. It's not really limited to inode writeback - it touches dquot flushing, changes the way the AIL pushes on buffers, adds xfsbufd sorting for delayed write buffers, adds a real non-blocking mode to inode reclaim and avoids physical inode writeback from the VFS while fixing bugs in handling delayed write inodes. Hence this is more about enabling efficient delayed write metadata than it is able killing async inode writeback. The idea behind this series is to make metadata buffers get written from xfsbufd via the delayed write queue rather than being issued asynchronously from all over the place. To do this, async buffer writeback is almost entirely removed from XFS, replaced instead by delayed writes and a method to expedite flushing of delayed write buffers when required. The result of funnelling all the buffer IO into a single place is that we can more tightly control and therefore optimise the submission of metadata IO. Aggregating the buffers before dispatch allows much better sort efficiency of the buffers as the sort window is not limited to the size of the elevator congestion hysteresis limit. Hence we can approach 100% merge effeciency on large numbers of buffers when dispatched for IO and greatly reduce the amount of seeking metadata writeback causes. The major change is to the inode flushing and reclaim code. Delayed write inodes hold the flush lock for much longer than for async writeback, and hence blocking on the flush lock can cause extremely long latencies without other mechanisms to expedite the release of the flush locks. To prevent needing to flush inodes immediately, all operations are done non-blocking unless synchronous. This required a significant rework of the inode reclaim code, but it greatly simplified other pieces of code (e.g. log item pushing). Version 5 - drop the fsync changes to xfs_fs_write_inode() and the associated locking changes, replace them with a targeted inode logging function from Christoph Hellwig to fix a performance regression on fs_mark -S4 workloads on an SSD. Version 4 - rework inode reclaim checks for better legibility - add warning to reclaim code when delwri flush errors occur - kill XFS_ITEM_FLUSHING now it is not used - clean up sync_mode flags being pushed into xfs_iflush() - kill the now unused xfs_bawrite() function - include Christoph's fsync cache flush fix - rework the inode locking and call to xfs_fsync() when doing synchronous inode writes to close races between the fsync and the background delwri flush afterwards. Version 3 - rework inode reclaim to: - separate it from xfs_iflush return values - provide a non-blocking mode for background operation - apply delwri buffer promotion tricks to dquot flushing - kill unneeded dquot flushing flags, similar to inode flushing flag removal - fix sync inode flush bug when trying to flush delwri inodes Version 2: - use generic list sort function - when unmounting, push the delwri buffers first, then do sync inode reclaim so that reclaim doesn't block for 15 seconds waiting for delwri inode buffers to be aged and written before the inodes can be reclaimed. Alex, the patch series is available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/dgc/xfs for-2.6.34 Christoph Hellwig (2): xfs: remove invalid barrier optimization from xfs_fsync xfs: log changed inodes instead of writing them synchronously Dave Chinner (7): xfs: Make inode reclaim states explicit xfs: Use delayed write for inodes rather than async V2 xfs: Don't issue buffer IO direct from AIL push V2 xfs: Sort delayed write buffers before dispatch xfs: Use delay write promotion for dquot flushing xfs: kill the unused XFS_QMOPT_* flush flags V2 xfs: kill xfs_bawrite fs/xfs/linux-2.6/xfs_buf.c | 135 ++++++++++++++++++++++++++-------------- fs/xfs/linux-2.6/xfs_buf.h | 3 +- fs/xfs/linux-2.6/xfs_super.c | 111 ++++++++++++++++++++++++--------- fs/xfs/linux-2.6/xfs_sync.c | 138 +++++++++++++++++++++++++++++++++------- fs/xfs/linux-2.6/xfs_trace.h | 1 + fs/xfs/quota/xfs_dquot.c | 38 +++++------- fs/xfs/quota/xfs_dquot_item.c | 87 ++++---------------------- fs/xfs/quota/xfs_dquot_item.h | 4 - fs/xfs/quota/xfs_qm.c | 14 ++--- fs/xfs/xfs_buf_item.c | 64 ++++++++++--------- fs/xfs/xfs_inode.c | 86 ++------------------------ fs/xfs/xfs_inode.h | 11 +--- fs/xfs/xfs_inode_item.c | 108 +++++++------------------------- fs/xfs/xfs_inode_item.h | 6 -- fs/xfs/xfs_mount.c | 13 ++++- fs/xfs/xfs_quota.h | 8 +-- fs/xfs/xfs_trans.h | 3 +- fs/xfs/xfs_trans_ail.c | 13 ++-- fs/xfs/xfs_vnodeops.c | 12 +--- 19 files changed, 410 insertions(+), 445 deletions(-) From SRS0+8Wp3+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:48 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tl8u182553 for ; Mon, 8 Feb 2010 21:55:47 -0600 X-ASG-Debug-ID: 1265687818-608101510000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 45453138A722 for ; Mon, 8 Feb 2010 19:56:59 -0800 (PST) Received: from mail.internode.on.net (bld-mail17.adl2.internode.on.net [150.101.137.102]) by cuda.sgi.com with ESMTP id nmBmX06F98tJjEB0 for ; Mon, 08 Feb 2010 19:56:59 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12912059-1927428 for ; Tue, 09 Feb 2010 14:26:58 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDc-0000OQ-9N for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:56 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDb-0002ST-Hv for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 6/9] xfs: kill the unused XFS_QMOPT_* flush flags V2 Subject: [PATCH 6/9] xfs: kill the unused XFS_QMOPT_* flush flags V2 Date: Tue, 9 Feb 2010 14:56:39 +1100 Message-Id: <1265687802-23043-7-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail17.adl2.internode.on.net[150.101.137.102] X-Barracuda-Start-Time: 1265687820 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22047 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean dquots are never flushed asynchronously. Remove the flag and the async write support from the flush function. Make the default flush a delwri flush to make the inode flush code, which leaves the XFS_QMOPT_SYNC the only flag remaining. Convert that to use SYNC_WAIT instead, just like the inode flush code. V2: - just pass flush flags straight through Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/quota/xfs_dquot.c | 13 ++++++------- fs/xfs/quota/xfs_dquot_item.c | 2 +- fs/xfs/quota/xfs_qm.c | 14 ++++++-------- fs/xfs/xfs_quota.h | 8 +------- 4 files changed, 14 insertions(+), 23 deletions(-) diff --git a/fs/xfs/quota/xfs_dquot.c b/fs/xfs/quota/xfs_dquot.c index 1620a56..5f79dd7 100644 --- a/fs/xfs/quota/xfs_dquot.c +++ b/fs/xfs/quota/xfs_dquot.c @@ -1187,7 +1187,7 @@ xfs_qm_dqflush( * block, nada. */ if (!XFS_DQ_IS_DIRTY(dqp) || - (!(flags & XFS_QMOPT_SYNC) && atomic_read(&dqp->q_pincount) > 0)) { + (!(flags & SYNC_WAIT) && atomic_read(&dqp->q_pincount) > 0)) { xfs_dqfunlock(dqp); return 0; } @@ -1251,18 +1251,17 @@ xfs_qm_dqflush( xfs_log_force(mp, 0); } - if (flags & XFS_QMOPT_DELWRI) { - xfs_bdwrite(mp, bp); - } else { + if (flags & SYNC_WAIT) error = xfs_bwrite(mp, bp); - } + else + xfs_bdwrite(mp, bp); trace_xfs_dqflush_done(dqp); /* * dqp is still locked, but caller is free to unlock it now. */ - return (error); + return error; } @@ -1443,7 +1442,7 @@ xfs_qm_dqpurge( * We don't care about getting disk errors here. We need * to purge this dquot anyway, so we go ahead regardless. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_SYNC); + error = xfs_qm_dqflush(dqp, SYNC_WAIT); if (error) xfs_fs_cmn_err(CE_WARN, mp, "xfs_qm_dqpurge: dquot %p flush failed", dqp); diff --git a/fs/xfs/quota/xfs_dquot_item.c b/fs/xfs/quota/xfs_dquot_item.c index dda0fb0..4e4ee9a 100644 --- a/fs/xfs/quota/xfs_dquot_item.c +++ b/fs/xfs/quota/xfs_dquot_item.c @@ -153,7 +153,7 @@ xfs_qm_dquot_logitem_push( * lock without sleeping, then there must not have been * anyone in the process of flushing the dquot. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dquot_logitem_push: push error %d on dqp %p", diff --git a/fs/xfs/quota/xfs_qm.c b/fs/xfs/quota/xfs_qm.c index 11cfd82..8699e51 100644 --- a/fs/xfs/quota/xfs_qm.c +++ b/fs/xfs/quota/xfs_qm.c @@ -450,7 +450,7 @@ xfs_qm_unmount_quotas( STATIC int xfs_qm_dqflush_all( xfs_mount_t *mp, - int flags) + int sync_mode) { int recl; xfs_dquot_t *dqp; @@ -486,7 +486,7 @@ again: * across a disk write. */ xfs_qm_mplist_unlock(mp); - error = xfs_qm_dqflush(dqp, flags); + error = xfs_qm_dqflush(dqp, sync_mode); xfs_dqunlock(dqp); if (error) return error; @@ -926,13 +926,11 @@ xfs_qm_sync( { int recl, restarts; xfs_dquot_t *dqp; - uint flush_flags; int error; if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp)) return 0; - flush_flags = (flags & SYNC_WAIT) ? XFS_QMOPT_SYNC : XFS_QMOPT_DELWRI; restarts = 0; again: @@ -992,7 +990,7 @@ xfs_qm_sync( * across a disk write */ xfs_qm_mplist_unlock(mp); - error = xfs_qm_dqflush(dqp, flush_flags); + error = xfs_qm_dqflush(dqp, flags); xfs_dqunlock(dqp); if (error && XFS_FORCED_SHUTDOWN(mp)) return 0; /* Need to prevent umount failure */ @@ -1796,7 +1794,7 @@ xfs_qm_quotacheck( * successfully. */ if (!error) - error = xfs_qm_dqflush_all(mp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush_all(mp, 0); /* * We can get this error if we couldn't do a dquot allocation inside @@ -2018,7 +2016,7 @@ xfs_qm_shake_freelist( * We flush it delayed write, so don't bother * releasing the mplock. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) { xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dqflush_all: dquot %p flush failed", dqp); @@ -2201,7 +2199,7 @@ xfs_qm_dqreclaim_one(void) * We flush it delayed write, so don't bother * releasing the freelist lock. */ - error = xfs_qm_dqflush(dqp, XFS_QMOPT_DELWRI); + error = xfs_qm_dqflush(dqp, 0); if (error) { xfs_fs_cmn_err(CE_WARN, dqp->q_mount, "xfs_qm_dqreclaim: dquot %p flush failed", dqp); diff --git a/fs/xfs/xfs_quota.h b/fs/xfs/xfs_quota.h index 21d11d9..fdcab3f 100644 --- a/fs/xfs/xfs_quota.h +++ b/fs/xfs/xfs_quota.h @@ -223,15 +223,9 @@ typedef struct xfs_qoff_logformat { #define XFS_QMOPT_RES_INOS 0x0800000 /* - * flags for dqflush and dqflush_all. - */ -#define XFS_QMOPT_SYNC 0x1000000 -#define XFS_QMOPT_DELWRI 0x4000000 - -/* * flags for dqalloc. */ -#define XFS_QMOPT_INHERIT 0x8000000 +#define XFS_QMOPT_INHERIT 0x1000000 /* * flags to xfs_trans_mod_dquot. -- 1.6.5 From SRS0+ev3T+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:48 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,LOCAL_GNU_PATCH autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tlZj182555 for ; Mon, 8 Feb 2010 21:55:47 -0600 X-ASG-Debug-ID: 1265687818-245400950000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 6209F1BBE09 for ; Mon, 8 Feb 2010 19:56:59 -0800 (PST) Received: from mail.internode.on.net (bld-mail19.adl2.internode.on.net [150.101.137.104]) by cuda.sgi.com with ESMTP id PmukB8oDooE2SVnD for ; Mon, 08 Feb 2010 19:56:59 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 12946199-1927428 for ; Tue, 09 Feb 2010 14:26:57 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDS-0000O1-1z for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:46 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDR-0002SF-Bc for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:45 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 2/9] xfs: Use delayed write for inodes rather than async V2 Subject: [PATCH 2/9] xfs: Use delayed write for inodes rather than async V2 Date: Tue, 9 Feb 2010 14:56:35 +1100 Message-Id: <1265687802-23043-3-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail19.adl2.internode.on.net[150.101.137.104] X-Barracuda-Start-Time: 1265687820 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean We currently do background inode flush asynchronously, resulting in inodes being written in whatever order the background writeback issues them. Not only that, there are also blocking and non-blocking asynchronous inode flushes, depending on where the flush comes from. This patch completely removes asynchronous inode writeback. It removes all the strange writeback modes and replaces them with either a synchronous flush or a non-blocking delayed write flush. That is, inode flushes will only issue IO directly if they are synchronous, and background flushing may do nothing if the operation would block (e.g. on a pinned inode or buffer lock). Delayed write flushes will now result in the inode buffer sitting in the delwri queue of the buffer cache to be flushed by either an AIL push or by the xfsbufd timing out the buffer. This will allow accumulation of dirty inode buffers in memory and allow optimisation of inode cluster writeback at the xfsbufd level where we have much greater queue depths than the block layer elevators. We will also get adjacent inode cluster buffer IO merging for free when a later patch in the series allows sorting of the delayed write buffers before dispatch. This effectively means that any inode that is written back by background writeback will be seen as flush locked during AIL pushing, and will result in the buffers being pushed from there. This writeback path is currently non-optimal, but the next patch in the series will fix that problem. A side effect of this delayed write mechanism is that background inode reclaim will no longer directly flush inodes, nor can it wait on the flush lock. The result is that inode reclaim must leave the inode in the reclaimable state until it is clean. Hence attempts to reclaim a dirty inode in the background will simply skip the inode until it is clean and this allows other mechanisms (i.e. xfsbufd) to do more optimal writeback of the dirty buffers. As a result, the inode reclaim code has been rewritten so that it no longer relies on the ambiguous return values of xfs_iflush() to determine whether it is safe to reclaim an inode. Portions of this patch are derived from patches by Christoph Hellwig. Version 2: - cleanup reclaim code as suggested by Christoph - log background reclaim inode flush errors - just pass sync flags to xfs_iflush Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_super.c | 4 +- fs/xfs/linux-2.6/xfs_sync.c | 105 ++++++++++++++++++++++++++++++------------ fs/xfs/xfs_inode.c | 75 ++---------------------------- fs/xfs/xfs_inode.h | 10 ---- fs/xfs/xfs_inode_item.c | 10 +++- fs/xfs/xfs_mount.c | 13 +++++- 6 files changed, 102 insertions(+), 115 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 6ce828e..3b5b46b 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -1064,7 +1064,7 @@ xfs_fs_write_inode( xfs_ilock(ip, XFS_ILOCK_SHARED); xfs_iflock(ip); - error = xfs_iflush(ip, XFS_IFLUSH_SYNC); + error = xfs_iflush(ip, SYNC_WAIT); } else { error = EAGAIN; if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) @@ -1072,7 +1072,7 @@ xfs_fs_write_inode( if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) goto out_unlock; - error = xfs_iflush(ip, XFS_IFLUSH_ASYNC_NOBLOCK); + error = xfs_iflush(ip, 0); } out_unlock: diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c index 525260c..a9f6d20 100644 --- a/fs/xfs/linux-2.6/xfs_sync.c +++ b/fs/xfs/linux-2.6/xfs_sync.c @@ -270,8 +270,7 @@ xfs_sync_inode_attr( goto out_unlock; } - error = xfs_iflush(ip, (flags & SYNC_WAIT) ? - XFS_IFLUSH_SYNC : XFS_IFLUSH_DELWRI); + error = xfs_iflush(ip, flags); out_unlock: xfs_iunlock(ip, XFS_ILOCK_SHARED); @@ -460,16 +459,18 @@ xfs_quiesce_fs( { int count = 0, pincount; + xfs_reclaim_inodes(mp, 0); xfs_flush_buftarg(mp->m_ddev_targp, 0); - xfs_reclaim_inodes(mp, XFS_IFLUSH_DELWRI_ELSE_ASYNC); /* * This loop must run at least twice. The first instance of the loop * will flush most meta data but that will generate more meta data * (typically directory updates). Which then must be flushed and - * logged before we can write the unmount record. + * logged before we can write the unmount record. We also so sync + * reclaim of inodes to catch any that the above delwri flush skipped. */ do { + xfs_reclaim_inodes(mp, SYNC_WAIT); xfs_sync_attr(mp, SYNC_WAIT); pincount = xfs_flush_buftarg(mp->m_ddev_targp, 1); if (!pincount) { @@ -585,7 +586,7 @@ xfs_sync_worker( if (!(mp->m_flags & XFS_MOUNT_RDONLY)) { xfs_log_force(mp, 0); - xfs_reclaim_inodes(mp, XFS_IFLUSH_DELWRI_ELSE_ASYNC); + xfs_reclaim_inodes(mp, 0); /* dgc: errors ignored here */ error = xfs_qm_sync(mp, SYNC_TRYLOCK); error = xfs_sync_fsdata(mp, SYNC_TRYLOCK); @@ -719,21 +720,42 @@ __xfs_inode_clear_reclaim_tag( * shutdown EIO unpin and reclaim * clean, unpinned 0 reclaim * stale, unpinned 0 reclaim - * clean, pinned(*) 0 unpin and reclaim - * stale, pinned 0 unpin and reclaim - * dirty, async 0 block on flush lock, reclaim - * dirty, sync flush 0 block on flush lock, reclaim + * clean, pinned(*) 0 requeue + * stale, pinned EAGAIN requeue + * dirty, delwri ok 0 requeue + * dirty, delwri blocked EAGAIN requeue + * dirty, sync flush 0 reclaim * * (*) dgc: I don't think the clean, pinned state is possible but it gets * handled anyway given the order of checks implemented. * + * As can be seen from the table, the return value of xfs_iflush() is not + * sufficient to correctly decide the reclaim action here. The checks in + * xfs_iflush() might look like duplicates, but they are not. + * + * Also, because we get the flush lock first, we know that any inode that has + * been flushed delwri has had the flush completed by the time we check that + * the inode is clean. The clean inode check needs to be done before flushing + * the inode delwri otherwise we would loop forever requeuing clean inodes as + * we cannot tell apart a successful delwri flush and a clean inode from the + * return value of xfs_iflush(). + * + * Note that because the inode is flushed delayed write by background + * writeback, the flush lock may already be held here and waiting on it can + * result in very long latencies. Hence for sync reclaims, where we wait on the + * flush lock, the caller should push out delayed write inodes first before + * trying to reclaim them to minimise the amount of time spent waiting. For + * background relaim, we just requeue the inode for the next pass. + * * Hence the order of actions after gaining the locks should be: * bad => reclaim * shutdown => unpin and reclaim - * pinned => unpin + * pinned, delwri => requeue + * pinned, sync => unpin * stale => reclaim * clean => reclaim - * dirty => flush, wait and reclaim + * dirty, delwri => flush and requeue + * dirty, sync => flush, wait and reclaim */ STATIC int xfs_reclaim_inode( @@ -741,7 +763,7 @@ xfs_reclaim_inode( struct xfs_perag *pag, int sync_mode) { - int error; + int error = 0; /* * The radix tree lock here protects a thread in xfs_iget from racing @@ -761,7 +783,11 @@ xfs_reclaim_inode( write_unlock(&pag->pag_ici_lock); xfs_ilock(ip, XFS_ILOCK_EXCL); - xfs_iflock(ip); + if (!xfs_iflock_nowait(ip)) { + if (!(sync_mode & SYNC_WAIT)) + goto out; + xfs_iflock(ip); + } if (is_bad_inode(VFS_I(ip))) goto reclaim; @@ -769,8 +795,13 @@ xfs_reclaim_inode( xfs_iunpin_wait(ip); goto reclaim; } - if (xfs_ipincount(ip)) + if (xfs_ipincount(ip)) { + if (!(sync_mode & SYNC_WAIT)) { + xfs_ifunlock(ip); + goto out; + } xfs_iunpin_wait(ip); + } if (xfs_iflags_test(ip, XFS_ISTALE)) goto reclaim; if (xfs_inode_clean(ip)) @@ -778,27 +809,43 @@ xfs_reclaim_inode( /* Now we have an inode that needs flushing */ error = xfs_iflush(ip, sync_mode); - if (!error) { - switch(sync_mode) { - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - case XFS_IFLUSH_DELWRI: - case XFS_IFLUSH_ASYNC: - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - case XFS_IFLUSH_SYNC: - /* IO issued, synchronise with IO completion */ - xfs_iflock(ip); - break; - default: - ASSERT(0); - break; - } + if (sync_mode & SYNC_WAIT) { + xfs_iflock(ip); + goto reclaim; } + /* + * When we have to flush an inode but don't have SYNC_WAIT set, we + * flush the inode out using a delwri buffer and wait for the next + * call into reclaim to find it in a clean state instead of waiting for + * it now. We also don't return errors here - if the error is transient + * then the next reclaim pass will flush the inode, and if the error + * is permanent then the next sync reclaim will relcaim the inode and + * pass on the error. + */ + if (error && !XFS_FORCED_SHUTDOWN(ip->i_mount)) { + xfs_fs_cmn_err(CE_WARN, ip->i_mount, + "inode 0x%llx background reclaim flush failed with %d", + (long long)ip->i_ino, error); + } +out: + xfs_iflags_clear(ip, XFS_IRECLAIM); + xfs_iunlock(ip, XFS_ILOCK_EXCL); + /* + * We could return EAGAIN here to make reclaim rescan the inode tree in + * a short while. However, this just burns CPU time scanning the tree + * waiting for IO to complete and xfssyncd never goes back to the idle + * state. Instead, return 0 to let the next scheduled background reclaim + * attempt to reclaim the inode again. + */ + return 0; + reclaim: xfs_ifunlock(ip); xfs_iunlock(ip, XFS_ILOCK_EXCL); xfs_ireclaim(ip); - return 0; + return error; + } int diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 8d0666d..fa31360 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2835,8 +2835,6 @@ xfs_iflush( xfs_dinode_t *dip; xfs_mount_t *mp; int error; - int noblock = (flags == XFS_IFLUSH_ASYNC_NOBLOCK); - enum { INT_DELWRI = (1 << 0), INT_ASYNC = (1 << 1) }; XFS_STATS_INC(xs_iflush_count); @@ -2859,7 +2857,7 @@ xfs_iflush( * in the same cluster are dirty, they will probably write the inode * out for us if they occur after the log force completes. */ - if (noblock && xfs_ipincount(ip)) { + if (!(flags & SYNC_WAIT) && xfs_ipincount(ip)) { xfs_iunpin_nowait(ip); xfs_ifunlock(ip); return EAGAIN; @@ -2893,60 +2891,10 @@ xfs_iflush( } /* - * Decide how buffer will be flushed out. This is done before - * the call to xfs_iflush_int because this field is zeroed by it. - */ - if (iip != NULL && iip->ili_format.ilf_fields != 0) { - /* - * Flush out the inode buffer according to the directions - * of the caller. In the cases where the caller has given - * us a choice choose the non-delwri case. This is because - * the inode is in the AIL and we need to get it out soon. - */ - switch (flags) { - case XFS_IFLUSH_SYNC: - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - flags = 0; - break; - case XFS_IFLUSH_ASYNC_NOBLOCK: - case XFS_IFLUSH_ASYNC: - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - flags = INT_ASYNC; - break; - case XFS_IFLUSH_DELWRI: - flags = INT_DELWRI; - break; - default: - ASSERT(0); - flags = 0; - break; - } - } else { - switch (flags) { - case XFS_IFLUSH_DELWRI_ELSE_SYNC: - case XFS_IFLUSH_DELWRI_ELSE_ASYNC: - case XFS_IFLUSH_DELWRI: - flags = INT_DELWRI; - break; - case XFS_IFLUSH_ASYNC_NOBLOCK: - case XFS_IFLUSH_ASYNC: - flags = INT_ASYNC; - break; - case XFS_IFLUSH_SYNC: - flags = 0; - break; - default: - ASSERT(0); - flags = 0; - break; - } - } - - /* * Get the buffer containing the on-disk inode. */ error = xfs_itobp(mp, NULL, ip, &dip, &bp, - noblock ? XBF_TRYLOCK : XBF_LOCK); + (flags & SYNC_WAIT) ? XBF_LOCK : XBF_TRYLOCK); if (error || !bp) { xfs_ifunlock(ip); return error; @@ -2974,13 +2922,10 @@ xfs_iflush( if (error) goto cluster_corrupt_out; - if (flags & INT_DELWRI) { - xfs_bdwrite(mp, bp); - } else if (flags & INT_ASYNC) { - error = xfs_bawrite(mp, bp); - } else { + if (flags & SYNC_WAIT) error = xfs_bwrite(mp, bp); - } + else + xfs_bdwrite(mp, bp); return error; corrupt_out: @@ -3015,16 +2960,6 @@ xfs_iflush_int( iip = ip->i_itemp; mp = ip->i_mount; - - /* - * If the inode isn't dirty, then just release the inode - * flush lock and do nothing. - */ - if (xfs_inode_clean(ip)) { - xfs_ifunlock(ip); - return 0; - } - /* set *dip = inode's place in the buffer */ dip = (xfs_dinode_t *)xfs_buf_offset(bp, ip->i_imap.im_boffset); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 8b618ea..6c912b0 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -420,16 +420,6 @@ static inline void xfs_ifunlock(xfs_inode_t *ip) #define XFS_ILOCK_DEP(flags) (((flags) & XFS_ILOCK_DEP_MASK) >> XFS_ILOCK_SHIFT) /* - * Flags for xfs_iflush() - */ -#define XFS_IFLUSH_DELWRI_ELSE_SYNC 1 -#define XFS_IFLUSH_DELWRI_ELSE_ASYNC 2 -#define XFS_IFLUSH_SYNC 3 -#define XFS_IFLUSH_ASYNC 4 -#define XFS_IFLUSH_DELWRI 5 -#define XFS_IFLUSH_ASYNC_NOBLOCK 6 - -/* * Flags for xfs_itruncate_start(). */ #define XFS_ITRUNC_DEFINITE 0x1 diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index 48ec1c0..207553e 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -866,10 +866,14 @@ xfs_inode_item_push( iip->ili_format.ilf_fields != 0); /* - * Write out the inode. The completion routine ('iflush_done') will - * pull it from the AIL, mark it clean, unlock the flush lock. + * Push the inode to it's backing buffer. This will not remove the + * inode from the AIL - a further push will be required to trigger a + * buffer push. However, this allows all the dirty inodes to be pushed + * to the buffer before it is pushed to disk. THe buffer IO completion + * will pull th einode from the AIL, mark it clean and unlock the flush + * lock. */ - (void) xfs_iflush(ip, XFS_IFLUSH_ASYNC); + (void) xfs_iflush(ip, 0); xfs_iunlock(ip, XFS_ILOCK_SHARED); return; diff --git a/fs/xfs/xfs_mount.c b/fs/xfs/xfs_mount.c index 5061149..6afaaeb 100644 --- a/fs/xfs/xfs_mount.c +++ b/fs/xfs/xfs_mount.c @@ -1468,7 +1468,18 @@ xfs_unmountfs( * need to force the log first. */ xfs_log_force(mp, XFS_LOG_SYNC); - xfs_reclaim_inodes(mp, XFS_IFLUSH_ASYNC); + + /* + * Do a delwri reclaim pass first so that as many dirty inodes are + * queued up for IO as possible. Then flush the buffers before making + * a synchronous path to catch all the remaining inodes are reclaimed. + * This makes the reclaim process as quick as possible by avoiding + * synchronous writeout and blocking on inodes already in the delwri + * state as much as possible. + */ + xfs_reclaim_inodes(mp, 0); + XFS_bflush(mp->m_ddev_targp); + xfs_reclaim_inodes(mp, SYNC_WAIT); xfs_qm_unmount(mp); -- 1.6.5 From SRS0+Ag99+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193trbf182599 for ; Mon, 8 Feb 2010 21:55:53 -0600 X-ASG-Debug-ID: 1265687825-287700400000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id DC44E1BBE0F for ; Mon, 8 Feb 2010 19:57:05 -0800 (PST) Received: from mail.internode.on.net (bld-mail12.adl6.internode.on.net [150.101.137.97]) by cuda.sgi.com with ESMTP id wegbsKG9MbXMmiZy for ; Mon, 08 Feb 2010 19:57:05 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 13074394-1927428 for multiple; Tue, 09 Feb 2010 14:26:58 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDc-0000OR-CA; Tue, 09 Feb 2010 14:56:56 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDb-0002Xr-Jn; Tue, 09 Feb 2010 14:56:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: Christoph Hellwig X-ASG-Orig-Subj: [PATCH 7/9] xfs: remove invalid barrier optimization from xfs_fsync Subject: [PATCH 7/9] xfs: remove invalid barrier optimization from xfs_fsync Date: Tue, 9 Feb 2010 14:56:40 +1100 Message-Id: <1265687802-23043-8-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail12.adl6.internode.on.net[150.101.137.97] X-Barracuda-Start-Time: 1265687826 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean From: Christoph Hellwig We always need to flush the disk write cache and can't skip it just because the no inode attributes have changed. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Signed-off-by: Dave Chinner --- fs/xfs/xfs_vnodeops.c | 12 ++---------- 1 files changed, 2 insertions(+), 10 deletions(-) diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index fd108b7..43241e2 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -597,7 +597,7 @@ xfs_fsync( { xfs_trans_t *tp; int error = 0; - int log_flushed = 0, changed = 1; + int log_flushed = 0; xfs_itrace_entry(ip); @@ -627,18 +627,10 @@ xfs_fsync( * disk yet, the inode will be still be pinned. If it is, * force the log. */ - xfs_iunlock(ip, XFS_ILOCK_SHARED); - if (xfs_ipincount(ip)) { error = _xfs_log_force(ip->i_mount, XFS_LOG_SYNC, &log_flushed); - } else { - /* - * If the inode is not pinned and nothing has changed - * we don't need to flush the cache. - */ - changed = 0; } } else { /* @@ -673,7 +665,7 @@ xfs_fsync( xfs_iunlock(ip, XFS_ILOCK_EXCL); } - if ((ip->i_mount->m_flags & XFS_MOUNT_BARRIER) && changed) { + if (ip->i_mount->m_flags & XFS_MOUNT_BARRIER) { /* * If the log write didn't issue an ordered tag we need * to flush the disk cache for the data device now. -- 1.6.5 From SRS0+WGMz+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:57 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193tuQf182615 for ; Mon, 8 Feb 2010 21:55:57 -0600 X-ASG-Debug-ID: 1265687828-14e502bd0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id D97591CB5F5D for ; Mon, 8 Feb 2010 19:57:08 -0800 (PST) Received: from mail.internode.on.net (bld-mail14.adl6.internode.on.net [150.101.137.99]) by cuda.sgi.com with ESMTP id qjCiCZHM4Ec9doRV for ; Mon, 08 Feb 2010 19:57:08 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 13236073-1927428 for ; Tue, 09 Feb 2010 14:27:07 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDm-0000Og-E6 for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:57:06 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDb-0002Xx-Ns for xfs@oss.sgi.com; Tue, 09 Feb 2010 14:56:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH 9/9] xfs: kill xfs_bawrite Subject: [PATCH 9/9] xfs: kill xfs_bawrite Date: Tue, 9 Feb 2010 14:56:42 +1100 Message-Id: <1265687802-23043-10-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail14.adl6.internode.on.net[150.101.137.99] X-Barracuda-Start-Time: 1265687829 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean There are no more users of this function left in the XFS code now that we've switched everything to delayed write flushing. Remove it. Signed-off-by: Dave Chinner Reviewed-by: Christoph Hellwig --- fs/xfs/linux-2.6/xfs_buf.c | 19 ------------------- fs/xfs/linux-2.6/xfs_buf.h | 1 - 2 files changed, 0 insertions(+), 20 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c index 4556a4c..d50df3a 100644 --- a/fs/xfs/linux-2.6/xfs_buf.c +++ b/fs/xfs/linux-2.6/xfs_buf.c @@ -1078,25 +1078,6 @@ xfs_bwrite( return error; } -int -xfs_bawrite( - void *mp, - struct xfs_buf *bp) -{ - trace_xfs_buf_bawrite(bp, _RET_IP_); - - ASSERT(bp->b_bn != XFS_BUF_DADDR_NULL); - - xfs_buf_delwri_dequeue(bp); - - bp->b_flags &= ~(XBF_READ | XBF_DELWRI | XBF_READ_AHEAD); - bp->b_flags |= (XBF_WRITE | XBF_ASYNC | _XBF_RUN_QUEUES); - - bp->b_mount = mp; - bp->b_strat = xfs_bdstrat_cb; - return xfs_bdstrat_cb(bp); -} - void xfs_bdwrite( void *mp, diff --git a/fs/xfs/linux-2.6/xfs_buf.h b/fs/xfs/linux-2.6/xfs_buf.h index be45e8c..386e736 100644 --- a/fs/xfs/linux-2.6/xfs_buf.h +++ b/fs/xfs/linux-2.6/xfs_buf.h @@ -233,7 +233,6 @@ extern void xfs_buf_unlock(xfs_buf_t *); /* Buffer Read and Write Routines */ extern int xfs_bwrite(struct xfs_mount *mp, struct xfs_buf *bp); -extern int xfs_bawrite(void *mp, xfs_buf_t *bp); extern void xfs_bdwrite(void *mp, xfs_buf_t *bp); extern void xfsbdstrat(struct xfs_mount *, struct xfs_buf *); -- 1.6.5 From SRS0+WGMz+69+fromorbit.com=dave@internode.on.net Mon Feb 8 21:55:59 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o193twYm182630 for ; Mon, 8 Feb 2010 21:55:58 -0600 X-ASG-Debug-ID: 1265687829-2453009e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 8C8E41BBE11 for ; Mon, 8 Feb 2010 19:57:10 -0800 (PST) Received: from mail.internode.on.net (bld-mail15.adl6.internode.on.net [150.101.137.100]) by cuda.sgi.com with ESMTP id 5Z5DTc6nIOq6v5rl for ; Mon, 08 Feb 2010 19:57:10 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 1461910-1927428 for multiple; Tue, 09 Feb 2010 14:26:58 +1030 (CDT) Received: from disturbed ([192.168.1.9]) by discord with esmtp (Exim 4.69) (envelope-from ) id 1NehDc-0000OT-Ds; Tue, 09 Feb 2010 14:56:56 +1100 Received: from dave by disturbed with local (Exim 4.71) (envelope-from ) id 1NehDb-0002Xt-MC; Tue, 09 Feb 2010 14:56:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: Christoph Hellwig X-ASG-Orig-Subj: [PATCH 8/9] xfs: log changed inodes instead of writing them synchronously Subject: [PATCH 8/9] xfs: log changed inodes instead of writing them synchronously Date: Tue, 9 Feb 2010 14:56:41 +1100 Message-Id: <1265687802-23043-9-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 1.6.5 In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: bld-mail15.adl6.internode.on.net[150.101.137.100] X-Barracuda-Start-Time: 1265687831 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22048 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean From: Christoph Hellwig When an inode has already be flushed delayed write, xfs_inode_clean() returns true and hence xfs_fs_write_inode() can return on a synchronous inode write without having written the inode. Currently these sycnhronous writes only come sync(1), unmount, a sycnhronous NFS export and cachefiles so should be relatively rare and out of common performance paths. Realistically, a synchronous inode write is not necessary here; we can avoid writing the inode by logging any non-transactional changes that are pending. This needs to be done with synchronous transactions, but it avoids seeking between the log and inode clusters as we do now. We don't force the log if the inode is pinned, though, so this differs from the fsync case. For normal sys_sync and unmount behaviour this is fine because we do a synchronous log force in xfs_sync_data which is called from the ->sync_fs code. It does however break the NFS synchronous export guarantees for now, but work is under way to fix this at a higher level or for the higher level to provide an additional flag in the writeback control to tell us that a log force is needed. Portions of this patch are based on work from Dave Chinner. Signed-off-by: Christoph Hellwig Reviewed-by: Dave Chinner Reviewed-by: Alex Elder Signed-off-by: Dave Chinner --- fs/xfs/linux-2.6/xfs_super.c | 111 +++++++++++++++++++++++++++++++----------- 1 files changed, 82 insertions(+), 29 deletions(-) diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 3b5b46b..25ea240 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -1021,12 +1021,45 @@ xfs_fs_dirty_inode( XFS_I(inode)->i_update_core = 1; } -/* - * Attempt to flush the inode, this will actually fail - * if the inode is pinned, but we dirty the inode again - * at the point when it is unpinned after a log write, - * since this is when the inode itself becomes flushable. - */ +STATIC int +xfs_log_inode( + struct xfs_inode *ip) +{ + struct xfs_mount *mp = ip->i_mount; + struct xfs_trans *tp; + int error; + + xfs_iunlock(ip, XFS_ILOCK_SHARED); + tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS); + error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0); + + if (error) { + xfs_trans_cancel(tp, 0); + /* we need to return with the lock hold shared */ + xfs_ilock(ip, XFS_ILOCK_SHARED); + return error; + } + + xfs_ilock(ip, XFS_ILOCK_EXCL); + + /* + * Note - it's possible that we might have pushed ourselves out of the + * way during trans_reserve which would flush the inode. But there's + * no guarantee that the inode buffer has actually gone out yet (it's + * delwri). Plus the buffer could be pinned anyway if it's part of + * an inode in another recent transaction. So we play it safe and + * fire off the transaction anyway. + */ + xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); + xfs_trans_ihold(tp, ip); + xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); + xfs_trans_set_sync(tp); + error = xfs_trans_commit(tp, 0); + xfs_ilock_demote(ip, XFS_ILOCK_EXCL); + + return error; +} + STATIC int xfs_fs_write_inode( struct inode *inode, @@ -1034,7 +1067,7 @@ xfs_fs_write_inode( { struct xfs_inode *ip = XFS_I(inode); struct xfs_mount *mp = ip->i_mount; - int error = 0; + int error = EAGAIN; xfs_itrace_entry(ip); @@ -1045,35 +1078,55 @@ xfs_fs_write_inode( error = xfs_wait_on_pages(ip, 0, -1); if (error) goto out; - } - - /* - * Bypass inodes which have already been cleaned by - * the inode flush clustering code inside xfs_iflush - */ - if (xfs_inode_clean(ip)) - goto out; - /* - * We make this non-blocking if the inode is contended, return - * EAGAIN to indicate to the caller that they did not succeed. - * This prevents the flush path from blocking on inodes inside - * another operation right now, they get caught later by xfs_sync. - */ - if (sync) { + /* + * Make sure the inode has hit stable storage. By using the + * log and the fsync transactions we reduce the IOs we have + * to do here from two (log and inode) to just the log. + * + * Note: We still need to do a delwri write of the inode after + * this to flush it to the backing buffer so that bulkstat + * works properly if this is the first time the inode has been + * written. Because we hold the ilock atomically over the + * transaction commit and the inode flush we are guaranteed + * that the inode is not pinned when it returns. If the flush + * lock is already held, then the inode has already been + * flushed once and we don't need to flush it again. Hence + * the code will only flush the inode if it isn't already + * being flushed. + */ xfs_ilock(ip, XFS_ILOCK_SHARED); - xfs_iflock(ip); - - error = xfs_iflush(ip, SYNC_WAIT); + if (ip->i_update_core) { + error = xfs_log_inode(ip); + if (error) + goto out_unlock; + } } else { - error = EAGAIN; + /* + * We make this non-blocking if the inode is contended, return + * EAGAIN to indicate to the caller that they did not succeed. + * This prevents the flush path from blocking on inodes inside + * another operation right now, they get caught later by xfs_sync. + */ if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) goto out; - if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) - goto out_unlock; + } + + if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip)) + goto out_unlock; - error = xfs_iflush(ip, 0); + /* + * Now we have the flush lock and the inode is not pinned, we can check + * if the inode is really clean as we know that there are no pending + * transaction completions, it is not waiting on the delayed write + * queue and there is no IO in progress. + */ + if (xfs_inode_clean(ip)) { + xfs_ifunlock(ip); + error = 0; + goto out_unlock; } + error = xfs_iflush(ip, 0); out_unlock: xfs_iunlock(ip, XFS_ILOCK_SHARED); -- 1.6.5 From patrick@news-service.com Tue Feb 9 02:47:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o198lVEP202585 for ; Tue, 9 Feb 2010 02:47:32 -0600 X-ASG-Debug-ID: 1265705322-2f34020f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from pu01.news-service.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 62A00138B50C for ; Tue, 9 Feb 2010 00:48:42 -0800 (PST) Received: from pu01.news-service.com (ns1.news-service.com [195.114.240.3]) by cuda.sgi.com with ESMTP id qkyP3e4f78DIrjyB for ; Tue, 09 Feb 2010 00:48:42 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by pu01.news-service.com (Postfix) with ESMTP id BF8C213E29; Tue, 9 Feb 2010 09:48:41 +0100 (CET) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Debian amavisd-new at pu01.news-service.com Received: from pu01.news-service.com ([127.0.0.1]) by localhost (pu01.nse [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UiG+Mt-WX8qX; Tue, 9 Feb 2010 09:48:39 +0100 (CET) Received: from [172.25.4.14] (fw01.nse [172.25.8.1]) by pu01.news-service.com (Postfix) with ESMTP id 03F1013E24; Tue, 9 Feb 2010 09:48:38 +0100 (CET) Message-ID: <4B712166.9010701@news-service.com> Date: Tue, 09 Feb 2010 09:48:38 +0100 From: Patrick Schreurs User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Christoph Hellwig CC: Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) References: <4AF0422D.1070104@news-service.com> <20091114162126.GB17658@infradead.org> <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> In-Reply-To: <20100208194226.GD9527@infradead.org> Content-Type: multipart/mixed; boundary="------------090506040001040402060507" X-Barracuda-Connect: ns1.news-service.com[195.114.240.3] X-Barracuda-Start-Time: 1265705323 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22065 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean This is a multi-part message in MIME format. --------------090506040001040402060507 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 8-2-2010 20:42, Christoph Hellwig wrote: > On Mon, Feb 01, 2010 at 05:52:30PM +0100, Patrick Schreurs wrote: >> Hello Dave, >> >> I'm afraid we had another crash a few days ago. Have a look at the >> screenshot. Can you make anything out of it? This server is running >> 2.6.32.3 with your inode-reclaim patch applied and XFS_DEBUG still >> enabled. > > Just wondering, which set of patches is this exactly? This is a clean 2.6.32.3 with the xfs-inode-reclaim-2.6.32 patch i received from Dave on January 8th (see attachment). Thanks, -Patrick --------------090506040001040402060507 Content-Type: text/plain; name="xfs-inode-reclaim-2.6.32" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xfs-inode-reclaim-2.6.32" diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 18a4b8e..f3c622a 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -930,13 +930,37 @@ xfs_fs_alloc_inode( */ STATIC void xfs_fs_destroy_inode( - struct inode *inode) + struct inode *inode) { - xfs_inode_t *ip = XFS_I(inode); + struct xfs_inode *ip = XFS_I(inode); + + xfs_itrace_entry(ip); XFS_STATS_INC(vn_reclaim); - if (xfs_reclaim(ip)) - panic("%s: cannot reclaim 0x%p\n", __func__, inode); + + /* bad inode, get out here ASAP */ + if (is_bad_inode(inode)) + goto out_reclaim; + + xfs_ioend_wait(ip); + + ASSERT(XFS_FORCED_SHUTDOWN(ip->i_mount) || ip->i_delayed_blks == 0); + + /* + * We should never get here with one of the reclaim flags already set. + */ + ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIMABLE)); + ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIM)); + + /* + * we always use background reclaim here because even if the + * inode is clean, it still may be under IO and hence we have + * to take the flush lock. The background reclaim path handles + * this more efficiently than we can here, so simply let background + * reclaim tear down all inodes. + */ +out_reclaim: + xfs_inode_set_reclaim_tag(ip); } /* diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c index 961df0a..897fcb5 100644 --- a/fs/xfs/linux-2.6/xfs_sync.c +++ b/fs/xfs/linux-2.6/xfs_sync.c @@ -54,7 +54,8 @@ xfs_inode_ag_lookup( struct xfs_mount *mp, struct xfs_perag *pag, uint32_t *first_index, - int tag) + int tag, + int write_lock) { int nr_found; struct xfs_inode *ip; @@ -64,7 +65,10 @@ xfs_inode_ag_lookup( * as the tree is sparse and a gang lookup walks to find * the number of objects requested. */ - read_lock(&pag->pag_ici_lock); + if (write_lock) + write_lock(&pag->pag_ici_lock); + else + read_lock(&pag->pag_ici_lock); if (tag == XFS_ICI_NO_TAG) { nr_found = radix_tree_gang_lookup(&pag->pag_ici_root, (void **)&ip, *first_index, 1); @@ -88,7 +92,10 @@ xfs_inode_ag_lookup( return ip; unlock: - read_unlock(&pag->pag_ici_lock); + if (write_lock) + write_unlock(&pag->pag_ici_lock); + else + read_unlock(&pag->pag_ici_lock); return NULL; } @@ -99,7 +106,8 @@ xfs_inode_ag_walk( int (*execute)(struct xfs_inode *ip, struct xfs_perag *pag, int flags), int flags, - int tag) + int tag, + int write_lock) { struct xfs_perag *pag = &mp->m_perag[ag]; uint32_t first_index; @@ -113,7 +121,8 @@ restart: int error = 0; xfs_inode_t *ip; - ip = xfs_inode_ag_lookup(mp, pag, &first_index, tag); + ip = xfs_inode_ag_lookup(mp, pag, &first_index, tag, + write_lock); if (!ip) break; @@ -147,7 +156,8 @@ xfs_inode_ag_iterator( int (*execute)(struct xfs_inode *ip, struct xfs_perag *pag, int flags), int flags, - int tag) + int tag, + int write_lock) { int error = 0; int last_error = 0; @@ -156,7 +166,8 @@ xfs_inode_ag_iterator( for (ag = 0; ag < mp->m_sb.sb_agcount; ag++) { if (!mp->m_perag[ag].pag_ici_init) continue; - error = xfs_inode_ag_walk(mp, ag, execute, flags, tag); + error = xfs_inode_ag_walk(mp, ag, execute, flags, tag, + write_lock); if (error) { last_error = error; if (error == EFSCORRUPTED) @@ -180,18 +191,20 @@ xfs_sync_inode_valid( return EFSCORRUPTED; } - /* - * If we can't get a reference on the inode, it must be in reclaim. - * Leave it for the reclaim code to flush. Also avoid inodes that - * haven't been fully initialised. - */ + /* avoid new or reclaimable inodes. Leave for reclaim code to flush */ + if (xfs_iflags_test(ip, XFS_INEW | XFS_IRECLAIMABLE | XFS_IRECLAIM)) { + read_unlock(&pag->pag_ici_lock); + return ENOENT; + } + + /* If we can't get a reference on the inode, it must be in reclaim. */ if (!igrab(inode)) { read_unlock(&pag->pag_ici_lock); return ENOENT; } read_unlock(&pag->pag_ici_lock); - if (is_bad_inode(inode) || xfs_iflags_test(ip, XFS_INEW)) { + if (is_bad_inode(inode)) { IRELE(ip); return ENOENT; } @@ -281,7 +294,7 @@ xfs_sync_data( ASSERT((flags & ~(SYNC_TRYLOCK|SYNC_WAIT)) == 0); error = xfs_inode_ag_iterator(mp, xfs_sync_inode_data, flags, - XFS_ICI_NO_TAG); + XFS_ICI_NO_TAG, 0); if (error) return XFS_ERROR(error); @@ -303,7 +316,7 @@ xfs_sync_attr( ASSERT((flags & ~SYNC_WAIT) == 0); return xfs_inode_ag_iterator(mp, xfs_sync_inode_attr, flags, - XFS_ICI_NO_TAG); + XFS_ICI_NO_TAG, 0); } STATIC int @@ -663,36 +676,11 @@ xfs_syncd_stop( kthread_stop(mp->m_sync_task); } -int +STATIC int xfs_reclaim_inode( xfs_inode_t *ip, - int locked, int sync_mode) { - xfs_perag_t *pag = xfs_get_perag(ip->i_mount, ip->i_ino); - - /* The hash lock here protects a thread in xfs_iget_core from - * racing with us on linking the inode back with a vnode. - * Once we have the XFS_IRECLAIM flag set it will not touch - * us. - */ - write_lock(&pag->pag_ici_lock); - spin_lock(&ip->i_flags_lock); - if (__xfs_iflags_test(ip, XFS_IRECLAIM) || - !__xfs_iflags_test(ip, XFS_IRECLAIMABLE)) { - spin_unlock(&ip->i_flags_lock); - write_unlock(&pag->pag_ici_lock); - if (locked) { - xfs_ifunlock(ip); - xfs_iunlock(ip, XFS_ILOCK_EXCL); - } - return -EAGAIN; - } - __xfs_iflags_set(ip, XFS_IRECLAIM); - spin_unlock(&ip->i_flags_lock); - write_unlock(&pag->pag_ici_lock); - xfs_put_perag(ip->i_mount, pag); - /* * If the inode is still dirty, then flush it out. If the inode * is not in the AIL, then it will be OK to flush it delwri as @@ -704,14 +692,14 @@ xfs_reclaim_inode( * We get the flush lock regardless, though, just to make sure * we don't free it while it is being flushed. */ - if (!locked) { - xfs_ilock(ip, XFS_ILOCK_EXCL); - xfs_iflock(ip); - } + xfs_ilock(ip, XFS_ILOCK_EXCL); + xfs_iflock(ip); /* * In the case of a forced shutdown we rely on xfs_iflush() to * wait for the inode to be unpinned before returning an error. + * Because we hold the flush lock, we know that the inode cannot + * be under IO, so if it reports clean it can be reclaimed. */ if (!is_bad_inode(VFS_I(ip)) && xfs_iflush(ip, sync_mode) == 0) { /* synchronize with xfs_iflush_done */ @@ -771,14 +759,24 @@ xfs_reclaim_inode_now( struct xfs_perag *pag, int flags) { - /* ignore if already under reclaim */ - if (xfs_iflags_test(ip, XFS_IRECLAIM)) { - read_unlock(&pag->pag_ici_lock); + /* + * The radix tree lock here protects a thread in xfs_iget from racing + * with us starting reclaim on the inode. Once we have the + * XFS_IRECLAIM flag set it will not touch us. + */ + spin_lock(&ip->i_flags_lock); + ASSERT_ALWAYS(__xfs_iflags_test(ip, XFS_IRECLAIMABLE)); + if (__xfs_iflags_test(ip, XFS_IRECLAIM)) { + /* ignore as it is already under reclaim */ + spin_unlock(&ip->i_flags_lock); + write_unlock(&pag->pag_ici_lock); return 0; } - read_unlock(&pag->pag_ici_lock); + __xfs_iflags_set(ip, XFS_IRECLAIM); + spin_unlock(&ip->i_flags_lock); + write_unlock(&pag->pag_ici_lock); - return xfs_reclaim_inode(ip, 0, flags); + return xfs_reclaim_inode(ip, flags); } int @@ -787,5 +785,5 @@ xfs_reclaim_inodes( int mode) { return xfs_inode_ag_iterator(mp, xfs_reclaim_inode_now, mode, - XFS_ICI_RECLAIM_TAG); + XFS_ICI_RECLAIM_TAG, 1); } diff --git a/fs/xfs/linux-2.6/xfs_sync.h b/fs/xfs/linux-2.6/xfs_sync.h index 27920eb..ea932b4 100644 --- a/fs/xfs/linux-2.6/xfs_sync.h +++ b/fs/xfs/linux-2.6/xfs_sync.h @@ -44,7 +44,6 @@ void xfs_quiesce_attr(struct xfs_mount *mp); void xfs_flush_inodes(struct xfs_inode *ip); -int xfs_reclaim_inode(struct xfs_inode *ip, int locked, int sync_mode); int xfs_reclaim_inodes(struct xfs_mount *mp, int mode); void xfs_inode_set_reclaim_tag(struct xfs_inode *ip); @@ -55,6 +54,6 @@ void __xfs_inode_clear_reclaim_tag(struct xfs_mount *mp, struct xfs_perag *pag, int xfs_sync_inode_valid(struct xfs_inode *ip, struct xfs_perag *pag); int xfs_inode_ag_iterator(struct xfs_mount *mp, int (*execute)(struct xfs_inode *ip, struct xfs_perag *pag, int flags), - int flags, int tag); + int flags, int tag, int write_lock); #endif diff --git a/fs/xfs/quota/xfs_qm_syscalls.c b/fs/xfs/quota/xfs_qm_syscalls.c index 5d1a3b9..f99cfa4 100644 --- a/fs/xfs/quota/xfs_qm_syscalls.c +++ b/fs/xfs/quota/xfs_qm_syscalls.c @@ -893,7 +893,7 @@ xfs_qm_dqrele_all_inodes( uint flags) { ASSERT(mp->m_quotainfo); - xfs_inode_ag_iterator(mp, xfs_dqrele_inode, flags, XFS_ICI_NO_TAG); + xfs_inode_ag_iterator(mp, xfs_dqrele_inode, flags, XFS_ICI_NO_TAG, 0); } /*------------------------------------------------------------------------*/ diff --git a/fs/xfs/xfs_iget.c b/fs/xfs/xfs_iget.c index 80e5264..40e8775 100644 --- a/fs/xfs/xfs_iget.c +++ b/fs/xfs/xfs_iget.c @@ -511,17 +511,21 @@ xfs_ireclaim( { struct xfs_mount *mp = ip->i_mount; struct xfs_perag *pag; + xfs_agino_t agino = XFS_INO_TO_AGINO(mp, ip->i_ino); XFS_STATS_INC(xs_ig_reclaims); /* - * Remove the inode from the per-AG radix tree. It doesn't matter - * if it was never added to it because radix_tree_delete can deal - * with that case just fine. + * Remove the inode from the per-AG radix tree. + * + * Because radix_tree_delete won't complain even if the item was never + * added to the tree assert that it's been there before to catch + * problems with the inode life time early on. */ pag = xfs_get_perag(mp, ip->i_ino); write_lock(&pag->pag_ici_lock); - radix_tree_delete(&pag->pag_ici_root, XFS_INO_TO_AGINO(mp, ip->i_ino)); + if (!radix_tree_delete(&pag->pag_ici_root, agino)) + ASSERT(0); write_unlock(&pag->pag_ici_lock); xfs_put_perag(mp, pag); diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index b92a4fa..13d7d21 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2877,10 +2877,14 @@ xfs_iflush( mp = ip->i_mount; /* - * If the inode isn't dirty, then just release the inode - * flush lock and do nothing. + * If the inode isn't dirty, then just release the inode flush lock and + * do nothing. Treat stale inodes the same; we cannot rely on the + * backing buffer remaining stale in cache for the remaining life of + * the stale inode and so xfs_itobp() below may give us a buffer that + * no longer contains inodes below. Doing this stale check here also + * avoids forcing the log on pinned, stale inodes. */ - if (xfs_inode_clean(ip)) { + if (xfs_inode_clean(ip) || xfs_iflags_test(ip, XFS_ISTALE)) { xfs_ifunlock(ip); return 0; } diff --git a/fs/xfs/xfs_vnodeops.c b/fs/xfs/xfs_vnodeops.c index b572f7e..3fac146 100644 --- a/fs/xfs/xfs_vnodeops.c +++ b/fs/xfs/xfs_vnodeops.c @@ -2456,46 +2456,6 @@ xfs_set_dmattrs( return error; } -int -xfs_reclaim( - xfs_inode_t *ip) -{ - - xfs_itrace_entry(ip); - - ASSERT(!VN_MAPPED(VFS_I(ip))); - - /* bad inode, get out here ASAP */ - if (is_bad_inode(VFS_I(ip))) { - xfs_ireclaim(ip); - return 0; - } - - xfs_ioend_wait(ip); - - ASSERT(XFS_FORCED_SHUTDOWN(ip->i_mount) || ip->i_delayed_blks == 0); - - /* - * If we have nothing to flush with this inode then complete the - * teardown now, otherwise break the link between the xfs inode and the - * linux inode and clean up the xfs inode later. This avoids flushing - * the inode to disk during the delete operation itself. - * - * When breaking the link, we need to set the XFS_IRECLAIMABLE flag - * first to ensure that xfs_iunpin() will never see an xfs inode - * that has a linux inode being reclaimed. Synchronisation is provided - * by the i_flags_lock. - */ - if (!ip->i_update_core && (ip->i_itemp == NULL)) { - xfs_ilock(ip, XFS_ILOCK_EXCL); - xfs_iflock(ip); - xfs_iflags_set(ip, XFS_IRECLAIMABLE); - return xfs_reclaim_inode(ip, 1, XFS_IFLUSH_DELWRI_ELSE_SYNC); - } - xfs_inode_set_reclaim_tag(ip); - return 0; -} - /* * xfs_alloc_file_space() * This routine allocates disk space for the given file. diff --git a/fs/xfs/xfs_vnodeops.h b/fs/xfs/xfs_vnodeops.h index a9e102d..167a467 100644 --- a/fs/xfs/xfs_vnodeops.h +++ b/fs/xfs/xfs_vnodeops.h @@ -38,7 +38,6 @@ int xfs_symlink(struct xfs_inode *dp, struct xfs_name *link_name, const char *target_path, mode_t mode, struct xfs_inode **ipp, cred_t *credp); int xfs_set_dmattrs(struct xfs_inode *ip, u_int evmask, u_int16_t state); -int xfs_reclaim(struct xfs_inode *ip); int xfs_change_file_space(struct xfs_inode *ip, int cmd, xfs_flock64_t *bf, xfs_off_t offset, int attr_flags); int xfs_rename(struct xfs_inode *src_dp, struct xfs_name *src_name, --------------090506040001040402060507-- From BATV+f4938f14c6ec46ffcbb1+2361+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 9 04:30:51 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,LOCAL_GNU_PATCH autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19AUm5m208889 for ; Tue, 9 Feb 2010 04:30:50 -0600 X-ASG-Debug-ID: 1265711521-3c2f00770000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id B26441BC9F4 for ; Tue, 9 Feb 2010 02:32:01 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id OqFIXf5ZqnsuQGjV for ; Tue, 09 Feb 2010 02:32:01 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NenNu-0002aR-1T; Tue, 09 Feb 2010 10:31:58 +0000 Date: Tue, 9 Feb 2010 05:31:57 -0500 From: Christoph Hellwig To: Patrick Schreurs Cc: Christoph Hellwig , Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Message-ID: <20100209103157.GA5197@infradead.org> References: <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> <4B712166.9010701@news-service.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B712166.9010701@news-service.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265711521 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 09, 2010 at 09:48:38AM +0100, Patrick Schreurs wrote: > This is a clean 2.6.32.3 with the xfs-inode-reclaim-2.6.32 patch i > received from Dave on January 8th (see attachment). I can't find anything interesting regarding I_RECLAIMABLE manipulation in there. The only thing I could think off going wrong is i_flags and i_update_core sitting in the same word and the compiler causing some read-modify-write cycles for it. Can you test the patch below? It fixes the abose issue up, and to make sure sure the assert you hit isn't as lethal changes it into a WARN_ON, which will still print the backtrace, but not crash the machine. Index: linux-2.6/fs/xfs/linux-2.6/xfs_super.c =================================================================== --- linux-2.6.orig/fs/xfs/linux-2.6/xfs_super.c 2010-02-09 10:38:51.771004413 +0100 +++ linux-2.6/fs/xfs/linux-2.6/xfs_super.c 2010-02-09 10:42:02.102254796 +0100 @@ -1004,13 +1004,13 @@ xfs_fs_inode_init_once( * Dirty the XFS inode when mark_inode_dirty_sync() is called so that * we catch unlogged VFS level updates to the inode. Care must be taken * here - the transaction code calls mark_inode_dirty_sync() to mark the - * VFS inode dirty in a transaction and clears the i_update_core field; + * VFS inode dirty in a transaction and clears the XFS_IDIRTY_CORE flag; * it must clear the field after calling mark_inode_dirty_sync() to * correctly indicate that the dirty state has been propagated into the * inode log item. * * We need the barrier() to maintain correct ordering between unlogged - * updates and the transaction commit code that clears the i_update_core + * updates and the transaction commit code that clears the XFS_IDIRTY_CORE * field. This requires all updates to be completed before marking the * inode dirty. */ @@ -1018,8 +1018,7 @@ STATIC void xfs_fs_dirty_inode( struct inode *inode) { - barrier(); - XFS_I(inode)->i_update_core = 1; + xfs_iflags_set(XFS_I(inode), XFS_IDIRTY_CORE); } /* Index: linux-2.6/fs/xfs/xfs_iget.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_iget.c 2010-02-09 10:38:41.343004133 +0100 +++ linux-2.6/fs/xfs/xfs_iget.c 2010-02-09 10:38:47.069003783 +0100 @@ -81,7 +81,6 @@ xfs_inode_alloc( ip->i_afp = NULL; memset(&ip->i_df, 0, sizeof(xfs_ifork_t)); ip->i_flags = 0; - ip->i_update_core = 0; ip->i_delayed_blks = 0; memset(&ip->i_d, 0, sizeof(xfs_icdinode_t)); ip->i_size = 0; Index: linux-2.6/fs/xfs/xfs_inode.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_inode.c 2010-02-09 10:37:28.821254796 +0100 +++ linux-2.6/fs/xfs/xfs_inode.c 2010-02-09 10:38:33.537253468 +0100 @@ -2098,7 +2098,7 @@ xfs_ifree_cluster( iip = ip->i_itemp; if (!iip) { - ip->i_update_core = 0; + xfs_iflags_clear(ip, XFS_IDIRTY_CORE); xfs_ifunlock(ip); xfs_iunlock(ip, XFS_ILOCK_EXCL); continue; @@ -2913,7 +2913,7 @@ xfs_iflush( * to disk, because the log record didn't make it to disk! */ if (XFS_FORCED_SHUTDOWN(mp)) { - ip->i_update_core = 0; + xfs_iflags_clear(ip, XFS_IDIRTY_CORE); if (iip) iip->ili_format.ilf_fields = 0; xfs_ifunlock(ip); @@ -3057,19 +3057,18 @@ xfs_iflush_int( dip = (xfs_dinode_t *)xfs_buf_offset(bp, ip->i_imap.im_boffset); /* - * Clear i_update_core before copying out the data. + * Clear XFS_IDIRTY_CORE before copying out the data. * This is for coordination with our timestamp updates * that don't hold the inode lock. They will always - * update the timestamps BEFORE setting i_update_core, - * so if we clear i_update_core after they set it we + * update the timestamps BEFORE setting XFS_IDIRTY_CORE, + * so if we clear XFS_IDIRTY_CORE after they set it we * are guaranteed to see their updates to the timestamps. * I believe that this depends on strongly ordered memory * semantics, but we have that. We use the SYNCHRONIZE * macro to make sure that the compiler does not reorder - * the i_update_core access below the data copy below. + * the XFS_IDIRTY_CORE access below the data copy below. */ - ip->i_update_core = 0; - SYNCHRONIZE(); + xfs_iflags_clear(ip, XFS_IDIRTY_CORE); /* * Make sure to get the latest timestamps from the Linux inode. @@ -3235,7 +3234,7 @@ xfs_iflush_int( } else { /* * We're flushing an inode which is not in the AIL and has - * not been logged but has i_update_core set. For this + * not been logged but has XFS_IDIRTY_CORE set. For this * case we can use a B_DELWRI flush and immediately drop * the inode flush lock because we can avoid the whole * AIL state thing. It's OK to drop the flush lock now, Index: linux-2.6/fs/xfs/xfs_inode.h =================================================================== --- linux-2.6.orig/fs/xfs/xfs_inode.h 2010-02-09 10:36:31.621003922 +0100 +++ linux-2.6/fs/xfs/xfs_inode.h 2010-02-09 10:37:23.518004062 +0100 @@ -259,8 +259,7 @@ typedef struct xfs_inode { wait_queue_head_t i_ipin_wait; /* inode pinning wait queue */ spinlock_t i_flags_lock; /* inode i_flags lock */ /* Miscellaneous state. */ - unsigned short i_flags; /* see defined flags below */ - unsigned char i_update_core; /* timestamps/size is dirty */ + unsigned int i_flags; /* see defined flags below */ unsigned int i_delayed_blks; /* count of delay alloc blks */ xfs_icdinode_t i_d; /* most of ondisk inode */ @@ -391,6 +390,7 @@ static inline void xfs_ifunlock(xfs_inod #define XFS_INEW 0x0008 /* inode has just been allocated */ #define XFS_IFILESTREAM 0x0010 /* inode is in a filestream directory */ #define XFS_ITRUNCATED 0x0020 /* truncated down so flush-on-close */ +#define XFS_IDIRTY_CORE 0x0040 /* non-transaction updates pending */ /* * Flags for inode locking. Index: linux-2.6/fs/xfs/xfs_inode_item.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_inode_item.c 2010-02-09 10:40:38.194253398 +0100 +++ linux-2.6/fs/xfs/xfs_inode_item.c 2010-02-09 10:41:44.883256052 +0100 @@ -233,15 +233,15 @@ xfs_inode_item_format( /* * Make sure the linux inode is dirty. We do this before - * clearing i_update_core as the VFS will call back into - * XFS here and set i_update_core, so we need to dirty the - * inode first so that the ordering of i_update_core and + * clearing XFS_IDIRTY_CORE as the VFS will call back into + * XFS here and set XFS_IDIRTY_CORE, so we need to dirty the + * inode first so that the ordering of XFS_IDIRTY_CORE and * unlogged modifications still works as described below. */ xfs_mark_inode_dirty_sync(ip); /* - * Clear i_update_core if the timestamps (or any other + * Clear XFS_IDIRTY_CORE if the timestamps (or any other * non-transactional modification) need flushing/logging * and we're about to log them with the rest of the core. * @@ -252,11 +252,11 @@ xfs_inode_item_format( * for the timestamps if both routines were to grab the * timestamps or not. That would be ok. * - * We clear i_update_core before copying out the data. + * We clear XFS_IDIRTY_CORE before copying out the data. * This is for coordination with our timestamp updates * that don't hold the inode lock. They will always - * update the timestamps BEFORE setting i_update_core, - * so if we clear i_update_core after they set it we + * update the timestamps BEFORE setting XFS_IDIRTY_CORE, + * so if we clear XFS_IDIRTY_CORE after they set it we * are guaranteed to see their updates to the timestamps * either here. Likewise, if they set it after we clear it * here, we'll see it either on the next commit of this @@ -264,12 +264,9 @@ xfs_inode_item_format( * xfs_iflush(). This depends on strongly ordered memory * semantics, but we have that. We use the SYNCHRONIZE * macro to make sure that the compiler does not reorder - * the i_update_core access below the data copy below. + * the XFS_IDIRTY_CORE access below the data copy below. */ - if (ip->i_update_core) { - ip->i_update_core = 0; - SYNCHRONIZE(); - } + xfs_iflags_clear(ip, XFS_IDIRTY_CORE); /* * Make sure to get the latest timestamps from the Linux inode. Index: linux-2.6/fs/xfs/xfs_inode_item.h =================================================================== --- linux-2.6.orig/fs/xfs/xfs_inode_item.h 2010-02-09 10:40:24.678024386 +0100 +++ linux-2.6/fs/xfs/xfs_inode_item.h 2010-02-09 10:40:35.674015377 +0100 @@ -162,7 +162,7 @@ static inline int xfs_inode_clean(xfs_in { return (!ip->i_itemp || !(ip->i_itemp->ili_format.ilf_fields & XFS_ILOG_ALL)) && - !ip->i_update_core; + !xfs_iflags_test(ip, XFS_IDIRTY_CORE); } extern void xfs_inode_item_init(struct xfs_inode *, struct xfs_mount *); Index: linux-2.6/fs/xfs/xfs_vnodeops.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_vnodeops.c 2010-02-09 10:39:37.171274212 +0100 +++ linux-2.6/fs/xfs/xfs_vnodeops.c 2010-02-09 10:40:13.425004481 +0100 @@ -405,7 +405,7 @@ xfs_setattr( inode->i_atime = iattr->ia_atime; ip->i_d.di_atime.t_sec = iattr->ia_atime.tv_sec; ip->i_d.di_atime.t_nsec = iattr->ia_atime.tv_nsec; - ip->i_update_core = 1; + xfs_iflags_set(ip, XFS_IDIRTY_CORE); } if (mask & ATTR_MTIME) { inode->i_mtime = iattr->ia_mtime; @@ -427,7 +427,7 @@ xfs_setattr( inode->i_ctime = iattr->ia_ctime; ip->i_d.di_ctime.t_sec = iattr->ia_ctime.tv_sec; ip->i_d.di_ctime.t_nsec = iattr->ia_ctime.tv_nsec; - ip->i_update_core = 1; + xfs_iflags_set(ip, XFS_IDIRTY_CORE); timeflags &= ~XFS_ICHGTIME_CHG; } @@ -633,7 +633,7 @@ xfs_fsync( */ xfs_ilock(ip, XFS_ILOCK_SHARED); - if (!ip->i_update_core) { + if (!xfs_iflags_test(ip, XFS_IDIRTY_CORE)) { /* * Timestamps/size haven't changed since last inode flush or * inode transaction commit. That means either nothing got Index: linux-2.6/fs/xfs/linux-2.6/xfs_sync.c =================================================================== --- linux-2.6.orig/fs/xfs/linux-2.6/xfs_sync.c 2010-02-09 10:43:25.201003853 +0100 +++ linux-2.6/fs/xfs/linux-2.6/xfs_sync.c 2010-02-09 10:44:00.886284060 +0100 @@ -765,8 +765,8 @@ xfs_reclaim_inode_now( * XFS_IRECLAIM flag set it will not touch us. */ spin_lock(&ip->i_flags_lock); - ASSERT_ALWAYS(__xfs_iflags_test(ip, XFS_IRECLAIMABLE)); - if (__xfs_iflags_test(ip, XFS_IRECLAIM)) { + if (WARN_ON(__xfs_iflags_test(ip, XFS_IRECLAIMABLE)) || + __xfs_iflags_test(ip, XFS_IRECLAIM)) { /* ignore as it is already under reclaim */ spin_unlock(&ip->i_flags_lock); write_unlock(&pag->pag_ici_lock); From aluno3@poczta.onet.pl Tue Feb 9 09:23:57 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.4 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, KB_DATE_CONTAINS_TAB,RCVD_IN_BRBL autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19FNu6S226873 for ; Tue, 9 Feb 2010 09:23:56 -0600 X-ASG-Debug-ID: 1265729105-0ddf01b50000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtpout5.poczta.onet.pl (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E50FB138D586 for ; Tue, 9 Feb 2010 07:25:05 -0800 (PST) Received: from smtpout5.poczta.onet.pl (smtpout5.poczta.onet.pl [213.180.147.165]) by cuda.sgi.com with ESMTP id LGWsWH1P1OnCUa41 for ; Tue, 09 Feb 2010 07:25:05 -0800 (PST) Received: from ip-83-238-22-2.netia.com.pl ([83.238.22.2]:53060 "EHLO [192.168.242.3]" rhost-flags-OK-FAIL-OK-FAIL) by ps1.m5r2.onet with ESMTPA id S50343186Ab0BIPS419vuR (ORCPT ); Tue, 9 Feb 2010 16:18:56 +0100 Message-ID: <4B717CCD.3040008@poczta.onet.pl> Date: Tue, 09 Feb 2010 16:18:37 +0100 From: "aluno3@poczta.onet.pl" User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: xfs@oss.sgi.com X-ASG-Orig-Subj: XFS problems with 2.6.27 Subject: XFS problems with 2.6.27 X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------090005010608050509040804" X-Barracuda-Connect: smtpout5.poczta.onet.pl[213.180.147.165] X-Barracuda-Start-Time: 1265729107 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22091 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This is a multi-part message in MIME format. --------------090005010608050509040804 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hello! We"ve got two different systems running with XFS, one of them is publishing shares via NFS, the other via Samba. We've recently encountered filesystem issues on both servers, and received the following call traces. They look very much alike. Are these something you are familiar with? We'd like to at least establish that the problem really is with XFS... We don't really have the possibility to update the kernels (2.6.27.10) on those machines, but we can possibly apply patches etc. These systems are usually under a fairly high load and host a large number of files. Filesystem "dm-16": Disabling barriers, trial barrier write failed XFS mounting filesystem dm-16 Starting XFS recovery on filesystem: dm-16 (logdev: internal) XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1590 of file fs/xfs/xfs_alloc.c. Caller 0xffffffff80398ca7 Pid: 15745, comm: mount Not tainted 2.6.27.10 #24 Call Trace: [] xfs_free_ag_extent+0x378/0x740 [] xfs_free_extent+0xc7/0x110 [] xlog_recover_process_efi+0x122/0x1a0 [] xlog_recover_process_efis+0x67/0x90 [] xlog_recover_finish+0x1c/0xd0 [] xfs_log_mount_finish+0x20/0x30 [] xfs_mountfs+0x2bd/0x640 [] xfs_fstrm_free_func+0x0/0x90 [] xfs_mru_cache_create+0x15a/0x1c0 [] xfs_fs_fill_super+0x214/0x450 [] set_bdev_super+0x0/0x10 [] get_sb_bdev+0x13f/0x180 [] xfs_fs_fill_super+0x0/0x450 [] vfs_kern_mount+0x81/0x160 [] do_kern_mount+0x4d/0x110 [] do_new_mount+0x9b/0xe0 [] do_mount+0x209/0x220 [] __alloc_pages_internal+0x92/0x430 [] __get_free_pages+0x17/0x80 [] compat_sys_mount+0xa4/0x270 [] ia32_sysret+0x0/0xa Failed to recover EFIs on filesystem: dm-16 XFS: log mount finish failed NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory NFSD: starting 90-second grace period bootsplash: status on console 0 changed to off usb 3-2: USB disconnect, address 2 00000000: 00 00 00 00 00 28 00 e0 00 00 00 00 00 00 00 00 .....(.......... Filesystem "dm-20": XFS internal error xfs_da_do_buf(2) at line 2112 of file fs/xfs/xfs_da_btree.c. Caller 0xffffffff803b34b4 Pid: 23023, comm: smbd Not tainted 2.6.27.10 #24 Call Trace: [] xfs_da_do_buf+0x626/0x6b0 [] xfs_da_read_buf+0x24/0x30 [] try_to_del_timer_sync+0x4f/0x60 [] xfs_da_read_buf+0x24/0x30 [] xfs_dir2_block_lookup_int+0x45/0x1a0 [] xfs_dir2_block_lookup_int+0x45/0x1a0 [] xfs_dir2_block_lookup+0x18/0xc0 [] xfs_dir2_isblock+0x1f/0x60 [] xfs_dir_lookup+0x19d/0x1c0 [] xfs_lookup+0x57/0xd0 [] _spin_lock_bh+0x9/0x20 [] xfs_vn_lookup+0x64/0xc0 [] d_alloc+0x125/0x1b0 [] do_lookup+0x175/0x220 [] generic_permission+0x69/0x130 [] __link_path_walk+0x801/0xdb0 [] sock_aio_read+0x163/0x170 [] path_walk+0x57/0xb0 [] do_path_lookup+0x123/0x1b0 [] user_path_at+0x44/0x80 [] autoremove_wake_function+0x0/0x30 [] mntput_no_expire+0x21/0x120 [] vfs_stat_fd+0x23/0x80 [] sys32_stat64+0x1f/0x70 [] ia32_sysret+0x0/0xa 00000000: 00 00 00 00 00 28 00 e0 00 00 00 00 00 00 00 00 .....(.......... Filesystem "dm-20": XFS internal error xfs_da_do_buf(2) at line 2112 of file fs/xfs/xfs_da_btree.c. Caller 0xffffffff803b34b4 Pid: 23023, comm: smbd Not tainted 2.6.27.10 #24 Call Trace: [] xfs_da_do_buf+0x626/0x6b0 [] xfs_da_read_buf+0x24/0x30 [] try_to_del_timer_sync+0x4f/0x60 [] xfs_da_read_buf+0x24/0x30 [] xfs_dir2_block_lookup_int+0x45/0x1a0 [] xfs_dir2_block_lookup_int+0x45/0x1a0 [] xfs_dir2_block_lookup+0x18/0xc0 [] xfs_dir2_isblock+0x1f/0x60 [] xfs_dir_lookup+0x19d/0x1c0 [] xfs_lookup+0x57/0xd0 [] _spin_lock_bh+0x9/0x20 [] xfs_vn_lookup+0x64/0xc0 [] d_alloc+0x125/0x1b0 [] do_lookup+0x175/0x220 [] generic_permission+0x69/0x130 [] __link_path_walk+0x801/0xdb0 [] sock_aio_read+0x163/0x170 [] path_walk+0x57/0xb0 [] do_path_lookup+0x123/0x1b0 [] user_path_at+0x44/0x80 [] autoremove_wake_function+0x0/0x30 [] mntput_no_expire+0x21/0x120 [] vfs_stat_fd+0x23/0x80 [] sys32_stat64+0x1f/0x70 [] ia32_sysret+0x0/0xa >From the other system we only have these logs: 2010/01/27 00:22:07|Pid: 17209, comm: nfsd Not tainted 2.6.27.10 #12 2010/01/27 00:22:07| 2010/01/27 00:22:07|Call Trace: 2010/01/27 00:22:07|[] xfs_da_do_buf+0x626/0x6b0 2010/01/27 00:22:07|[] xfs_da_read_buf+0x24/0x30 2010/01/27 00:22:07|[] xfs_buf_read_flags+0x67/0x90 2010/01/27 00:22:07|[] xfs_trans_read_buf+0x147/0x310 2010/01/27 00:22:07|[] xfs_da_read_buf+0x24/0x30 2010/01/27 00:22:07|[] xfs_da_node_lookup_int+0x88/0x2a0 2010/01/27 00:22:07|[] xfs_da_node_lookup_int+0x88/0x2a0 2010/01/27 00:22:07|[] xfs_dir2_node_lookup+0x48/0x120 2010/01/27 00:22:07|[] xfs_dir_lookup+0x1aa/0x1c0 2010/01/27 00:22:07|[] xfs_lookup+0x57/0xd0 2010/01/27 00:22:07|[] nfsd_permission+0x7e/0x130 2010/01/27 00:22:07|[] xfs_vn_lookup+0x64/0xc0 2010/01/27 00:22:07|[] d_alloc+0x125/0x1b0 2010/01/27 00:22:07|[] __lookup_hash+0xec/0x180 2010/01/27 00:22:07|[] lookup_one_len+0x59/0x60 2010/01/27 00:22:07|[] nfsd_lookup_dentry+0x12c/0x4d0 2010/01/27 00:22:07|[] nfsd_lookup+0x30/0x100 2010/01/27 00:22:07|[] nfsd3_proc_lookup+0xa7/0x100 2010/01/27 00:22:07|[] nfsd_dispatch+0xae/0x260 2010/01/27 00:22:07|[] svc_process+0x307/0x740 2010/01/27 00:22:07|[] nfsd+0x172/0x2a0 2010/01/27 00:22:07|[] nfsd+0x0/0x2a0 2010/01/27 00:22:07|[] kthread+0x6c/0xa0 2010/01/27 00:22:07|[] child_rip+0xa/0x11 2010/01/27 00:22:07|[] kthread+0x0/0xa0 2010/01/27 00:22:07|[] child_rip+0x0/0x11 2010/01/27 00:22:07| 2010/01/27 00:22:09|Pid: 17189, comm: nfsd Not tainted 2.6.27.10 #12 2010/01/27 00:22:09| 2010/01/27 00:22:09|Call Trace: 2010/01/27 00:22:09|[] xfs_da_do_buf+0x626/0x6b0 2010/01/27 00:22:09|[] xfs_da_read_buf+0x24/0x30 2010/01/27 00:22:09|[] xfs_da_buf_make+0x13d/0x150 2010/01/27 00:22:09|[] xfs_trans_read_buf+0x147/0x310 2010/01/27 00:22:09|[] xfs_da_read_buf+0x24/0x30 2010/01/27 00:22:09|[] xfs_dir2_leafn_lookup_for_entry+0x16b/0x350 2010/01/27 00:22:09|[] xfs_dir2_leafn_lookup_for_entry+0x16b/0x350 2010/01/27 00:22:09|[] xfs_da_node_lookup_int+0x237/0x2a0 2010/01/27 00:22:09|[] xfs_dir2_node_lookup+0x48/0x120 2010/01/27 00:22:09|[] xfs_dir_lookup+0x1aa/0x1c0 2010/01/27 00:22:09|[] xfs_lookup+0x57/0xd0 2010/01/27 00:22:09|[] nfsd_permission+0x7e/0x130 2010/01/27 00:22:09|[] xfs_vn_lookup+0x64/0xc0 2010/01/27 00:22:09|[] d_alloc+0x125/0x1b0 2010/01/27 00:22:09|[] __lookup_hash+0xec/0x180 2010/01/27 00:22:09|[] lookup_one_len+0x59/0x60 2010/01/27 00:22:09|[] nfsd_lookup_dentry+0x12c/0x4d0 2010/01/27 00:22:09|[] nfsd_lookup+0x30/0x100 2010/01/27 00:22:09|[] nfsd3_proc_lookup+0xa7/0x100 2010/01/27 00:22:09|[] nfsd_dispatch+0xae/0x260 2010/01/27 00:22:09|[] svc_process+0x307/0x740 2010/01/27 00:22:09|[] nfsd+0x172/0x2a0 2010/01/27 00:22:09|[] nfsd+0x0/0x2a0 2010/01/27 00:22:09|[] kthread+0x6c/0xa0 2010/01/27 00:22:09|[] child_rip+0xa/0x11 2010/01/27 00:22:09|[] kthread+0x0/0xa0 2010/01/27 00:22:09|[] child_rip+0x0/0x11 --------------090005010608050509040804 Content-Type: text/plain; name="dmesg1" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="dmesg1" RVhUMyBGUyBvbiByYW0xNSwgaW50ZXJuYWwgam91cm5hbApFWFQzLWZzOiBtb3VudGVkIGZp bGVzeXN0ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4Ka2pvdXJuYWxkIHN0YXJ0aW5nLiAg Q29tbWl0IGludGVydmFsIDUgc2Vjb25kcwpFWFQzLWZzIHdhcm5pbmc6IG1heGltYWwgbW91 bnQgY291bnQgcmVhY2hlZCwgcnVubmluZyBlMmZzY2sgaXMgcmVjb21tZW5kZWQKRVhUMyBG UyBvbiBkbS0xNywgaW50ZXJuYWwgam91cm5hbApFWFQzLWZzOiBtb3VudGVkIGZpbGVzeXN0 ZW0gd2l0aCBvcmRlcmVkIGRhdGEgbW9kZS4KRmlsZXN5c3RlbSAiZG0tMjAiOiBEaXNhYmxp bmcgYmFycmllcnMsIHRyaWFsIGJhcnJpZXIgd3JpdGUgZmFpbGVkClhGUyBtb3VudGluZyBm aWxlc3lzdGVtIGRtLTIwCkVuZGluZyBjbGVhbiBYRlMgbW91bnQgZm9yIGZpbGVzeXN0ZW06 IGRtLTIwCmtqb3VybmFsZCBzdGFydGluZy4gIENvbW1pdCBpbnRlcnZhbCA1IHNlY29uZHMK RVhUMy1mcyB3YXJuaW5nOiBtYXhpbWFsIG1vdW50IGNvdW50IHJlYWNoZWQsIHJ1bm5pbmcg ZTJmc2NrIGlzIHJlY29tbWVuZGVkCkVYVDMgRlMgb24gZG0tMTMsIGludGVybmFsIGpvdXJu YWwKRVhUMy1mczogbW91bnRlZCBmaWxlc3lzdGVtIHdpdGggb3JkZXJlZCBkYXRhIG1vZGUu CkZpbGVzeXN0ZW0gImRtLTE2IjogRGlzYWJsaW5nIGJhcnJpZXJzLCB0cmlhbCBiYXJyaWVy IHdyaXRlIGZhaWxlZApYRlMgbW91bnRpbmcgZmlsZXN5c3RlbSBkbS0xNgpTdGFydGluZyBY RlMgcmVjb3Zlcnkgb24gZmlsZXN5c3RlbTogZG0tMTYgKGxvZ2RldjogaW50ZXJuYWwpClhG UyBpbnRlcm5hbCBlcnJvciBYRlNfV0FOVF9DT1JSVVBURURfR09UTyBhdCBsaW5lIDE1OTAg b2YgZmlsZSBmcy94ZnMveGZzX2FsbG9jLmMuICBDYWxsZXIgMHhmZmZmZmZmZjgwMzk4Y2E3 ClBpZDogMTU3NDUsIGNvbW06IG1vdW50IE5vdCB0YWludGVkIDIuNi4yNy4xMCAjMjQKCkNh bGwgVHJhY2U6CiBbPGZmZmZmZmZmODAzOTZmYjg+XSB4ZnNfZnJlZV9hZ19leHRlbnQrMHgz NzgvMHg3NDAKIFs8ZmZmZmZmZmY4MDM5OGNhNz5dIHhmc19mcmVlX2V4dGVudCsweGM3LzB4 MTEwCiBbPGZmZmZmZmZmODAzZDc2NDI+XSB4bG9nX3JlY292ZXJfcHJvY2Vzc19lZmkrMHgx MjIvMHgxYTAKIFs8ZmZmZmZmZmY4MDNkNzcyNz5dIHhsb2dfcmVjb3Zlcl9wcm9jZXNzX2Vm aXMrMHg2Ny8weDkwCiBbPGZmZmZmZmZmODAzZDg3ZGM+XSB4bG9nX3JlY292ZXJfZmluaXNo KzB4MWMvMHhkMAogWzxmZmZmZmZmZjgwM2QwNzkwPl0geGZzX2xvZ19tb3VudF9maW5pc2gr MHgyMC8weDMwCiBbPGZmZmZmZmZmODAzZGFjM2Q+XSB4ZnNfbW91bnRmcysweDJiZC8weDY0 MAogWzxmZmZmZmZmZjgwM2MwNDQwPl0geGZzX2ZzdHJtX2ZyZWVfZnVuYysweDAvMHg5MAog WzxmZmZmZmZmZjgwM2RiYWNhPl0geGZzX21ydV9jYWNoZV9jcmVhdGUrMHgxNWEvMHgxYzAK IFs8ZmZmZmZmZmY4MDNmMmMwND5dIHhmc19mc19maWxsX3N1cGVyKzB4MjE0LzB4NDUwCiBb PGZmZmZmZmZmODAyOTkxMDA+XSBzZXRfYmRldl9zdXBlcisweDAvMHgxMAogWzxmZmZmZmZm ZjgwMjk5MjVmPl0gZ2V0X3NiX2JkZXYrMHgxM2YvMHgxODAKIFs8ZmZmZmZmZmY4MDNmMjlm MD5dIHhmc19mc19maWxsX3N1cGVyKzB4MC8weDQ1MAogWzxmZmZmZmZmZjgwMjk5NGYxPl0g dmZzX2tlcm5fbW91bnQrMHg4MS8weDE2MAogWzxmZmZmZmZmZjgwMjk5NjFkPl0gZG9fa2Vy bl9tb3VudCsweDRkLzB4MTEwCiBbPGZmZmZmZmZmODAyYjBmYmI+XSBkb19uZXdfbW91bnQr MHg5Yi8weGUwCiBbPGZmZmZmZmZmODAyYjE4NDk+XSBkb19tb3VudCsweDIwOS8weDIyMAog WzxmZmZmZmZmZjgwMjc0ZWEyPl0gX19hbGxvY19wYWdlc19pbnRlcm5hbCsweDkyLzB4NDMw CiBbPGZmZmZmZmZmODAyNzUyZDc+XSBfX2dldF9mcmVlX3BhZ2VzKzB4MTcvMHg4MAogWzxm ZmZmZmZmZjgwMmNiZGY0Pl0gY29tcGF0X3N5c19tb3VudCsweGE0LzB4MjcwCiBbPGZmZmZm ZmZmODAyMjgyODI+XSBpYTMyX3N5c3JldCsweDAvMHhhCgpGYWlsZWQgdG8gcmVjb3ZlciBF RklzIG9uIGZpbGVzeXN0ZW06IGRtLTE2ClhGUzogbG9nIG1vdW50IGZpbmlzaCBmYWlsZWQK TkZTRDogVXNpbmcgL3Zhci9saWIvbmZzL3Y0cmVjb3ZlcnkgYXMgdGhlIE5GU3Y0IHN0YXRl IHJlY292ZXJ5IGRpcmVjdG9yeQpORlNEOiBzdGFydGluZyA5MC1zZWNvbmQgZ3JhY2UgcGVy aW9kCmJvb3RzcGxhc2g6IHN0YXR1cyBvbiBjb25zb2xlIDAgY2hhbmdlZCB0byBvZmYKdXNi IDMtMjogVVNCIGRpc2Nvbm5lY3QsIGFkZHJlc3MgMgowMDAwMDAwMDogMDAgMDAgMDAgMDAg MDAgMjggMDAgZTAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIC4uLi4uKC4uLi4uLi4uLi4K RmlsZXN5c3RlbSAiZG0tMjAiOiBYRlMgaW50ZXJuYWwgZXJyb3IgeGZzX2RhX2RvX2J1Zigy KSBhdCBsaW5lIDIxMTIgb2YgZmlsZSBmcy94ZnMveGZzX2RhX2J0cmVlLmMuICBDYWxsZXIg MHhmZmZmZmZmZjgwM2IzNGI0ClBpZDogMjMwMjMsIGNvbW06IHNtYmQgTm90IHRhaW50ZWQg Mi42LjI3LjEwICMyNAoKQ2FsbCBUcmFjZToKIFs8ZmZmZmZmZmY4MDNiMzNiNj5dIHhmc19k YV9kb19idWYrMHg2MjYvMHg2YjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19kYV9yZWFk X2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDIzZWUwZj5dIHRyeV90b19kZWxfdGltZXJf c3luYysweDRmLzB4NjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19kYV9yZWFkX2J1Zisw eDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDNiNzdlNT5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cF9p bnQrMHg0NS8weDFhMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9v a3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc5NTg+XSB4ZnNfZGlyMl9ibG9j a19sb29rdXArMHgxOC8weGMwCiBbPGZmZmZmZmZmODAzYjYyNGY+XSB4ZnNfZGlyMl9pc2Js b2NrKzB4MWYvMHg2MAogWzxmZmZmZmZmZjgwM2I2OTZkPl0geGZzX2Rpcl9sb29rdXArMHgx OWQvMHgxYzAKIFs8ZmZmZmZmZmY4MDNlMzI2Nz5dIHhmc19sb29rdXArMHg1Ny8weGQwCiBb PGZmZmZmZmZmODA2OWRmZDk+XSBfc3Bpbl9sb2NrX2JoKzB4OS8weDIwCiBbPGZmZmZmZmZm ODAzZWU2ZTQ+XSB4ZnNfdm5fbG9va3VwKzB4NjQvMHhjMAogWzxmZmZmZmZmZjgwMmE5Zjk1 Pl0gZF9hbGxvYysweDEyNS8weDFiMAogWzxmZmZmZmZmZjgwMjlmMTU1Pl0gZG9fbG9va3Vw KzB4MTc1LzB4MjIwCiBbPGZmZmZmZmZmODAyOWVhNzk+XSBnZW5lcmljX3Blcm1pc3Npb24r MHg2OS8weDEzMAogWzxmZmZmZmZmZjgwMjlmYTAxPl0gX19saW5rX3BhdGhfd2FsaysweDgw MS8weGRiMAogWzxmZmZmZmZmZjgwNWRlNjYzPl0gc29ja19haW9fcmVhZCsweDE2My8weDE3 MAogWzxmZmZmZmZmZjgwMmEwMDA3Pl0gcGF0aF93YWxrKzB4NTcvMHhiMAogWzxmZmZmZmZm ZjgwMmEwMTgzPl0gZG9fcGF0aF9sb29rdXArMHgxMjMvMHgxYjAKIFs8ZmZmZmZmZmY4MDJh MDZmND5dIHVzZXJfcGF0aF9hdCsweDQ0LzB4ODAKIFs8ZmZmZmZmZmY4MDI0YTQ1MD5dIGF1 dG9yZW1vdmVfd2FrZV9mdW5jdGlvbisweDAvMHgzMAogWzxmZmZmZmZmZjgwMmFmNWMxPl0g bW50cHV0X25vX2V4cGlyZSsweDIxLzB4MTIwCiBbPGZmZmZmZmZmODAyOWExZjM+XSB2ZnNf c3RhdF9mZCsweDIzLzB4ODAKIFs8ZmZmZmZmZmY4MDIyODc5Zj5dIHN5czMyX3N0YXQ2NCsw eDFmLzB4NzAKIFs8ZmZmZmZmZmY4MDIyODI4Mj5dIGlhMzJfc3lzcmV0KzB4MC8weGEKCjAw MDAwMDAwOiAwMCAwMCAwMCAwMCAwMCAyOCAwMCBlMCAwMCAwMCAwMCAwMCAwMCAwMCAwMCAw MCAgLi4uLi4oLi4uLi4uLi4uLgpGaWxlc3lzdGVtICJkbS0yMCI6IFhGUyBpbnRlcm5hbCBl cnJvciB4ZnNfZGFfZG9fYnVmKDIpIGF0IGxpbmUgMjExMiBvZiBmaWxlIGZzL3hmcy94ZnNf ZGFfYnRyZWUuYy4gIENhbGxlciAweGZmZmZmZmZmODAzYjM0YjQKUGlkOiAyMzAyMywgY29t bTogc21iZCBOb3QgdGFpbnRlZCAyLjYuMjcuMTAgIzI0CgpDYWxsIFRyYWNlOgogWzxmZmZm ZmZmZjgwM2IzM2I2Pl0geGZzX2RhX2RvX2J1ZisweDYyNi8weDZiMAogWzxmZmZmZmZmZjgw M2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgwMjNlZTBm Pl0gdHJ5X3RvX2RlbF90aW1lcl9zeW5jKzB4NGYvMHg2MAogWzxmZmZmZmZmZjgwM2IzNGI0 Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZz X2RpcjJfYmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc3ZTU+ XSB4ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50KzB4NDUvMHgxYTAKIFs8ZmZmZmZmZmY4MDNi Nzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cCsweDE4LzB4YzAKIFs8ZmZmZmZmZmY4MDNi NjI0Zj5dIHhmc19kaXIyX2lzYmxvY2srMHgxZi8weDYwCiBbPGZmZmZmZmZmODAzYjY5NmQ+ XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8weDFjMAogWzxmZmZmZmZmZjgwM2UzMjY3Pl0geGZz X2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZmZmZmZmY4MDY5ZGZkOT5dIF9zcGluX2xvY2tfYmgr MHg5LzB4MjAKIFs8ZmZmZmZmZmY4MDNlZTZlND5dIHhmc192bl9sb29rdXArMHg2NC8weGMw CiBbPGZmZmZmZmZmODAyYTlmOTU+XSBkX2FsbG9jKzB4MTI1LzB4MWIwCiBbPGZmZmZmZmZm ODAyOWYxNTU+XSBkb19sb29rdXArMHgxNzUvMHgyMjAKIFs8ZmZmZmZmZmY4MDI5ZWE3OT5d IGdlbmVyaWNfcGVybWlzc2lvbisweDY5LzB4MTMwCiBbPGZmZmZmZmZmODAyOWZhMDE+XSBf X2xpbmtfcGF0aF93YWxrKzB4ODAxLzB4ZGIwCiBbPGZmZmZmZmZmODA1ZGU2NjM+XSBzb2Nr X2Fpb19yZWFkKzB4MTYzLzB4MTcwCiBbPGZmZmZmZmZmODAyYTAwMDc+XSBwYXRoX3dhbGsr MHg1Ny8weGIwCiBbPGZmZmZmZmZmODAyYTAxODM+XSBkb19wYXRoX2xvb2t1cCsweDEyMy8w eDFiMAogWzxmZmZmZmZmZjgwMmEwNmY0Pl0gdXNlcl9wYXRoX2F0KzB4NDQvMHg4MAogWzxm ZmZmZmZmZjgwMjRhNDUwPl0gYXV0b3JlbW92ZV93YWtlX2Z1bmN0aW9uKzB4MC8weDMwCiBb PGZmZmZmZmZmODAyYWY1YzE+XSBtbnRwdXRfbm9fZXhwaXJlKzB4MjEvMHgxMjAKIFs8ZmZm ZmZmZmY4MDI5YTFmMz5dIHZmc19zdGF0X2ZkKzB4MjMvMHg4MAogWzxmZmZmZmZmZjgwMjI4 NzlmPl0gc3lzMzJfc3RhdDY0KzB4MWYvMHg3MAogWzxmZmZmZmZmZjgwMjI4MjgyPl0gaWEz Ml9zeXNyZXQrMHgwLzB4YQoKMDAwMDAwMDA6IDAwIDAwIDAwIDAwIDAwIDI4IDAwIGUwIDAw IDAwIDAwIDAwIDAwIDAwIDAwIDAwICAuLi4uLiguLi4uLi4uLi4uCkZpbGVzeXN0ZW0gImRt LTIwIjogWEZTIGludGVybmFsIGVycm9yIHhmc19kYV9kb19idWYoMikgYXQgbGluZSAyMTEy IG9mIGZpbGUgZnMveGZzL3hmc19kYV9idHJlZS5jLiAgQ2FsbGVyIDB4ZmZmZmZmZmY4MDNi MzRiNApQaWQ6IDIzMDIzLCBjb21tOiBzbWJkIE5vdCB0YWludGVkIDIuNi4yNy4xMCAjMjQK CkNhbGwgVHJhY2U6CiBbPGZmZmZmZmZmODAzYjMzYjY+XSB4ZnNfZGFfZG9fYnVmKzB4NjI2 LzB4NmIwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMw CiBbPGZmZmZmZmZmODAyYTAwMTY+XSBwYXRoX3dhbGsrMHg2Ni8weGIwCiBbPGZmZmZmZmZm ODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAzYjZk NTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysweDk5LzB4MjAwCiBbPGZmZmZmZmZmODAz YjZkNTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysweDk5LzB4MjAwCiBbPGZmZmZmZmZm ODAzZWI4YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8weDYwCiBbPGZmZmZmZmZmODAzZWI4 YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8weDYwCiBbPGZmZmZmZmZmODAzYjY2MGI+XSB4 ZnNfcmVhZGRpcisweGJiLzB4ZjAKIFs8ZmZmZmZmZmY4MDJjYzMwMD5dIGNvbXBhdF9maWxs ZGlyNjQrMHgwLzB4ZTAKIFs8ZmZmZmZmZmY4MDNlYjliZT5dIHhmc19maWxlX3JlYWRkaXIr MHhiZS8weDE3MAogWzxmZmZmZmZmZjgwMmNjMzAwPl0gY29tcGF0X2ZpbGxkaXI2NCsweDAv MHhlMAogWzxmZmZmZmZmZjgwMmE0ODgwPl0gdmZzX3JlYWRkaXIrMHhjMC8weGUwCiBbPGZm ZmZmZmZmODAyY2M0NmE+XSBjb21wYXRfc3lzX2dldGRlbnRzNjQrMHg4YS8weGYwCiBbPGZm ZmZmZmZmODAyMjgyODI+XSBpYTMyX3N5c3JldCsweDAvMHhhCgowMDAwMDAwMDogMDAgMDAg MDAgMDAgMDAgMjggMDAgZTAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIC4uLi4uKC4uLi4u Li4uLi4KRmlsZXN5c3RlbSAiZG0tMjAiOiBYRlMgaW50ZXJuYWwgZXJyb3IgeGZzX2RhX2Rv X2J1ZigyKSBhdCBsaW5lIDIxMTIgb2YgZmlsZSBmcy94ZnMveGZzX2RhX2J0cmVlLmMuICBD YWxsZXIgMHhmZmZmZmZmZjgwM2IzNGI0ClBpZDogMjMwMjMsIGNvbW06IHNtYmQgTm90IHRh aW50ZWQgMi42LjI3LjEwICMyNAoKQ2FsbCBUcmFjZToKIFs8ZmZmZmZmZmY4MDNiMzNiNj5d IHhmc19kYV9kb19idWYrMHg2MjYvMHg2YjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19k YV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDI0ZTRhNj5dIHVwKzB4MTYvMHg1 MAogWzxmZmZmZmZmZjgwMjM2MWM3Pl0gcmVsZWFzZV9jb25zb2xlX3NlbSsweDFhNy8weDFk MAogWzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxm ZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEw CiBbPGZmZmZmZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50KzB4NDUv MHgxYTAKIFs8ZmZmZmZmZmY4MDQyNTcwMT5dIF9fdXBfcmVhZCsweDIxLzB4YjAKIFs8ZmZm ZmZmZmY4MDNiNzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cCsweDE4LzB4YzAKIFs8ZmZm ZmZmZmY4MDNiNjI0Zj5dIHhmc19kaXIyX2lzYmxvY2srMHgxZi8weDYwCiBbPGZmZmZmZmZm ODAzYjY5NmQ+XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8weDFjMAogWzxmZmZmZmZmZjgwM2Uz MjY3Pl0geGZzX2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZmZmZmZmY4MDNlZTZlND5dIHhmc192 bl9sb29rdXArMHg2NC8weGMwCiBbPGZmZmZmZmZmODAyYTlmOTU+XSBkX2FsbG9jKzB4MTI1 LzB4MWIwCiBbPGZmZmZmZmZmODAyOWYxNTU+XSBkb19sb29rdXArMHgxNzUvMHgyMjAKIFs8 ZmZmZmZmZmY4MDI5ZmEwMT5dIF9fbGlua19wYXRoX3dhbGsrMHg4MDEvMHhkYjAKIFs8ZmZm ZmZmZmY4MDJhMDAwNz5dIHBhdGhfd2FsaysweDU3LzB4YjAKIFs8ZmZmZmZmZmY4MDJhMDE4 Mz5dIGRvX3BhdGhfbG9va3VwKzB4MTIzLzB4MWIwCiBbPGZmZmZmZmZmODAyYTA2ZjQ+XSB1 c2VyX3BhdGhfYXQrMHg0NC8weDgwCiBbPGZmZmZmZmZmODAyYWY1YzE+XSBtbnRwdXRfbm9f ZXhwaXJlKzB4MjEvMHgxMjAKIFs8ZmZmZmZmZmY4MDI5YTI4MD5dIHZmc19sc3RhdF9mZCsw eDIwLzB4ODAKIFs8ZmZmZmZmZmY4MDIyODgwZj5dIHN5czMyX2xzdGF0NjQrMHgxZi8weDcw CiBbPGZmZmZmZmZmODAyMjgyODI+XSBpYTMyX3N5c3JldCsweDAvMHhhCgowMDAwMDAwMDog MDAgMDAgYzAgNDkgMDAgMzAgMDAgYzMgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAgIC4uLkku MC4uLi4uLi4uLi4KRmlsZXN5c3RlbSAiZG0tMjAiOiBYRlMgaW50ZXJuYWwgZXJyb3IgeGZz X2RhX2RvX2J1ZigyKSBhdCBsaW5lIDIxMTIgb2YgZmlsZSBmcy94ZnMveGZzX2RhX2J0cmVl LmMuICBDYWxsZXIgMHhmZmZmZmZmZjgwM2IzNGI0ClBpZDogMjMwNDEsIGNvbW06IHNtYmQg Tm90IHRhaW50ZWQgMi42LjI3LjEwICMyNAoKQ2FsbCBUcmFjZToKIFs8ZmZmZmZmZmY4MDNi MzNiNj5dIHhmc19kYV9kb19idWYrMHg2MjYvMHg2YjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5d IHhmc19kYV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDIzZWUwZj5dIHRyeV90 b19kZWxfdGltZXJfc3luYysweDRmLzB4NjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19k YV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDNiNzdlNT5dIHhmc19kaXIyX2Js b2NrX2xvb2t1cF9pbnQrMHg0NS8weDFhMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2Rp cjJfYmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc5NTg+XSB4 ZnNfZGlyMl9ibG9ja19sb29rdXArMHgxOC8weGMwCiBbPGZmZmZmZmZmODAzYjYyNGY+XSB4 ZnNfZGlyMl9pc2Jsb2NrKzB4MWYvMHg2MAogWzxmZmZmZmZmZjgwM2I2OTZkPl0geGZzX2Rp cl9sb29rdXArMHgxOWQvMHgxYzAKIFs8ZmZmZmZmZmY4MDNlMzI2Nz5dIHhmc19sb29rdXAr MHg1Ny8weGQwCiBbPGZmZmZmZmZmODA2OWRmZDk+XSBfc3Bpbl9sb2NrX2JoKzB4OS8weDIw CiBbPGZmZmZmZmZmODAzZWU2ZTQ+XSB4ZnNfdm5fbG9va3VwKzB4NjQvMHhjMAogWzxmZmZm ZmZmZjgwMmE5Zjk1Pl0gZF9hbGxvYysweDEyNS8weDFiMAogWzxmZmZmZmZmZjgwMjlmMTU1 Pl0gZG9fbG9va3VwKzB4MTc1LzB4MjIwCiBbPGZmZmZmZmZmODAyOWVhNzk+XSBnZW5lcmlj X3Blcm1pc3Npb24rMHg2OS8weDEzMAogWzxmZmZmZmZmZjgwMjlmYTAxPl0gX19saW5rX3Bh dGhfd2FsaysweDgwMS8weGRiMAogWzxmZmZmZmZmZjgwNWRlNjYzPl0gc29ja19haW9fcmVh ZCsweDE2My8weDE3MAogWzxmZmZmZmZmZjgwMmEwMDA3Pl0gcGF0aF93YWxrKzB4NTcvMHhi MAogWzxmZmZmZmZmZjgwMmEwMTgzPl0gZG9fcGF0aF9sb29rdXArMHgxMjMvMHgxYjAKIFs8 ZmZmZmZmZmY4MDJhMDZmND5dIHVzZXJfcGF0aF9hdCsweDQ0LzB4ODAKIFs8ZmZmZmZmZmY4 MDI0YTQ1MD5dIGF1dG9yZW1vdmVfd2FrZV9mdW5jdGlvbisweDAvMHgzMAogWzxmZmZmZmZm ZjgwMmFmNWMxPl0gbW50cHV0X25vX2V4cGlyZSsweDIxLzB4MTIwCiBbPGZmZmZmZmZmODAy OWExZjM+XSB2ZnNfc3RhdF9mZCsweDIzLzB4ODAKIFs8ZmZmZmZmZmY4MDIyODc5Zj5dIHN5 czMyX3N0YXQ2NCsweDFmLzB4NzAKIFs8ZmZmZmZmZmY4MDIyODI4Mj5dIGlhMzJfc3lzcmV0 KzB4MC8weGEKCjAwMDAwMDAwOiAwMCAwMCBjMCA0OSAwMCAzMCAwMCBjMyAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAgLi4uSS4wLi4uLi4uLi4uLgpGaWxlc3lzdGVtICJkbS0yMCI6IFhG UyBpbnRlcm5hbCBlcnJvciB4ZnNfZGFfZG9fYnVmKDIpIGF0IGxpbmUgMjExMiBvZiBmaWxl IGZzL3hmcy94ZnNfZGFfYnRyZWUuYy4gIENhbGxlciAweGZmZmZmZmZmODAzYjM0YjQKUGlk OiAyMzA0MSwgY29tbTogc21iZCBOb3QgdGFpbnRlZCAyLjYuMjcuMTAgIzI0CgpDYWxsIFRy YWNlOgogWzxmZmZmZmZmZjgwM2IzM2I2Pl0geGZzX2RhX2RvX2J1ZisweDYyNi8weDZiMAog WzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZm ZmZmZjgwMjNlZTBmPl0gdHJ5X3RvX2RlbF90aW1lcl9zeW5jKzB4NGYvMHg2MAogWzxmZmZm ZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgw M2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZm ZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50KzB4NDUvMHgxYTAKIFs8 ZmZmZmZmZmY4MDNiNzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cCsweDE4LzB4YzAKIFs8 ZmZmZmZmZmY4MDNiNjI0Zj5dIHhmc19kaXIyX2lzYmxvY2srMHgxZi8weDYwCiBbPGZmZmZm ZmZmODAzYjY5NmQ+XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8weDFjMAogWzxmZmZmZmZmZjgw M2UzMjY3Pl0geGZzX2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZmZmZmZmY4MDY5ZGZkOT5dIF9z cGluX2xvY2tfYmgrMHg5LzB4MjAKIFs8ZmZmZmZmZmY4MDNlZTZlND5dIHhmc192bl9sb29r dXArMHg2NC8weGMwCiBbPGZmZmZmZmZmODAyYTlmOTU+XSBkX2FsbG9jKzB4MTI1LzB4MWIw CiBbPGZmZmZmZmZmODAyOWYxNTU+XSBkb19sb29rdXArMHgxNzUvMHgyMjAKIFs8ZmZmZmZm ZmY4MDI5ZWE3OT5dIGdlbmVyaWNfcGVybWlzc2lvbisweDY5LzB4MTMwCiBbPGZmZmZmZmZm ODAyOWZhMDE+XSBfX2xpbmtfcGF0aF93YWxrKzB4ODAxLzB4ZGIwCiBbPGZmZmZmZmZmODA1 ZGU2NjM+XSBzb2NrX2Fpb19yZWFkKzB4MTYzLzB4MTcwCiBbPGZmZmZmZmZmODAyYTAwMDc+ XSBwYXRoX3dhbGsrMHg1Ny8weGIwCiBbPGZmZmZmZmZmODAyYTAxODM+XSBkb19wYXRoX2xv b2t1cCsweDEyMy8weDFiMAogWzxmZmZmZmZmZjgwMmEwNmY0Pl0gdXNlcl9wYXRoX2F0KzB4 NDQvMHg4MAogWzxmZmZmZmZmZjgwMjRhNDUwPl0gYXV0b3JlbW92ZV93YWtlX2Z1bmN0aW9u KzB4MC8weDMwCiBbPGZmZmZmZmZmODAyYWY1YzE+XSBtbnRwdXRfbm9fZXhwaXJlKzB4MjEv MHgxMjAKIFs8ZmZmZmZmZmY4MDI5YTFmMz5dIHZmc19zdGF0X2ZkKzB4MjMvMHg4MAogWzxm ZmZmZmZmZjgwMjI4NzlmPl0gc3lzMzJfc3RhdDY0KzB4MWYvMHg3MAogWzxmZmZmZmZmZjgw MjI4MjgyPl0gaWEzMl9zeXNyZXQrMHgwLzB4YQoKMDAwMDAwMDA6IDAwIDAwIGMwIDQ5IDAw IDMwIDAwIGMzIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICAuLi5JLjAuLi4uLi4uLi4uCkZp bGVzeXN0ZW0gImRtLTIwIjogWEZTIGludGVybmFsIGVycm9yIHhmc19kYV9kb19idWYoMikg YXQgbGluZSAyMTEyIG9mIGZpbGUgZnMveGZzL3hmc19kYV9idHJlZS5jLiAgQ2FsbGVyIDB4 ZmZmZmZmZmY4MDNiMzRiNApQaWQ6IDIzMDQxLCBjb21tOiBzbWJkIE5vdCB0YWludGVkIDIu Ni4yNy4xMCAjMjQKCkNhbGwgVHJhY2U6CiBbPGZmZmZmZmZmODAzYjMzYjY+XSB4ZnNfZGFf ZG9fYnVmKzB4NjI2LzB4NmIwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9i dWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAyYTAwMTY+XSBwYXRoX3dhbGsrMHg2Ni8weGIw CiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZm ZmZmZmZmODAzYjZkNTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysweDk5LzB4MjAwCiBb PGZmZmZmZmZmODAzYjZkNTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysweDk5LzB4MjAw CiBbPGZmZmZmZmZmODAzZWI4YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8weDYwCiBbPGZm ZmZmZmZmODAzZWI4YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8weDYwCiBbPGZmZmZmZmZm ODAzYjY2MGI+XSB4ZnNfcmVhZGRpcisweGJiLzB4ZjAKIFs8ZmZmZmZmZmY4MDJjYzMwMD5d IGNvbXBhdF9maWxsZGlyNjQrMHgwLzB4ZTAKIFs8ZmZmZmZmZmY4MDNlYjliZT5dIHhmc19m aWxlX3JlYWRkaXIrMHhiZS8weDE3MAogWzxmZmZmZmZmZjgwMmNjMzAwPl0gY29tcGF0X2Zp bGxkaXI2NCsweDAvMHhlMAogWzxmZmZmZmZmZjgwMmE0ODgwPl0gdmZzX3JlYWRkaXIrMHhj MC8weGUwCiBbPGZmZmZmZmZmODAyY2M0NmE+XSBjb21wYXRfc3lzX2dldGRlbnRzNjQrMHg4 YS8weGYwCiBbPGZmZmZmZmZmODAyMjgyODI+XSBpYTMyX3N5c3JldCsweDAvMHhhCgowMDAw MDAwMDogMDAgMDAgYzAgNDkgMDAgMzAgMDAgYzMgMDAgMDAgMDAgMDAgMDAgMDAgMDAgMDAg IC4uLkkuMC4uLi4uLi4uLi4KRmlsZXN5c3RlbSAiZG0tMjAiOiBYRlMgaW50ZXJuYWwgZXJy b3IgeGZzX2RhX2RvX2J1ZigyKSBhdCBsaW5lIDIxMTIgb2YgZmlsZSBmcy94ZnMveGZzX2Rh X2J0cmVlLmMuICBDYWxsZXIgMHhmZmZmZmZmZjgwM2IzNGI0ClBpZDogMjMwNDEsIGNvbW06 IHNtYmQgTm90IHRhaW50ZWQgMi42LjI3LjEwICMyNAoKQ2FsbCBUcmFjZToKIFs8ZmZmZmZm ZmY4MDNiMzNiNj5dIHhmc19kYV9kb19idWYrMHg2MjYvMHg2YjAKIFs8ZmZmZmZmZmY4MDNi MzRiND5dIHhmc19kYV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDI0ZTRhNj5d IHVwKzB4MTYvMHg1MAogWzxmZmZmZmZmZjgwMjM2MWM3Pl0gcmVsZWFzZV9jb25zb2xlX3Nl bSsweDFhNy8weDFkMAogWzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4 MjQvMHgzMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwX2lu dCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9ibG9ja19sb29r dXBfaW50KzB4NDUvMHgxYTAKIFs8ZmZmZmZmZmY4MDQyNTcwMT5dIF9fdXBfcmVhZCsweDIx LzB4YjAKIFs8ZmZmZmZmZmY4MDNiNzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cCsweDE4 LzB4YzAKIFs8ZmZmZmZmZmY4MDNiNjI0Zj5dIHhmc19kaXIyX2lzYmxvY2srMHgxZi8weDYw CiBbPGZmZmZmZmZmODAzYjY5NmQ+XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8weDFjMAogWzxm ZmZmZmZmZjgwM2UzMjY3Pl0geGZzX2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZmZmZmZmY4MDNl ZTZlND5dIHhmc192bl9sb29rdXArMHg2NC8weGMwCiBbPGZmZmZmZmZmODAyYTlmOTU+XSBk X2FsbG9jKzB4MTI1LzB4MWIwCiBbPGZmZmZmZmZmODAyOWYxNTU+XSBkb19sb29rdXArMHgx NzUvMHgyMjAKIFs8ZmZmZmZmZmY4MDI5ZWE3OT5dIGdlbmVyaWNfcGVybWlzc2lvbisweDY5 LzB4MTMwCiBbPGZmZmZmZmZmODAyOWZhMDE+XSBfX2xpbmtfcGF0aF93YWxrKzB4ODAxLzB4 ZGIwCiBbPGZmZmZmZmZmODAyYTAwMDc+XSBwYXRoX3dhbGsrMHg1Ny8weGIwCiBbPGZmZmZm ZmZmODAyYTAxODM+XSBkb19wYXRoX2xvb2t1cCsweDEyMy8weDFiMAogWzxmZmZmZmZmZjgw MmEwNmY0Pl0gdXNlcl9wYXRoX2F0KzB4NDQvMHg4MAogWzxmZmZmZmZmZjgwMmFmNWMxPl0g bW50cHV0X25vX2V4cGlyZSsweDIxLzB4MTIwCiBbPGZmZmZmZmZmODAyOWEyODA+XSB2ZnNf bHN0YXRfZmQrMHgyMC8weDgwCiBbPGZmZmZmZmZmODAyMjg4MGY+XSBzeXMzMl9sc3RhdDY0 KzB4MWYvMHg3MAogWzxmZmZmZmZmZjgwMjI4MjgyPl0gaWEzMl9zeXNyZXQrMHgwLzB4YQoK MDAwMDAwMDA6IDExIDBhIDczIGJiIDBmIGM5IDAxIGMyIDAwIDAwIGZmIGZmIDAwIDAwIGZm IGZmICAuLnMuLi4uLi4uLi4uLi4uCkZpbGVzeXN0ZW0gImRtLTIwIjogWEZTIGludGVybmFs IGVycm9yIHhmc19kYV9kb19idWYoMikgYXQgbGluZSAyMTEyIG9mIGZpbGUgZnMveGZzL3hm c19kYV9idHJlZS5jLiAgQ2FsbGVyIDB4ZmZmZmZmZmY4MDNiMzRiNApQaWQ6IDI3OTA5LCBj b21tOiBzbWJkIE5vdCB0YWludGVkIDIuNi4yNy4xMCAjMjQKCkNhbGwgVHJhY2U6CiBbPGZm ZmZmZmZmODAzYjMzYjY+XSB4ZnNfZGFfZG9fYnVmKzB4NjI2LzB4NmIwCiBbPGZmZmZmZmZm ODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAyM2Vl MGY+XSB0cnlfdG9fZGVsX3RpbWVyX3N5bmMrMHg0Zi8weDYwCiBbPGZmZmZmZmZmODAzYjM0 YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAzYjc3ZTU+XSB4 ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50KzB4NDUvMHgxYTAKIFs8ZmZmZmZmZmY4MDNiNzdl NT5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cF9pbnQrMHg0NS8weDFhMAogWzxmZmZmZmZmZjgw M2I3OTU4Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwKzB4MTgvMHhjMAogWzxmZmZmZmZmZjgw M2I2MjRmPl0geGZzX2RpcjJfaXNibG9jaysweDFmLzB4NjAKIFs8ZmZmZmZmZmY4MDNiNjk2 ZD5dIHhmc19kaXJfbG9va3VwKzB4MTlkLzB4MWMwCiBbPGZmZmZmZmZmODAzZTMyNjc+XSB4 ZnNfbG9va3VwKzB4NTcvMHhkMAogWzxmZmZmZmZmZjgwNjlkZmQ5Pl0gX3NwaW5fbG9ja19i aCsweDkvMHgyMAogWzxmZmZmZmZmZjgwM2VlNmU0Pl0geGZzX3ZuX2xvb2t1cCsweDY0LzB4 YzAKIFs8ZmZmZmZmZmY4MDJhOWY5NT5dIGRfYWxsb2MrMHgxMjUvMHgxYjAKIFs8ZmZmZmZm ZmY4MDI5ZjE1NT5dIGRvX2xvb2t1cCsweDE3NS8weDIyMAogWzxmZmZmZmZmZjgwMjllYTc5 Pl0gZ2VuZXJpY19wZXJtaXNzaW9uKzB4NjkvMHgxMzAKIFs8ZmZmZmZmZmY4MDI5ZmEwMT5d IF9fbGlua19wYXRoX3dhbGsrMHg4MDEvMHhkYjAKIFs8ZmZmZmZmZmY4MDVkZTY2Mz5dIHNv Y2tfYWlvX3JlYWQrMHgxNjMvMHgxNzAKIFs8ZmZmZmZmZmY4MDJhMDAwNz5dIHBhdGhfd2Fs aysweDU3LzB4YjAKIFs8ZmZmZmZmZmY4MDJhMDE4Mz5dIGRvX3BhdGhfbG9va3VwKzB4MTIz LzB4MWIwCiBbPGZmZmZmZmZmODAyYTA2ZjQ+XSB1c2VyX3BhdGhfYXQrMHg0NC8weDgwCiBb PGZmZmZmZmZmODAyNGE0NTA+XSBhdXRvcmVtb3ZlX3dha2VfZnVuY3Rpb24rMHgwLzB4MzAK IFs8ZmZmZmZmZmY4MDJhZjVjMT5dIG1udHB1dF9ub19leHBpcmUrMHgyMS8weDEyMAogWzxm ZmZmZmZmZjgwMjlhMWYzPl0gdmZzX3N0YXRfZmQrMHgyMy8weDgwCiBbPGZmZmZmZmZmODAy Mjg3OWY+XSBzeXMzMl9zdGF0NjQrMHgxZi8weDcwCiBbPGZmZmZmZmZmODAyMjgyODI+XSBp YTMyX3N5c3JldCsweDAvMHhhCgowMDAwMDAwMDogMTEgMGEgNzMgYmIgMGYgYzkgMDEgYzIg MDAgMDAgZmYgZmYgMDAgMDAgZmYgZmYgIC4ucy4uLi4uLi4uLi4uLi4KRmlsZXN5c3RlbSAi ZG0tMjAiOiBYRlMgaW50ZXJuYWwgZXJyb3IgeGZzX2RhX2RvX2J1ZigyKSBhdCBsaW5lIDIx MTIgb2YgZmlsZSBmcy94ZnMveGZzX2RhX2J0cmVlLmMuICBDYWxsZXIgMHhmZmZmZmZmZjgw M2IzNGI0ClBpZDogMjc5MDksIGNvbW06IHNtYmQgTm90IHRhaW50ZWQgMi42LjI3LjEwICMy NAoKQ2FsbCBUcmFjZToKIFs8ZmZmZmZmZmY4MDNiMzNiNj5dIHhmc19kYV9kb19idWYrMHg2 MjYvMHg2YjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19kYV9yZWFkX2J1ZisweDI0LzB4 MzAKIFs8ZmZmZmZmZmY4MDIzZWUwZj5dIHRyeV90b19kZWxfdGltZXJfc3luYysweDRmLzB4 NjAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19kYV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8 ZmZmZmZmZmY4MDNiNzdlNT5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cF9pbnQrMHg0NS8weDFh MAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwX2ludCsweDQ1 LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc5NTg+XSB4ZnNfZGlyMl9ibG9ja19sb29rdXArMHgx OC8weGMwCiBbPGZmZmZmZmZmODAzYjYyNGY+XSB4ZnNfZGlyMl9pc2Jsb2NrKzB4MWYvMHg2 MAogWzxmZmZmZmZmZjgwM2I2OTZkPl0geGZzX2Rpcl9sb29rdXArMHgxOWQvMHgxYzAKIFs8 ZmZmZmZmZmY4MDNlMzI2Nz5dIHhmc19sb29rdXArMHg1Ny8weGQwCiBbPGZmZmZmZmZmODA2 OWRmZDk+XSBfc3Bpbl9sb2NrX2JoKzB4OS8weDIwCiBbPGZmZmZmZmZmODAzZWU2ZTQ+XSB4 ZnNfdm5fbG9va3VwKzB4NjQvMHhjMAogWzxmZmZmZmZmZjgwMmE5Zjk1Pl0gZF9hbGxvYysw eDEyNS8weDFiMAogWzxmZmZmZmZmZjgwMjlmMTU1Pl0gZG9fbG9va3VwKzB4MTc1LzB4MjIw CiBbPGZmZmZmZmZmODAyOWVhNzk+XSBnZW5lcmljX3Blcm1pc3Npb24rMHg2OS8weDEzMAog WzxmZmZmZmZmZjgwMjlmYTAxPl0gX19saW5rX3BhdGhfd2FsaysweDgwMS8weGRiMAogWzxm ZmZmZmZmZjgwNWRlNjYzPl0gc29ja19haW9fcmVhZCsweDE2My8weDE3MAogWzxmZmZmZmZm ZjgwMmEwMDA3Pl0gcGF0aF93YWxrKzB4NTcvMHhiMAogWzxmZmZmZmZmZjgwMmEwMTgzPl0g ZG9fcGF0aF9sb29rdXArMHgxMjMvMHgxYjAKIFs8ZmZmZmZmZmY4MDJhMDZmND5dIHVzZXJf cGF0aF9hdCsweDQ0LzB4ODAKIFs8ZmZmZmZmZmY4MDI0YTQ1MD5dIGF1dG9yZW1vdmVfd2Fr ZV9mdW5jdGlvbisweDAvMHgzMAogWzxmZmZmZmZmZjgwMmFmNWMxPl0gbW50cHV0X25vX2V4 cGlyZSsweDIxLzB4MTIwCiBbPGZmZmZmZmZmODAyOWExZjM+XSB2ZnNfc3RhdF9mZCsweDIz LzB4ODAKIFs8ZmZmZmZmZmY4MDIyODc5Zj5dIHN5czMyX3N0YXQ2NCsweDFmLzB4NzAKIFs8 ZmZmZmZmZmY4MDIyODI4Mj5dIGlhMzJfc3lzcmV0KzB4MC8weGEKCjAwMDAwMDAwOiAxMSAw YSA3MyBiYiAwZiBjOSAwMSBjMiAwMCAwMCBmZiBmZiAwMCAwMCBmZiBmZiAgLi5zLi4uLi4u Li4uLi4uLgpGaWxlc3lzdGVtICJkbS0yMCI6IFhGUyBpbnRlcm5hbCBlcnJvciB4ZnNfZGFf ZG9fYnVmKDIpIGF0IGxpbmUgMjExMiBvZiBmaWxlIGZzL3hmcy94ZnNfZGFfYnRyZWUuYy4g IENhbGxlciAweGZmZmZmZmZmODAzYjM0YjQKUGlkOiAyNzkwOSwgY29tbTogc21iZCBOb3Qg dGFpbnRlZCAyLjYuMjcuMTAgIzI0CgpDYWxsIFRyYWNlOgogWzxmZmZmZmZmZjgwM2IzM2I2 Pl0geGZzX2RhX2RvX2J1ZisweDYyNi8weDZiMAogWzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZz X2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgwMmEwMDE2Pl0gcGF0aF93YWxr KzB4NjYvMHhiMAogWzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQv MHgzMAogWzxmZmZmZmZmZjgwM2I2ZDU5Pl0geGZzX2RpcjJfYmxvY2tfZ2V0ZGVudHMrMHg5 OS8weDIwMAogWzxmZmZmZmZmZjgwM2I2ZDU5Pl0geGZzX2RpcjJfYmxvY2tfZ2V0ZGVudHMr MHg5OS8weDIwMAogWzxmZmZmZmZmZjgwM2ViOGEwPl0geGZzX2hhY2tfZmlsbGRpcisweDAv MHg2MAogWzxmZmZmZmZmZjgwM2ViOGEwPl0geGZzX2hhY2tfZmlsbGRpcisweDAvMHg2MAog WzxmZmZmZmZmZjgwM2I2NjBiPl0geGZzX3JlYWRkaXIrMHhiYi8weGYwCiBbPGZmZmZmZmZm ODAyY2MzMDA+XSBjb21wYXRfZmlsbGRpcjY0KzB4MC8weGUwCiBbPGZmZmZmZmZmODAzZWI5 YmU+XSB4ZnNfZmlsZV9yZWFkZGlyKzB4YmUvMHgxNzAKIFs8ZmZmZmZmZmY4MDJjYzMwMD5d IGNvbXBhdF9maWxsZGlyNjQrMHgwLzB4ZTAKIFs8ZmZmZmZmZmY4MDJhNDg4MD5dIHZmc19y ZWFkZGlyKzB4YzAvMHhlMAogWzxmZmZmZmZmZjgwMmNjNDZhPl0gY29tcGF0X3N5c19nZXRk ZW50czY0KzB4OGEvMHhmMAogWzxmZmZmZmZmZjgwMjI4MjgyPl0gaWEzMl9zeXNyZXQrMHgw LzB4YQoKMDAwMDAwMDA6IDExIDBhIDczIGJiIDBmIGM5IDAxIGMyIDAwIDAwIGZmIGZmIDAw IDAwIGZmIGZmICAuLnMuLi4uLi4uLi4uLi4uCkZpbGVzeXN0ZW0gImRtLTIwIjogWEZTIGlu dGVybmFsIGVycm9yIHhmc19kYV9kb19idWYoMikgYXQgbGluZSAyMTEyIG9mIGZpbGUgZnMv eGZzL3hmc19kYV9idHJlZS5jLiAgQ2FsbGVyIDB4ZmZmZmZmZmY4MDNiMzRiNApQaWQ6IDI3 OTA5LCBjb21tOiBzbWJkIE5vdCB0YWludGVkIDIuNi4yNy4xMCAjMjQKCkNhbGwgVHJhY2U6 CiBbPGZmZmZmZmZmODAzYjMzYjY+XSB4ZnNfZGFfZG9fYnVmKzB4NjI2LzB4NmIwCiBbPGZm ZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZmZmZmZmZm ODAyNGU0YTY+XSB1cCsweDE2LzB4NTAKIFs8ZmZmZmZmZmY4MDIzNjFjNz5dIHJlbGVhc2Vf Y29uc29sZV9zZW0rMHgxYTcvMHgxZDAKIFs8ZmZmZmZmZmY4MDNiMzRiND5dIHhmc19kYV9y ZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4MDNiNzdlNT5dIHhmc19kaXIyX2Jsb2Nr X2xvb2t1cF9pbnQrMHg0NS8weDFhMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJf YmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODA0MjU3MDE+XSBfX3Vw X3JlYWQrMHgyMS8weGIwCiBbPGZmZmZmZmZmODAzYjc5NTg+XSB4ZnNfZGlyMl9ibG9ja19s b29rdXArMHgxOC8weGMwCiBbPGZmZmZmZmZmODAzYjYyNGY+XSB4ZnNfZGlyMl9pc2Jsb2Nr KzB4MWYvMHg2MAogWzxmZmZmZmZmZjgwM2I2OTZkPl0geGZzX2Rpcl9sb29rdXArMHgxOWQv MHgxYzAKIFs8ZmZmZmZmZmY4MDNlMzI2Nz5dIHhmc19sb29rdXArMHg1Ny8weGQwCiBbPGZm ZmZmZmZmODAzZWU2ZTQ+XSB4ZnNfdm5fbG9va3VwKzB4NjQvMHhjMAogWzxmZmZmZmZmZjgw MmE5Zjk1Pl0gZF9hbGxvYysweDEyNS8weDFiMAogWzxmZmZmZmZmZjgwMjlmMTU1Pl0gZG9f bG9va3VwKzB4MTc1LzB4MjIwCiBbPGZmZmZmZmZmODAyOWVhNzk+XSBnZW5lcmljX3Blcm1p c3Npb24rMHg2OS8weDEzMAogWzxmZmZmZmZmZjgwMjlmYTAxPl0gX19saW5rX3BhdGhfd2Fs aysweDgwMS8weGRiMAogWzxmZmZmZmZmZjgwMmEwMDA3Pl0gcGF0aF93YWxrKzB4NTcvMHhi MAogWzxmZmZmZmZmZjgwMmEwMTgzPl0gZG9fcGF0aF9sb29rdXArMHgxMjMvMHgxYjAKIFs8 ZmZmZmZmZmY4MDJhMDZmND5dIHVzZXJfcGF0aF9hdCsweDQ0LzB4ODAKIFs8ZmZmZmZmZmY4 MDJhZjVjMT5dIG1udHB1dF9ub19leHBpcmUrMHgyMS8weDEyMAogWzxmZmZmZmZmZjgwMjlh MjgwPl0gdmZzX2xzdGF0X2ZkKzB4MjAvMHg4MAogWzxmZmZmZmZmZjgwMjI4ODBmPl0gc3lz MzJfbHN0YXQ2NCsweDFmLzB4NzAKIFs8ZmZmZmZmZmY4MDIyODI4Mj5dIGlhMzJfc3lzcmV0 KzB4MC8weGEKCjAwMDAwMDAwOiAwMCAwMCBjMCA0OSAwMCAwMyAwMCAwMyAwMCAwMCAwMCAw MCAwMCAwMCAwMCAwMCAgLi4uSS4uLi4uLi4uLi4uLgpGaWxlc3lzdGVtICJkbS0yMCI6IFhG UyBpbnRlcm5hbCBlcnJvciB4ZnNfZGFfZG9fYnVmKDIpIGF0IGxpbmUgMjExMiBvZiBmaWxl IGZzL3hmcy94ZnNfZGFfYnRyZWUuYy4gIENhbGxlciAweGZmZmZmZmZmODAzYjM0YjQKUGlk OiAzMTMzMywgY29tbTogc21iZCBOb3QgdGFpbnRlZCAyLjYuMjcuMTAgIzI0CgpDYWxsIFRy YWNlOgogWzxmZmZmZmZmZjgwM2IzM2I2Pl0geGZzX2RhX2RvX2J1ZisweDYyNi8weDZiMAog WzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZm ZmZmZjgwMjNlZTBmPl0gdHJ5X3RvX2RlbF90aW1lcl9zeW5jKzB4NGYvMHg2MAogWzxmZmZm ZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3JlYWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgw M2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tfbG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZm ZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50KzB4NDUvMHgxYTAKIFs8 ZmZmZmZmZmY4MDNiNzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1cCsweDE4LzB4YzAKIFs8 ZmZmZmZmZmY4MDNiNjI0Zj5dIHhmc19kaXIyX2lzYmxvY2srMHgxZi8weDYwCiBbPGZmZmZm ZmZmODAzYjY5NmQ+XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8weDFjMAogWzxmZmZmZmZmZjgw M2UzMjY3Pl0geGZzX2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZmZmZmZmY4MDY5ZGZkOT5dIF9z cGluX2xvY2tfYmgrMHg5LzB4MjAKIFs8ZmZmZmZmZmY4MDNlZTZlND5dIHhmc192bl9sb29r dXArMHg2NC8weGMwCiBbPGZmZmZmZmZmODAyYTlmOTU+XSBkX2FsbG9jKzB4MTI1LzB4MWIw CiBbPGZmZmZmZmZmODAyOWYxNTU+XSBkb19sb29rdXArMHgxNzUvMHgyMjAKIFs8ZmZmZmZm ZmY4MDI5ZWE3OT5dIGdlbmVyaWNfcGVybWlzc2lvbisweDY5LzB4MTMwCiBbPGZmZmZmZmZm ODAyOWZhMDE+XSBfX2xpbmtfcGF0aF93YWxrKzB4ODAxLzB4ZGIwCiBbPGZmZmZmZmZmODA1 ZGU2NjM+XSBzb2NrX2Fpb19yZWFkKzB4MTYzLzB4MTcwCiBbPGZmZmZmZmZmODAyYTAwMDc+ XSBwYXRoX3dhbGsrMHg1Ny8weGIwCiBbPGZmZmZmZmZmODAyYTAxODM+XSBkb19wYXRoX2xv b2t1cCsweDEyMy8weDFiMAogWzxmZmZmZmZmZjgwMmEwNmY0Pl0gdXNlcl9wYXRoX2F0KzB4 NDQvMHg4MAogWzxmZmZmZmZmZjgwMjRhNDUwPl0gYXV0b3JlbW92ZV93YWtlX2Z1bmN0aW9u KzB4MC8weDMwCiBbPGZmZmZmZmZmODAyYWY1YzE+XSBtbnRwdXRfbm9fZXhwaXJlKzB4MjEv MHgxMjAKIFs8ZmZmZmZmZmY4MDI5YTFmMz5dIHZmc19zdGF0X2ZkKzB4MjMvMHg4MAogWzxm ZmZmZmZmZjgwMjI4NzlmPl0gc3lzMzJfc3RhdDY0KzB4MWYvMHg3MAogWzxmZmZmZmZmZjgw MjI4MjgyPl0gaWEzMl9zeXNyZXQrMHgwLzB4YQoKMDAwMDAwMDA6IDAwIDAwIGMwIDQ5IDAw IDAzIDAwIDAzIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICAuLi5JLi4uLi4uLi4uLi4uCkZp bGVzeXN0ZW0gImRtLTIwIjogWEZTIGludGVybmFsIGVycm9yIHhmc19kYV9kb19idWYoMikg YXQgbGluZSAyMTEyIG9mIGZpbGUgZnMveGZzL3hmc19kYV9idHJlZS5jLiAgQ2FsbGVyIDB4 ZmZmZmZmZmY4MDNiMzRiNApQaWQ6IDMxMzMzLCBjb21tOiBzbWJkIE5vdCB0YWludGVkIDIu Ni4yNy4xMCAjMjQKCkNhbGwgVHJhY2U6CiBbPGZmZmZmZmZmODAzYjMzYjY+XSB4ZnNfZGFf ZG9fYnVmKzB4NjI2LzB4NmIwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9i dWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAyM2VlMGY+XSB0cnlfdG9fZGVsX3RpbWVyX3N5 bmMrMHg0Zi8weDYwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgy NC8weDMwCiBbPGZmZmZmZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9ibG9ja19sb29rdXBfaW50 KzB4NDUvMHgxYTAKIFs8ZmZmZmZmZmY4MDNiNzdlNT5dIHhmc19kaXIyX2Jsb2NrX2xvb2t1 cF9pbnQrMHg0NS8weDFhMAogWzxmZmZmZmZmZjgwM2I3OTU4Pl0geGZzX2RpcjJfYmxvY2tf bG9va3VwKzB4MTgvMHhjMAogWzxmZmZmZmZmZjgwM2I2MjRmPl0geGZzX2RpcjJfaXNibG9j aysweDFmLzB4NjAKIFs8ZmZmZmZmZmY4MDNiNjk2ZD5dIHhmc19kaXJfbG9va3VwKzB4MTlk LzB4MWMwCiBbPGZmZmZmZmZmODAzZTMyNjc+XSB4ZnNfbG9va3VwKzB4NTcvMHhkMAogWzxm ZmZmZmZmZjgwNjlkZmQ5Pl0gX3NwaW5fbG9ja19iaCsweDkvMHgyMAogWzxmZmZmZmZmZjgw M2VlNmU0Pl0geGZzX3ZuX2xvb2t1cCsweDY0LzB4YzAKIFs8ZmZmZmZmZmY4MDJhOWY5NT5d IGRfYWxsb2MrMHgxMjUvMHgxYjAKIFs8ZmZmZmZmZmY4MDI5ZjE1NT5dIGRvX2xvb2t1cCsw eDE3NS8weDIyMAogWzxmZmZmZmZmZjgwMjllYTc5Pl0gZ2VuZXJpY19wZXJtaXNzaW9uKzB4 NjkvMHgxMzAKIFs8ZmZmZmZmZmY4MDI5ZmEwMT5dIF9fbGlua19wYXRoX3dhbGsrMHg4MDEv MHhkYjAKIFs8ZmZmZmZmZmY4MDVkZTY2Mz5dIHNvY2tfYWlvX3JlYWQrMHgxNjMvMHgxNzAK IFs8ZmZmZmZmZmY4MDJhMDAwNz5dIHBhdGhfd2FsaysweDU3LzB4YjAKIFs8ZmZmZmZmZmY4 MDJhMDE4Mz5dIGRvX3BhdGhfbG9va3VwKzB4MTIzLzB4MWIwCiBbPGZmZmZmZmZmODAyYTA2 ZjQ+XSB1c2VyX3BhdGhfYXQrMHg0NC8weDgwCiBbPGZmZmZmZmZmODAyMmIwNmM+XSBfX2Rl cXVldWVfZW50aXR5KzB4NmMvMHhhMAogWzxmZmZmZmZmZjgwMjBiMWE1Pl0gX19zd2l0Y2hf dG8rMHgzMTUvMHgzODAKIFs8ZmZmZmZmZmY4MDI5YTFmMz5dIHZmc19zdGF0X2ZkKzB4MjMv MHg4MAogWzxmZmZmZmZmZjgwMjI4NzlmPl0gc3lzMzJfc3RhdDY0KzB4MWYvMHg3MAogWzxm ZmZmZmZmZjgwMjI4MjgyPl0gaWEzMl9zeXNyZXQrMHgwLzB4YQoKMDAwMDAwMDA6IDAwIDAw IGMwIDQ5IDAwIDAzIDAwIDAzIDAwIDAwIDAwIDAwIDAwIDAwIDAwIDAwICAuLi5JLi4uLi4u Li4uLi4uCkZpbGVzeXN0ZW0gImRtLTIwIjogWEZTIGludGVybmFsIGVycm9yIHhmc19kYV9k b19idWYoMikgYXQgbGluZSAyMTEyIG9mIGZpbGUgZnMveGZzL3hmc19kYV9idHJlZS5jLiAg Q2FsbGVyIDB4ZmZmZmZmZmY4MDNiMzRiNApQaWQ6IDMxMzMzLCBjb21tOiBzbWJkIE5vdCB0 YWludGVkIDIuNi4yNy4xMCAjMjQKCkNhbGwgVHJhY2U6CiBbPGZmZmZmZmZmODAzYjMzYjY+ XSB4ZnNfZGFfZG9fYnVmKzB4NjI2LzB4NmIwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNf ZGFfcmVhZF9idWYrMHgyNC8weDMwCiBbPGZmZmZmZmZmODAyYTAwMTY+XSBwYXRoX3dhbGsr MHg2Ni8weGIwCiBbPGZmZmZmZmZmODAzYjM0YjQ+XSB4ZnNfZGFfcmVhZF9idWYrMHgyNC8w eDMwCiBbPGZmZmZmZmZmODAzYjZkNTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysweDk5 LzB4MjAwCiBbPGZmZmZmZmZmODAzYjZkNTk+XSB4ZnNfZGlyMl9ibG9ja19nZXRkZW50cysw eDk5LzB4MjAwCiBbPGZmZmZmZmZmODAzZWI4YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8w eDYwCiBbPGZmZmZmZmZmODAzZWI4YTA+XSB4ZnNfaGFja19maWxsZGlyKzB4MC8weDYwCiBb PGZmZmZmZmZmODAzYjY2MGI+XSB4ZnNfcmVhZGRpcisweGJiLzB4ZjAKIFs8ZmZmZmZmZmY4 MDJjYzMwMD5dIGNvbXBhdF9maWxsZGlyNjQrMHgwLzB4ZTAKIFs8ZmZmZmZmZmY4MDNlYjli ZT5dIHhmc19maWxlX3JlYWRkaXIrMHhiZS8weDE3MAogWzxmZmZmZmZmZjgwMmNjMzAwPl0g Y29tcGF0X2ZpbGxkaXI2NCsweDAvMHhlMAogWzxmZmZmZmZmZjgwMmE0ODgwPl0gdmZzX3Jl YWRkaXIrMHhjMC8weGUwCiBbPGZmZmZmZmZmODAyY2M0NmE+XSBjb21wYXRfc3lzX2dldGRl bnRzNjQrMHg4YS8weGYwCiBbPGZmZmZmZmZmODAyMjgyODI+XSBpYTMyX3N5c3JldCsweDAv MHhhCgowMDAwMDAwMDogMDAgMDAgYzAgNDkgMDAgMDMgMDAgMDMgMDAgMDAgMDAgMDAgMDAg MDAgMDAgMDAgIC4uLkkuLi4uLi4uLi4uLi4KRmlsZXN5c3RlbSAiZG0tMjAiOiBYRlMgaW50 ZXJuYWwgZXJyb3IgeGZzX2RhX2RvX2J1ZigyKSBhdCBsaW5lIDIxMTIgb2YgZmlsZSBmcy94 ZnMveGZzX2RhX2J0cmVlLmMuICBDYWxsZXIgMHhmZmZmZmZmZjgwM2IzNGI0ClBpZDogMzEz MzMsIGNvbW06IHNtYmQgTm90IHRhaW50ZWQgMi42LjI3LjEwICMyNAoKQ2FsbCBUcmFjZToK IFs8ZmZmZmZmZmY4MDNiMzNiNj5dIHhmc19kYV9kb19idWYrMHg2MjYvMHg2YjAKIFs8ZmZm ZmZmZmY4MDNiMzRiND5dIHhmc19kYV9yZWFkX2J1ZisweDI0LzB4MzAKIFs8ZmZmZmZmZmY4 MDI0ZTRhNj5dIHVwKzB4MTYvMHg1MAogWzxmZmZmZmZmZjgwMjM2MWM3Pl0gcmVsZWFzZV9j b25zb2xlX3NlbSsweDFhNy8weDFkMAogWzxmZmZmZmZmZjgwM2IzNGI0Pl0geGZzX2RhX3Jl YWRfYnVmKzB4MjQvMHgzMAogWzxmZmZmZmZmZjgwM2I3N2U1Pl0geGZzX2RpcjJfYmxvY2tf bG9va3VwX2ludCsweDQ1LzB4MWEwCiBbPGZmZmZmZmZmODAzYjc3ZTU+XSB4ZnNfZGlyMl9i bG9ja19sb29rdXBfaW50KzB4NDUvMHgxYTAKIFs8ZmZmZmZmZmY4MDQyNTcwMT5dIF9fdXBf cmVhZCsweDIxLzB4YjAKIFs8ZmZmZmZmZmY4MDNiNzk1OD5dIHhmc19kaXIyX2Jsb2NrX2xv b2t1cCsweDE4LzB4YzAKIFs8ZmZmZmZmZmY4MDNiNjI0Zj5dIHhmc19kaXIyX2lzYmxvY2sr MHgxZi8weDYwCiBbPGZmZmZmZmZmODAzYjY5NmQ+XSB4ZnNfZGlyX2xvb2t1cCsweDE5ZC8w eDFjMAogWzxmZmZmZmZmZjgwM2UzMjY3Pl0geGZzX2xvb2t1cCsweDU3LzB4ZDAKIFs8ZmZm ZmZmZmY4MDNlZTZlND5dIHhmc192bl9sb29rdXArMHg2NC8weGMwCiBbPGZmZmZmZmZmODAy YTlmOTU+XSBkX2FsbG9jKzB4MTI1LzB4MWIwCiBbPGZmZmZmZmZmODAyOWYxNTU+XSBkb19s b29rdXArMHgxNzUvMHgyMjAKIFs8ZmZmZmZmZmY4MDI5ZWE3OT5dIGdlbmVyaWNfcGVybWlz c2lvbisweDY5LzB4MTMwCiBbPGZmZmZmZmZmODAyOWZhMDE+XSBfX2xpbmtfcGF0aF93YWxr KzB4ODAxLzB4ZGIwCiBbPGZmZmZmZmZmODAyYTAwMDc+XSBwYXRoX3dhbGsrMHg1Ny8weGIw CiBbPGZmZmZmZmZmODAyYTAxODM+XSBkb19wYXRoX2xvb2t1cCsweDEyMy8weDFiMAogWzxm ZmZmZmZmZjgwMmEwNmY0Pl0gdXNlcl9wYXRoX2F0KzB4NDQvMHg4MAogWzxmZmZmZmZmZjgw MmFmNWMxPl0gbW50cHV0X25vX2V4cGlyZSsweDIxLzB4MTIwCiBbPGZmZmZmZmZmODAyOWEy ODA+XSB2ZnNfbHN0YXRfZmQrMHgyMC8weDgwCiBbPGZmZmZmZmZmODAyMjg4MGY+XSBzeXMz Ml9sc3RhdDY0KzB4MWYvMHg3MAogWzxmZmZmZmZmZjgwMjI4MjgyPl0gaWEzMl9zeXNyZXQr MHgwLzB4YQoK --------------090005010608050509040804-- From sandeen@sandeen.net Tue Feb 9 10:53:52 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_93 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19GrgkE234523 for ; Tue, 9 Feb 2010 10:53:47 -0600 X-ASG-Debug-ID: 1265734494-13bf013b0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A635F1CB906F for ; Tue, 9 Feb 2010 08:54:54 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id BEdKDz0mtnvksb2d for ; Tue, 09 Feb 2010 08:54:54 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 839388F4FF6; Tue, 9 Feb 2010 10:54:54 -0600 (CST) Message-ID: <4B71935E.7090907@sandeen.net> Date: Tue, 09 Feb 2010 10:54:54 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: Theodore Tso , xfs-oss X-ASG-Orig-Subj: [PATCH V2] xfstests: fix up fs_perms test used by 126 Subject: [PATCH V2] xfstests: fix up fs_perms test used by 126 References: <4B6C4E81.6060201@sandeen.net> <20100208194058.GC9527@infradead.org> In-Reply-To: <20100208194058.GC9527@infradead.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265734495 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22098 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Test 126 was failing intermittently for Ted & I; it seems that this is because we were passing an unterminated string to fopen for the mode; I'm not certain why this made it fail, but it's pretty clearly not a good thing to do, and fixing it fixes the test. Rather than passing around characters, do things string-wise, since that is what is ultimately used in fopen(). Reported-by: Theodore Tso Signed-off-by: Eric Sandeen --- V2: Just pass around pointer to passed-in mode argument diff --git a/src/fs_perms.c b/src/fs_perms.c index 2c5e3fa..ea188c4 100644 --- a/src/fs_perms.c +++ b/src/fs_perms.c @@ -42,7 +42,7 @@ int testsetup(mode_t mode, int cuserId, int cgroupId); int testfperm(int userId, int groupId, char* fperm); int main( int argc, char *argv[]) { - char fperm[1]; + char *fperm; int result, exresult=0, cuserId=0, cgroupId=0, userId=0, groupId=0; mode_t mode; @@ -53,7 +53,7 @@ int main( int argc, char *argv[]) { cgroupId = atoi(argv[3]); userId = atoi(argv[4]); groupId = atoi(argv[5]); - fperm[0] = *argv[6]; + fperm = argv[6]; exresult = atoi(argv[7]); break; default: @@ -64,7 +64,7 @@ int main( int argc, char *argv[]) { testsetup(mode,cuserId,cgroupId); result=testfperm(userId,groupId,fperm); system("rm test.file"); - printf("%c a %03o file owned by (%d/%d) as user/group(%d/%d) ",fperm[0],mode,cuserId,cgroupId,userId,groupId); + printf("%s a %03o file owned by (%d/%d) as user/group(%d/%d) ",fperm,mode,cuserId,cgroupId,userId,groupId); if (result == exresult) { printf("PASS\n"); exit(0); @@ -102,8 +102,7 @@ int testfperm(int userId, int groupId, char* fperm) { return(-1); } - switch(tolower(fperm[0])) { - case 'x': + if (!strcmp("x", fperm)) { PID = fork(); if (PID == 0) { execlp("./test.file","test.file",NULL); @@ -114,8 +113,7 @@ int testfperm(int userId, int groupId, char* fperm) { seteuid(0); setegid(0); return(nuthertmpi); - - default: + } else { if((testfile=fopen("test.file",fperm))){ fclose(testfile); seteuid(0); From BATV+f4938f14c6ec46ffcbb1+2361+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 9 11:51:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19Hpr3u238557 for ; Tue, 9 Feb 2010 11:51:54 -0600 X-ASG-Debug-ID: 1265737986-561702210000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5AC2E1BE0B1 for ; Tue, 9 Feb 2010 09:53:07 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 1qvGhlM5Y1zF4JvW for ; Tue, 09 Feb 2010 09:53:07 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeuGn-0002zk-R2; Tue, 09 Feb 2010 17:53:05 +0000 Date: Tue, 9 Feb 2010 12:53:05 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: Christoph Hellwig , Theodore Tso , xfs-oss X-ASG-Orig-Subj: Re: [PATCH V2] xfstests: fix up fs_perms test used by 126 Subject: Re: [PATCH V2] xfstests: fix up fs_perms test used by 126 Message-ID: <20100209175305.GA10833@infradead.org> References: <4B6C4E81.6060201@sandeen.net> <20100208194058.GC9527@infradead.org> <4B71935E.7090907@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B71935E.7090907@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265737987 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 09, 2010 at 10:54:54AM -0600, Eric Sandeen wrote: > Test 126 was failing intermittently for Ted & I; it seems that > this is because we were passing an unterminated string to > fopen for the mode; I'm not certain why this made it fail, > but it's pretty clearly not a good thing to do, and fixing > it fixes the test. > > Rather than passing around characters, do things string-wise, > since that is what is ultimately used in fopen(). > > Reported-by: Theodore Tso > Signed-off-by: Eric Sandeen Looks good, Reviewed-by: Christoph Hellwig From aelder@sgi.com Tue Feb 9 13:08:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19J8sMr243495 for ; Tue, 9 Feb 2010 13:08:54 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay2.corp.sgi.com (Postfix) with ESMTP id 686F530408B; Tue, 9 Feb 2010 11:10:05 -0800 (PST) Received: from [128.162.232.155] ([128.162.232.155]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 9 Feb 2010 13:10:05 -0600 Subject: Re: [PATCH 0/9] Delayed write metadata writeback V5 From: Alex Elder Reply-To: aelder@sgi.com To: Dave Chinner Cc: xfs@oss.sgi.com In-Reply-To: <1265687802-23043-1-git-send-email-david@fromorbit.com> References: <1265687802-23043-1-git-send-email-david@fromorbit.com> Content-Type: text/plain; charset="UTF-8" Date: Tue, 09 Feb 2010 13:10:04 -0600 Message-ID: <1265742604.26394.1.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 09 Feb 2010 19:10:05.0018 (UTC) FILETIME=[7CBC63A0:01CAA9BB] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, 2010-02-09 at 14:56 +1100, Dave Chinner wrote: > While I started with killing async inode writeback, the series has > grown. It's not really limited to inode writeback - it touches dquot > flushing, changes the way the AIL pushes on buffers, adds xfsbufd > sorting for delayed write buffers, adds a real non-blocking mode to > inode reclaim and avoids physical inode writeback from the VFS while > fixing bugs in handling delayed write inodes. Hence this is more > about enabling efficient delayed write metadata than it is able > killing async inode writeback. > > The idea behind this series is to make metadata buffers get > written from xfsbufd via the delayed write queue rather than being > issued asynchronously from all over the place. To do this, async > buffer writeback is almost entirely removed from XFS, replaced > instead by delayed writes and a method to expedite flushing of > delayed write buffers when required. > > The result of funnelling all the buffer IO into a single place > is that we can more tightly control and therefore optimise the > submission of metadata IO. Aggregating the buffers before dispatch > allows much better sort efficiency of the buffers as the sort window > is not limited to the size of the elevator congestion hysteresis > limit. Hence we can approach 100% merge effeciency on large numbers > of buffers when dispatched for IO and greatly reduce the amount > of seeking metadata writeback causes. > > The major change is to the inode flushing and reclaim code. Delayed > write inodes hold the flush lock for much longer than for async > writeback, and hence blocking on the flush lock can cause extremely > long latencies without other mechanisms to expedite the release of > the flush locks. To prevent needing to flush inodes immediately, > all operations are done non-blocking unless synchronous. This > required a significant rework of the inode reclaim code, but it > greatly simplified other pieces of code (e.g. log item pushing). > > Version 5 > - drop the fsync changes to xfs_fs_write_inode() and the associated > locking changes, replace them with a targeted inode logging > function from Christoph Hellwig to fix a performance regression on > fs_mark -S4 workloads on an SSD. > > Version 4 > - rework inode reclaim checks for better legibility > - add warning to reclaim code when delwri flush errors occur > - kill XFS_ITEM_FLUSHING now it is not used > - clean up sync_mode flags being pushed into xfs_iflush() > - kill the now unused xfs_bawrite() function > - include Christoph's fsync cache flush fix > - rework the inode locking and call to xfs_fsync() when doing > synchronous inode writes to close races between the fsync and > the background delwri flush afterwards. > > Version 3 > - rework inode reclaim to: > - separate it from xfs_iflush return values > - provide a non-blocking mode for background operation > - apply delwri buffer promotion tricks to dquot flushing > - kill unneeded dquot flushing flags, similar to inode flushing flag > removal > - fix sync inode flush bug when trying to flush delwri inodes > > Version 2: > - use generic list sort function > - when unmounting, push the delwri buffers first, then do sync inode > reclaim so that reclaim doesn't block for 15 seconds waiting for > delwri inode buffers to be aged and written before the inodes can > be reclaimed. > > Alex, the patch series is available in the git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/dgc/xfs for-2.6.34 I looked over the whole series again and it all looks good to me. I will pull from your for-2.6.34 branch and will post it on OSS after I've tested it a bit. Signed-off-by: Alex Elder -Alex > Christoph Hellwig (2): > xfs: remove invalid barrier optimization from xfs_fsync > xfs: log changed inodes instead of writing them synchronously > > Dave Chinner (7): > xfs: Make inode reclaim states explicit > xfs: Use delayed write for inodes rather than async V2 > xfs: Don't issue buffer IO direct from AIL push V2 > xfs: Sort delayed write buffers before dispatch > xfs: Use delay write promotion for dquot flushing > xfs: kill the unused XFS_QMOPT_* flush flags V2 > xfs: kill xfs_bawrite > > fs/xfs/linux-2.6/xfs_buf.c | 135 ++++++++++++++++++++++++++-------------- > fs/xfs/linux-2.6/xfs_buf.h | 3 +- > fs/xfs/linux-2.6/xfs_super.c | 111 ++++++++++++++++++++++++--------- > fs/xfs/linux-2.6/xfs_sync.c | 138 +++++++++++++++++++++++++++++++++------- > fs/xfs/linux-2.6/xfs_trace.h | 1 + > fs/xfs/quota/xfs_dquot.c | 38 +++++------- > fs/xfs/quota/xfs_dquot_item.c | 87 ++++---------------------- > fs/xfs/quota/xfs_dquot_item.h | 4 - > fs/xfs/quota/xfs_qm.c | 14 ++--- > fs/xfs/xfs_buf_item.c | 64 ++++++++++--------- > fs/xfs/xfs_inode.c | 86 ++------------------------ > fs/xfs/xfs_inode.h | 11 +--- > fs/xfs/xfs_inode_item.c | 108 +++++++------------------------- > fs/xfs/xfs_inode_item.h | 6 -- > fs/xfs/xfs_mount.c | 13 ++++- > fs/xfs/xfs_quota.h | 8 +-- > fs/xfs/xfs_trans.h | 3 +- > fs/xfs/xfs_trans_ail.c | 13 ++-- > fs/xfs/xfs_vnodeops.c | 12 +--- > 19 files changed, 410 insertions(+), 445 deletions(-) > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From sandeen@redhat.com Tue Feb 9 13:25:22 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19JPLA8244488 for ; Tue, 9 Feb 2010 13:25:22 -0600 X-ASG-Debug-ID: 1265743594-759303be0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mx1.redhat.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2F4311390AED for ; Tue, 9 Feb 2010 11:26:34 -0800 (PST) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id LT4z6YojEhjL1Hg9 for ; Tue, 09 Feb 2010 11:26:34 -0800 (PST) Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id o19JQXlQ010455 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 9 Feb 2010 14:26:34 -0500 Received: from liberator.sandeen.net (ovpn01.gateway.prod.ext.phx2.redhat.com [10.5.9.1]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id o19JQVlA020418 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 9 Feb 2010 14:26:33 -0500 Message-ID: <4B71B6E7.1000203@redhat.com> Date: Tue, 09 Feb 2010 13:26:31 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss CC: ext4 development X-ASG-Orig-Subj: [PATCH] xfstests: optionally run all tests under quota Subject: [PATCH] xfstests: optionally run all tests under quota Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1265743595 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22106 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean This patch might be a little heavy handed, but it seems to work; if you set USE_QUOTA=1 in your environment, all tests should be run with quota on and enabled. This will hopefully help shake out some of the ext4 quota problems, although one needs to keep an eye on the console to see whether warnings scroll by. Signed-off-by: Eric Sandeen --- diff --git a/common.rc b/common.rc index 6424871..4fde921 100644 --- a/common.rc +++ b/common.rc @@ -64,6 +64,9 @@ _mount_opts() *) ;; esac + if [ ! -z "$USE_QUOTA" ]; then + export MOUNT_OPTIONS="$MOUNT_OPTIONS -o usrquota,grpquota" + fi } _mkfs_opts() @@ -161,6 +164,10 @@ _test_options() type=$1 TEST_OPTIONS="" + if [ ! -z "$USE_QUOTA" ]; then + TEST_OPTIONS="-o usrquota,grpquota" + fi + if [ "$FSTYP" != "xfs" ]; then return fi @@ -202,6 +209,25 @@ _mount_ops_filter() } +_setup_quota() +{ + mountpoint=$1 + if [ ! -z "$USE_QUOTA" ]; then + case $FSTYP in + xfs) + ;; + ext*|reiserfs) + quotaoff $mountpoint &>/dev/null + quotacheck -u -g $mountpoint + quotaon $mountpoint + ;; + *) + _fail "Don't know how to turn on quota on $FSTYP" + ;; + esac + fi +} + _scratch_mount_options() { _scratch_options mount @@ -212,6 +238,7 @@ _scratch_mount_options() _scratch_mount() { _mount -t $FSTYP `_scratch_mount_options $*` + _setup_quota $SCRATCH_MNT } _scratch_unmount() @@ -229,6 +256,7 @@ _test_mount() { _test_options mount _mount -t $FSTYP $TEST_OPTIONS $TEST_FS_MOUNT_OPTS $* $TEST_DEV $TEST_DIR + _setup_quota $TEST_DIR } _scratch_mkfs_options() From aelder@sgi.com Tue Feb 9 14:21:43 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX, J_CHICKENPOX_12 autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19KLhGW247951 for ; Tue, 9 Feb 2010 14:21:43 -0600 Received: from cf--amer001e--3.americas.sgi.com (cf--amer001e--3.americas.sgi.com [137.38.100.5]) by relay1.corp.sgi.com (Postfix) with ESMTP id 524538F80CE; Tue, 9 Feb 2010 12:22:54 -0800 (PST) Received: from [128.162.232.155] ([128.162.232.155]) by cf--amer001e--3.americas.sgi.com with Microsoft SMTPSVC(6.0.3790.3959); Tue, 9 Feb 2010 14:22:54 -0600 Subject: Re: [PATCH] xfs: only clear the suid bit once in xfs_write From: Alex Elder Reply-To: aelder@sgi.com To: Christoph Hellwig Cc: xfs@oss.sgi.com In-Reply-To: <20100203194331.GA20199@infradead.org> References: <20100203194331.GA20199@infradead.org> Content-Type: text/plain; charset="UTF-8" Date: Tue, 09 Feb 2010 14:22:53 -0600 Message-ID: <1265746973.26394.9.camel@doink1> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 09 Feb 2010 20:22:54.0249 (UTC) FILETIME=[A9002190:01CAA9C5] X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, 2010-02-03 at 14:43 -0500, Christoph Hellwig wrote: > file_remove_suid already calls into ->setattr to clear the suid and sgid > bits if needed, no need to start a second transaction to do it ourselves. > > Note that xfs_write_clear_setuid issues a sync transaction while the > path through ->setattr doesn't, but that is consistant with the other > filesystems. > > Signed-off-by: Christoph Hellwig Looks good. Those S*ID bits sure get checked and cleared in a lot of places... Reviewed-by: Alex Elder > Index: xfs/fs/xfs/linux-2.6/xfs_lrw.c > =================================================================== > --- xfs.orig/fs/xfs/linux-2.6/xfs_lrw.c 2010-02-03 20:37:45.956003749 +0100 > +++ xfs/fs/xfs/linux-2.6/xfs_lrw.c 2010-02-03 20:37:49.348003520 +0100 > @@ -630,18 +630,9 @@ start: > * by root. This keeps people from modifying setuid and > * setgid binaries. > */ > - > - if (((xip->i_d.di_mode & S_ISUID) || > - ((xip->i_d.di_mode & (S_ISGID | S_IXGRP)) == > - (S_ISGID | S_IXGRP))) && > - !capable(CAP_FSETID)) { > - error = xfs_write_clear_setuid(xip); > - if (likely(!error)) > - error = -file_remove_suid(file); > - if (unlikely(error)) { > - goto out_unlock_internal; > - } > - } > + error = -file_remove_suid(file); > + if (unlikely(error)) > + goto out_unlock_internal; > > /* We can write back this queue in page reclaim */ > current->backing_dev_info = mapping->backing_dev_info; > Index: xfs/fs/xfs/xfs_rw.c > =================================================================== > --- xfs.orig/fs/xfs/xfs_rw.c 2010-02-03 20:37:45.965004126 +0100 > +++ xfs/fs/xfs/xfs_rw.c 2010-02-03 20:37:49.349010212 +0100 > @@ -47,48 +47,6 @@ > #include "xfs_trace.h" > > /* > - * This is a subroutine for xfs_write() and other writers (xfs_ioctl) > - * which clears the setuid and setgid bits when a file is written. > - */ > -int > -xfs_write_clear_setuid( > - xfs_inode_t *ip) > -{ > - xfs_mount_t *mp; > - xfs_trans_t *tp; > - int error; > - > - mp = ip->i_mount; > - tp = xfs_trans_alloc(mp, XFS_TRANS_WRITEID); > - if ((error = xfs_trans_reserve(tp, 0, > - XFS_WRITEID_LOG_RES(mp), > - 0, 0, 0))) { > - xfs_trans_cancel(tp, 0); > - return error; > - } > - xfs_ilock(ip, XFS_ILOCK_EXCL); > - xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); > - xfs_trans_ihold(tp, ip); > - ip->i_d.di_mode &= ~S_ISUID; > - > - /* > - * Note that we don't have to worry about mandatory > - * file locking being disabled here because we only > - * clear the S_ISGID bit if the Group execute bit is > - * on, but if it was on then mandatory locking wouldn't > - * have been enabled. > - */ > - if (ip->i_d.di_mode & S_IXGRP) { > - ip->i_d.di_mode &= ~S_ISGID; > - } > - xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); > - xfs_trans_set_sync(tp); > - error = xfs_trans_commit(tp, 0); > - xfs_iunlock(ip, XFS_ILOCK_EXCL); > - return 0; > -} > - > -/* > * Force a shutdown of the filesystem instantly while keeping > * the filesystem consistent. We don't do an unmount here; just shutdown > * the shop, make sure that absolutely nothing persistent happens to > Index: xfs/fs/xfs/xfs_rw.h > =================================================================== > --- xfs.orig/fs/xfs/xfs_rw.h 2010-02-03 20:37:53.085003394 +0100 > +++ xfs/fs/xfs/xfs_rw.h 2010-02-03 20:37:57.946033654 +0100 > @@ -39,7 +39,6 @@ xfs_fsb_to_db(struct xfs_inode *ip, xfs_ > /* > * Prototypes for functions in xfs_rw.c. > */ > -extern int xfs_write_clear_setuid(struct xfs_inode *ip); > extern int xfs_read_buf(struct xfs_mount *mp, xfs_buftarg_t *btp, > xfs_daddr_t blkno, int len, uint flags, > struct xfs_buf **bpp); > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From sandeen@sandeen.net Tue Feb 9 14:42:00 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19Kg0Vh249343 for ; Tue, 9 Feb 2010 14:42:00 -0600 X-ASG-Debug-ID: 1265748192-5ea200630000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1E1861391828 for ; Tue, 9 Feb 2010 12:43:12 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id YcUfyIAEwBRwMyVD for ; Tue, 09 Feb 2010 12:43:12 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id B3C31127664A for ; Tue, 9 Feb 2010 14:43:11 -0600 (CST) Message-ID: <4B71C8DF.1090909@sandeen.net> Date: Tue, 09 Feb 2010 14:43:11 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss X-ASG-Orig-Subj: [PATCH] xfstests: make 204 generic Subject: [PATCH] xfstests: make 204 generic Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265748193 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.32 X-Barracuda-Spam-Status: No, SCORE=-1.32 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MARKETING_SUBJECT, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22112 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean 204 can be generic. Also remove a stray _scratch_mkfs that snuck into _scratch_mkfs_sized :/ Signed-off-by: Eric Sandeen --- diff --git a/204 b/204 index 1ac9ebf..14ebcdc 100755 --- a/204 +++ b/204 @@ -35,12 +35,13 @@ status=1 # failure is the default! . ./common.filter # real QA test starts here -_supported_fs xfs +_supported_fs generic _supported_os Linux _require_scratch -_scratch_mkfs_xfs -d size=104m >/dev/null +SIZE=104*1024*1024 +_scratch_mkfs_sized $SIZE &> /dev/null _scratch_mount for i in `seq 1 22500`; do diff --git a/common.rc b/common.rc index 0a02a2b..c76bcde 100644 --- a/common.rc +++ b/common.rc @@ -325,7 +325,6 @@ _scratch_mkfs_sized() _notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized" ;; esac - _scratch_mkfs } # Emulate an N-data-disk stripe w/ various stripe units From BATV+f4938f14c6ec46ffcbb1+2361+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 9 16:01:49 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19M1k0Y254972 for ; Tue, 9 Feb 2010 16:01:48 -0600 X-ASG-Debug-ID: 1265752979-5e9b02160000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A4F0D139276C for ; Tue, 9 Feb 2010 14:02:59 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id uIWOKpio10amOcas for ; Tue, 09 Feb 2010 14:02:59 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeyAc-0005Im-Pr; Tue, 09 Feb 2010 22:02:58 +0000 Date: Tue, 9 Feb 2010 17:02:58 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs-oss X-ASG-Orig-Subj: Re: [PATCH] xfstests: make 204 generic Subject: Re: [PATCH] xfstests: make 204 generic Message-ID: <20100209220258.GA20187@infradead.org> References: <4B71C8DF.1090909@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B71C8DF.1090909@sandeen.net> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265752979 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 09, 2010 at 02:43:11PM -0600, Eric Sandeen wrote: > 204 can be generic. Looks good. > Also remove a stray _scratch_mkfs that snuck into > _scratch_mkfs_sized :/ Heh, that might have made the ENOSPC tests pass rather easily before.. > Signed-off-by: Eric Sandeen Reviewed-by: Christoph Hellwig From BATV+f4938f14c6ec46ffcbb1+2361+infradead.org+hch@bombadil.srs.infradead.org Tue Feb 9 16:03:31 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19M3UsU255071 for ; Tue, 9 Feb 2010 16:03:30 -0600 X-ASG-Debug-ID: 1265753083-5e9f02340000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 201DF139278A for ; Tue, 9 Feb 2010 14:04:43 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id TkfZl2eMiq8oLhd3 for ; Tue, 09 Feb 2010 14:04:43 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NeyCJ-0005Ms-Ie; Tue, 09 Feb 2010 22:04:43 +0000 Date: Tue, 9 Feb 2010 17:04:43 -0500 From: Christoph Hellwig To: Eric Sandeen Cc: xfs-oss , ext4 development X-ASG-Orig-Subj: Re: [PATCH] xfstests: optionally run all tests under quota Subject: Re: [PATCH] xfstests: optionally run all tests under quota Message-ID: <20100209220443.GB20187@infradead.org> References: <4B71B6E7.1000203@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B71B6E7.1000203@redhat.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265753084 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 09, 2010 at 01:26:31PM -0600, Eric Sandeen wrote: > This patch might be a little heavy handed, but it seems to > work; if you set USE_QUOTA=1 in your environment, all > tests should be run with quota on and enabled. I'd rather prefer a -quota option to ./check than a magic environment variable. From sandeen@sandeen.net Tue Feb 9 16:03:54 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o19M3sN7255107 for ; Tue, 9 Feb 2010 16:03:54 -0600 X-ASG-Debug-ID: 1265753107-2a3f02090000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A494B1BF22A for ; Tue, 9 Feb 2010 14:05:07 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id 2tkeFz5zzTkhnX5X for ; Tue, 09 Feb 2010 14:05:07 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id DC74B12AEBEC; Tue, 9 Feb 2010 16:05:06 -0600 (CST) Message-ID: <4B71DC12.9010408@sandeen.net> Date: Tue, 09 Feb 2010 16:05:06 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Christoph Hellwig CC: xfs-oss X-ASG-Orig-Subj: Re: [PATCH] xfstests: make 204 generic Subject: Re: [PATCH] xfstests: make 204 generic References: <4B71C8DF.1090909@sandeen.net> <20100209220258.GA20187@infradead.org> In-Reply-To: <20100209220258.GA20187@infradead.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265753107 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.32 X-Barracuda-Spam-Status: No, SCORE=-1.32 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=MARKETING_SUBJECT, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22117 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Christoph Hellwig wrote: > On Tue, Feb 09, 2010 at 02:43:11PM -0600, Eric Sandeen wrote: >> 204 can be generic. > > Looks good. > >> Also remove a stray _scratch_mkfs that snuck into >> _scratch_mkfs_sized :/ > > Heh, that might have made the ENOSPC tests pass rather easily before.. Yeah :/ >> Signed-off-by: Eric Sandeen > > Reviewed-by: Christoph Hellwig > Thanks, -eric From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 03:06:41 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1A96c8l035220 for ; Wed, 10 Feb 2010 03:06:41 -0600 X-ASG-Debug-ID: 1265792871-7d1f03590000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 4961A13940A9; Wed, 10 Feb 2010 01:07:51 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id bPQQGEQORHkP3UrK; Wed, 10 Feb 2010 01:07:51 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nf8Y2-00026j-Mn; Wed, 10 Feb 2010 09:07:50 +0000 Date: Wed, 10 Feb 2010 04:07:50 -0500 From: Christoph Hellwig To: Ben Myers Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Subject: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Message-ID: <20100210090750.GB21875@infradead.org> References: <20100210003220.6021.74943.stgit@case> <20100210003337.6021.10942.stgit@case> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100210003337.6021.10942.stgit@case> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265792872 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Tue, Feb 09, 2010 at 06:33:37PM -0600, Ben Myers wrote: > Here is the commit_metadata export_operation for xfs. We take two dentries and > force the log up to the larger lsn. It looks to me that in nfsd the child is > always modified after the parent so generally we expect the child's lsn to be > larger. If that's not the case we'll just force the entire thing. > > The basic form of this is based upon one of Christoph's suggestions. I'm an > xfs newbie so I'm not very comfortable with it yet. My understanding is that I > need to verify that all of the necessary changes make it into the transations > we're forcing into the log here. I am still looking into that and hopefully > the XFS gurus can continue to provide guidance. Ccing the xfs list would help with that :) Anyway, I think it looks pretty good, but there's quite a few smaller nitpicks: > +STATIC int > +xfs_fs_nfs_commit_metadata( > + struct dentry *parent, > + struct dentry *child) > +{ > + struct xfs_inode *p_xip = NULL, *c_xip = NULL; Normal xfs naming would be dp for the parent, and ip for the child, it would be good to stick to that. > + struct xfs_mount *i_mount = NULL; Normal name all over xfs would be mp. > + } else if (parent && child) { > + p_xip = XFS_I(parent->d_inode); > + c_xip = XFS_I(child->d_inode); > + xfs_ilock(p_xip, XFS_ILOCK_SHARED); > + xfs_ilock(c_xip, XFS_ILOCK_SHARED); If we need to lock both parent and child we need to use xfs_lock_two_inodes to make sure the lock order is correct. > + if (xfs_ipincount(c_xip)) { > + /* > + * AFAICS the child is always modified after the parent > + * in nfsd so should always have a larger lsn. > + */ > + if (c_xip->i_itemp->ili_last_lsn > force_lsn) { > + force_lsn = c_xip->i_itemp->ili_last_lsn; > + } else { > + force_lsn = 0; /* whole thing */ > + } I wouldn't rely on that and always take the larger one. Now with the simplification of always having a non-zero first argument suggested in the previous mail this might be simplified down to: STATIC int xfs_fs_nfs_commit_metadata( struct dentry *parent, struct dentry *child) { struct xfs_inode *dp = XFS_I(parent->d_inode); struct xfs_inode *ip = NULL; struct xfs_mount *mp = dp->i_mount; xfs_lsn_t force_lsn = 0; int error = 0; if (child) { ip = XFS_I(child->d_inode); xfs_lock_two_inodes(dp, ip, XFS_ILOCK_SHARED); } else { xfs_ilock(dp, XFS_ILOCK_SHARED); } if (xfs_ipincount(dp)) force_lsn = dp->i_itemp->ili_last_lsn; if (ip && xfs_ipincount(ip)) force_lsn = max(force_lsn, ip->i_itemp->ili_last_lsn); error = _xfs_log_force_lsn(mp, force_lsn, NULL); if (ip) xfs_iunlock(ip, XFS_ILOCK_SHARED); xfs_iunlock(dp, XFS_ILOCK_SHARED); return error; } Note that _xfs_log_force_lsn is new in the XFS tree, mainline still has _xfs_log_force with an lsn argument. Also note that ->commit_metadata probably should take just two inodes instead of two dentires given the level it operates on, but it shouldn't matter too much. From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 04:10:21 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AAAHr1039059 for ; Wed, 10 Feb 2010 04:10:20 -0600 X-ASG-Debug-ID: 1265796690-686302300000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 1EBA31C0C79; Wed, 10 Feb 2010 02:11:30 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id QAH3FjxR5jZP82iI; Wed, 10 Feb 2010 02:11:30 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1Nf9Xe-00051F-FV; Wed, 10 Feb 2010 10:11:30 +0000 Date: Wed, 10 Feb 2010 05:11:30 -0500 From: Christoph Hellwig To: Ben Myers Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Subject: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Message-ID: <20100210101130.GA7993@infradead.org> References: <20100210003220.6021.74943.stgit@case> <20100210003337.6021.10942.stgit@case> <20100210090750.GB21875@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100210090750.GB21875@infradead.org> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265796691 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 10, 2010 at 04:07:50AM -0500, Christoph Hellwig wrote: > > + /* > > + * AFAICS the child is always modified after the parent > > + * in nfsd so should always have a larger lsn. > > + */ > > + if (c_xip->i_itemp->ili_last_lsn > force_lsn) { > > + force_lsn = c_xip->i_itemp->ili_last_lsn; > > + } else { > > + force_lsn = 0; /* whole thing */ > > + } > > I wouldn't rely on that and always take the larger one. Or we could use that fact for making the prototype saner: - the commit_metadata only takes a single inode to force out - we make sure to always call in on the child first. For any log based filesystem that will force the parent update, too. - we then call it on the parent, which will be a no-op and thus fast for a log based filesystem, but still provide a fallback if that is not the case. From patrick@news-service.com Wed Feb 10 06:41:46 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX,J_BACKHAIR_26 autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1ACfkdO047507 for ; Wed, 10 Feb 2010 06:41:46 -0600 X-ASG-Debug-ID: 1265805777-1f10009e0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from pu01.news-service.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0EE671C13C2 for ; Wed, 10 Feb 2010 04:42:58 -0800 (PST) Received: from pu01.news-service.com (ns1.news-service.com [195.114.240.3]) by cuda.sgi.com with ESMTP id 7QKqrlxiKoSGZlES for ; Wed, 10 Feb 2010 04:42:58 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by pu01.news-service.com (Postfix) with ESMTP id 1FB4313E41; Wed, 10 Feb 2010 13:42:57 +0100 (CET) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Debian amavisd-new at pu01.news-service.com Received: from pu01.news-service.com ([127.0.0.1]) by localhost (pu01.nse [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M8al5kBE3ppn; Wed, 10 Feb 2010 13:42:54 +0100 (CET) Received: from [172.25.0.47] (nse01.nse [172.25.0.47]) by pu01.news-service.com (Postfix) with ESMTP id D533F13E24; Wed, 10 Feb 2010 13:42:54 +0100 (CET) Message-ID: <4B72A9D1.8030101@news-service.com> Date: Wed, 10 Feb 2010 13:42:57 +0100 From: Patrick Schreurs User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Christoph Hellwig CC: Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) References: <4B0A8075.8080008@news-service.com> <20091211115932.GA20632@infradead.org> <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> <4B712166.9010701@news-service.com> <20100209103157.GA5197@infradead.org> In-Reply-To: <20100209103157.GA5197@infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ns1.news-service.com[195.114.240.3] X-Barracuda-Start-Time: 1265805779 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -0.86 X-Barracuda-Spam-Status: No, SCORE=-0.86 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE_7582B, INTERRUPTUS, INTERRUPTUS_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22167 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.01 INTERRUPTUS RAW: Message looks to contain HTML-interrupted text 0.65 INTERRUPTUS_2 Message looks to contain HTML-interrupted text 0.50 BSF_RULE_7582B Custom Rule 7582B X-Virus-Status: Clean On 9-2-2010 11:31, Christoph Hellwig wrote: > On Tue, Feb 09, 2010 at 09:48:38AM +0100, Patrick Schreurs wrote: >> This is a clean 2.6.32.3 with the xfs-inode-reclaim-2.6.32 patch i >> received from Dave on January 8th (see attachment). > > I can't find anything interesting regarding I_RECLAIMABLE manipulation > in there. The only thing I could think off going wrong is i_flags > and i_update_core sitting in the same word and the compiler causing > some read-modify-write cycles for it. Can you test the patch below? > It fixes the abose issue up, and to make sure sure the assert you hit > isn't as lethal changes it into a WARN_ON, which will still print the > backtrace, but not crash the machine. Thanks for the patch. After having this patch applied we saw *a lot* warnings. They all look like this: Feb 10 13:20:38 sb06 kernel: ------------[ cut here ]------------ Feb 10 13:20:38 sb06 kernel: WARNING: at fs/xfs/linux-2.6/xfs_sync.c:768 xfs_reclaim_inode_now+0x3d/0x84() Feb 10 13:20:38 sb06 kernel: Hardware name: PowerEdge 1950 Feb 10 13:20:38 sb06 kernel: Modules linked in: acpi_cpufreq cpufreq_ondemand ipmi_si ipmi_devintf ipmi_msghandler bonding mptspi serio_raw rng_core scsi_transport_spi bnx2 thermal processor thermal_sys Feb 10 13:20:38 sb06 kernel: Pid: 3145, comm: xfssyncd Not tainted 2.6.32.3 #2 Feb 10 13:20:38 sb06 kernel: Call Trace: Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x3d/0x84 Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x3d/0x84 Feb 10 13:20:38 sb06 kernel: [] ? warn_slowpath_common+0x77/0xa3 Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x3d/0x84 Feb 10 13:20:38 sb06 kernel: [] ? xfs_inode_ag_walk+0x68/0xa2 Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x0/0x84 Feb 10 13:20:38 sb06 kernel: [] ? xfs_inode_ag_iterator+0x50/0x7e Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x0/0x84 Feb 10 13:20:38 sb06 kernel: [] ? xfs_sync_worker+0x26/0x52 Feb 10 13:20:38 sb06 kernel: [] ? xfssyncd+0x123/0x180 Feb 10 13:20:38 sb06 kernel: [] ? xfssyncd+0x0/0x180 Feb 10 13:20:38 sb06 kernel: [] ? kthread+0x79/0x81 Feb 10 13:20:38 sb06 kernel: [] ? child_rip+0xa/0x20 Feb 10 13:20:38 sb06 kernel: [] ? kthread+0x0/0x81 Feb 10 13:20:38 sb06 kernel: [] ? child_rip+0x0/0x20 Feb 10 13:20:38 sb06 kernel: ---[ end trace 1ae862ca12666a87 ]--- and some look like this: Feb 10 13:20:38 sb06 kernel: ------------[ cut here ]------------ Feb 10 13:20:38 sb06 kernel: WARNING: at fs/xfs/linux-2.6/xfs_sync.c:768 xfs_reclaim_inode_now+0x3d/0x84() Feb 10 13:20:38 sb06 kernel: Hardware name: PowerEdge 1950 Feb 10 13:20:38 sb06 kernel: Modules linked in: acpi_cpufreq cpufreq_ondemand ipmi_si ipmi_devintf ipmi_msghandlerspes 13]n2r4a 2f1>nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospies 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13ode_no]n2r4a 2f1>nx85k7[<4x0-[e :7we_ospes 13nx85k7[<4x0-[e :7we_ospes 13] ? xfs_inode_ag_iterator+0x50/0x7e Feb 10 13:20:38 sb06 kernel: [] ? xfs_reclaim_inode_now+0x0/0x84 Feb 10 13:20:38 sb06 kernel: [] ? xfs_sync_worker+0x26/0x52 Feb 10 13:20:38 sb06 kernel: [] ? xfssyncd+0x123/0x180 Feb 10 13:20:38 sb06 kernel: [] ? xfssyncd+0x0/0x180 Feb 10 13:20:38 sb06 kernel: [] ? kthread+0x79/0x81 Feb 10 13:20:38 sb06 kernel: [] ? child_rip+0xa/0x20 Feb 10 13:20:38 sb06 kernel: [] ? kthread+0x0/0x81 Feb 10 13:20:38 sb06 kernel: [] ? child_rip+0x0/0x20 Feb 10 13:20:38 sb06 kernel: ---[ end trace 1ae862ca12666b1c ]--- I hope this clarifies things. If you need more info, don't hesitate to contact me. Thanks, -Patrick From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 08:54:01 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AErvEu055471 for ; Wed, 10 Feb 2010 08:54:01 -0600 X-ASG-Debug-ID: 1265813711-581c01150000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E356E1CBCAF7 for ; Wed, 10 Feb 2010 06:55:11 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id V54jJesv3HnEPiR3 for ; Wed, 10 Feb 2010 06:55:11 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NfDy8-0008C0-8A; Wed, 10 Feb 2010 14:55:08 +0000 Date: Wed, 10 Feb 2010 09:55:08 -0500 From: Christoph Hellwig To: Patrick Schreurs Cc: Christoph Hellwig , Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Message-ID: <20100210145508.GA29047@infradead.org> References: <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> <4B712166.9010701@news-service.com> <20100209103157.GA5197@infradead.org> <4B72A9D1.8030101@news-service.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B72A9D1.8030101@news-service.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265813711 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 10, 2010 at 01:42:57PM +0100, Patrick Schreurs wrote: > Thanks for the patch. After having this patch applied we saw *a lot* > warnings. They all look like this: Ok, looks like that is not an issue, so you can discard that patch. I went down to the radix tree code to look for races in it's tag handling, but then noticed that we might have an issue with our usage of the radix-tree API. Can you try the patch below ontop of Dave's rollup, and instead of my previous one? --- From: Christoph Hellwig Subject: xfs: fix locking for inode cache radix tree tag updates The radix-tree code requires it's users to serialize tag updates against other updates to the tree. While XFS protects tag updates against each other it does not serialize them against updates of the tree contents, which can lead to tag corruption. Fix the inode cache to always take pag_ici_lock in exclusive mode when updating radix tree tags. Signed-off-by: Christoph Hellwig Index: linux-2.6/fs/xfs/linux-2.6/xfs_sync.c =================================================================== --- linux-2.6.orig/fs/xfs/linux-2.6/xfs_sync.c 2010-02-10 14:28:46.648004203 +0100 +++ linux-2.6/fs/xfs/linux-2.6/xfs_sync.c 2010-02-10 14:29:56.657023619 +0100 @@ -734,12 +734,12 @@ xfs_inode_set_reclaim_tag( xfs_mount_t *mp = ip->i_mount; xfs_perag_t *pag = xfs_get_perag(mp, ip->i_ino); - read_lock(&pag->pag_ici_lock); + write_lock(&pag->pag_ici_lock); spin_lock(&ip->i_flags_lock); __xfs_inode_set_reclaim_tag(pag, ip); __xfs_iflags_set(ip, XFS_IRECLAIMABLE); spin_unlock(&ip->i_flags_lock); - read_unlock(&pag->pag_ici_lock); + write_unlock(&pag->pag_ici_lock); xfs_put_perag(mp, pag); } Index: linux-2.6/fs/xfs/xfs_iget.c =================================================================== --- linux-2.6.orig/fs/xfs/xfs_iget.c 2010-02-10 14:30:01.092254586 +0100 +++ linux-2.6/fs/xfs/xfs_iget.c 2010-02-10 14:34:00.199005529 +0100 @@ -228,13 +228,12 @@ xfs_iget_cache_hit( xfs_itrace_exit_tag(ip, "xfs_iget.alloc"); /* - * We need to set XFS_INEW atomically with clearing the - * reclaimable tag so that we do have an indicator of the - * inode still being initialized. + * We need to set XFS_IRECLAIM to prevent xfs_reclaim_inode + * from stomping over us while we recycle the inode. We can't + * clear the radix tree reclaimable tag yet as it requires + * pag_ici_lock to be helt exclusive. */ - ip->i_flags |= XFS_INEW; - ip->i_flags &= ~XFS_IRECLAIMABLE; - __xfs_inode_clear_reclaim_tag(mp, pag, ip); + ip->i_flags |= XFS_IRECLAIM; spin_unlock(&ip->i_flags_lock); read_unlock(&pag->pag_ici_lock); @@ -253,7 +252,15 @@ xfs_iget_cache_hit( __xfs_inode_set_reclaim_tag(pag, ip); goto out_error; } - inode->i_state = I_LOCK|I_NEW; + + write_lock(&pag->pag_ici_lock); + spin_lock(&ip->i_flags_lock); + ip->i_flags &= ~(XFS_IRECLAIMABLE | XFS_IRECLAIM); + ip->i_flags |= XFS_INEW; + __xfs_inode_clear_reclaim_tag(mp, pag, ip); + inode->i_state = I_LOCK | I_NEW; + spin_unlock(&ip->i_flags_lock); + write_unlock(&pag->pag_ici_lock); } else { /* If the VFS inode is being torn down, pause and try again. */ if (!igrab(inode)) { From patrick@news-service.com Wed Feb 10 09:41:32 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.5 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AFfWpw058443 for ; Wed, 10 Feb 2010 09:41:32 -0600 X-ASG-Debug-ID: 1265816564-278f039f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from pu01.news-service.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id BB5EE139564B for ; Wed, 10 Feb 2010 07:42:44 -0800 (PST) Received: from pu01.news-service.com (ns1.news-service.com [195.114.240.3]) by cuda.sgi.com with ESMTP id ayDGLgEU0o2fzkq7 for ; Wed, 10 Feb 2010 07:42:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by pu01.news-service.com (Postfix) with ESMTP id C53E213E41; Wed, 10 Feb 2010 16:42:43 +0100 (CET) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Scanned: Debian amavisd-new at pu01.news-service.com Received: from pu01.news-service.com ([127.0.0.1]) by localhost (pu01.nse [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZOLqkgo+p9vx; Wed, 10 Feb 2010 16:42:41 +0100 (CET) Received: from [172.25.0.47] (nse01.nse [172.25.0.47]) by pu01.news-service.com (Postfix) with ESMTP id ABACA13E24; Wed, 10 Feb 2010 16:42:41 +0100 (CET) Message-ID: <4B72D3F3.2040308@news-service.com> Date: Wed, 10 Feb 2010 16:42:43 +0100 From: Patrick Schreurs User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: Christoph Hellwig CC: Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) References: <4B3F9F88.9030307@news-service.com> <20100107110446.GA13802@discord.disaster> <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> <4B712166.9010701@news-service.com> <20100209103157.GA5197@infradead.org> <4B72A9D1.8030101@news-service.com> <20100210145508.GA29047@infradead.org> In-Reply-To: <20100210145508.GA29047@infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ns1.news-service.com[195.114.240.3] X-Barracuda-Start-Time: 1265816565 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22178 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Status: Clean On 10-2-2010 15:55, Christoph Hellwig wrote: > On Wed, Feb 10, 2010 at 01:42:57PM +0100, Patrick Schreurs wrote: >> Thanks for the patch. After having this patch applied we saw *a lot* >> warnings. They all look like this: > > Ok, looks like that is not an issue, so you can discard that patch. > > I went down to the radix tree code to look for races in it's tag > handling, but then noticed that we might have an issue with our > usage of the radix-tree API. Can you try the patch below ontop > of Dave's rollup, and instead of my previous one? Okay. This patch is currently active. Thanks. I don't have a way to trigger it, so we'll have to wait and see what happens. Obviously we'll keep you posted. Have any of these patches been sent to the stable team? And have these patches been submitted to the upcoming 2.6.33 kernel? -Patrick From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 09:45:53 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AFjqHG058696 for ; Wed, 10 Feb 2010 09:45:53 -0600 X-ASG-Debug-ID: 1265816826-586002fd0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 0E0F41CBD0EB for ; Wed, 10 Feb 2010 07:47:06 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id 7gl21Rwo0GnX5FG8 for ; Wed, 10 Feb 2010 07:47:06 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NfEmP-0007gM-Bq; Wed, 10 Feb 2010 15:47:05 +0000 Date: Wed, 10 Feb 2010 10:47:05 -0500 From: Christoph Hellwig To: Patrick Schreurs Cc: Christoph Hellwig , Dave Chinner , Tommy van Leeuwen , xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Subject: Re: [PATCH] Inode reclaim fixes (was Re: 2.6.31 xfs_fs_destroy_inode: cannot reclaim) Message-ID: <20100210154705.GA28809@infradead.org> References: <4B45CFAC.4000607@news-service.com> <20100108113114.GA8654@discord.disaster> <4B504B03.7050604@news-service.com> <4B6706CE.1020207@news-service.com> <20100208194226.GD9527@infradead.org> <4B712166.9010701@news-service.com> <20100209103157.GA5197@infradead.org> <4B72A9D1.8030101@news-service.com> <20100210145508.GA29047@infradead.org> <4B72D3F3.2040308@news-service.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B72D3F3.2040308@news-service.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265816827 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 10, 2010 at 04:42:43PM +0100, Patrick Schreurs wrote: > Have any of these patches been sent to the stable team? And have these > patches been submitted to the upcoming 2.6.33 kernel? Everything before the current test patches is in 2.6.33-rc, I'm not sure what has made it to stable yet, but if I remember correctly Dave was going to send his fixes to -stable. From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 12:07:47 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AI7djF068074 for ; Wed, 10 Feb 2010 12:07:47 -0600 X-ASG-Debug-ID: 1265825332-64cb00c60000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id A81A71396794 for ; Wed, 10 Feb 2010 10:08:52 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id prYW2uDRlmHVahXa for ; Wed, 10 Feb 2010 10:08:52 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NfGzb-0003EF-Re; Wed, 10 Feb 2010 18:08:51 +0000 Date: Wed, 10 Feb 2010 13:08:51 -0500 From: Christoph Hellwig To: linux-kernel@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: XFS status update for January 2010 Subject: XFS status update for January 2010 Message-ID: <20100210180851.GA9854@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265825333 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean January saw additional release candidates of the Linux 2.6.33 kernel, including a couple of bug fixes for XFS. In the meantime the XFS tree has been growing a large number of patches destined for the Linux 2.6.34 merge window: a large rework of the handling of per-AG data, support for the quota netlink interface, and better power saving behavior of the XFS kernel threads, and of course various cleanups. A large patch series to replace the current asynchronous inode writeback with a new scheme that uses the delayed write buffers was posted to the list. The new scheme, which allows archive better I/O locality by dispatching meta-data I/O from a single place has been discussed extensively and is expected to be merged in February. On the userspace side January saw the 3.1.0 and 3.1.1 releases of xfsprogs, as well as the 3.0.4 release of xfsdump. The biggest changes in xfsprogs 3.1.0 were optimizations in xfs_repair that lead to a much lower memory usage, and optional use of the blkid library for filesystem detection and retrieving storage topology information. The 3.1.1 release contained various important bug fixes for these changes and a various improvements to the build system. The major feature of xfsdump 3.0.4 were fixes for time stamp handling on 64-bit systems. The xfstests package also lots of activity including various new testcases and an improved build system. From sandeen@sandeen.net Wed Feb 10 13:31:27 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AJVR6h073204 for ; Wed, 10 Feb 2010 13:31:27 -0600 X-ASG-Debug-ID: 1265830359-426601f40000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id C887B1CBE071 for ; Wed, 10 Feb 2010 11:32:39 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id pXynTW3AaF1xs8Zn for ; Wed, 10 Feb 2010 11:32:39 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 7C6BA8F4FF6 for ; Wed, 10 Feb 2010 13:32:39 -0600 (CST) Message-ID: <4B7309D7.5090800@sandeen.net> Date: Wed, 10 Feb 2010 13:32:39 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: xfs-oss X-ASG-Orig-Subj: [PATCH] enable inode64 by default when possible Subject: [PATCH] enable inode64 by default when possible Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265830360 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.42 X-Barracuda-Spam-Status: No, SCORE=-1.42 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M, RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22192 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Taking another swing at this. As XFS continues to position itself as the choice for very large Linux filesystems, we need to be mindful of the problems that the 32-bit inode restriction can cause with allocations and performance. As such, this patch changes the default to inode64 whenever XFS_BIG_INUMS is set, which in turn depends on either CONFIG_LBDAF or 64-bit longs. Going forward, we may wish to do this unconditionally for all filesystems by choosing CONFIG_LBDAF by default when xfs is chosen, but I'll leave that for later. This patch adds a "noinode64" option for backwards compatibility. (Minor update to documentation for "nobarrier" as well, which had not been previously documented). Signed-off-by: Eric Sandeen --- diff --git a/Documentation/filesystems/xfs.txt b/Documentation/filesystems/xfs.txt index 9878f50..05b845a 100644 --- a/Documentation/filesystems/xfs.txt +++ b/Documentation/filesystems/xfs.txt @@ -37,7 +37,10 @@ When mounting an XFS filesystem, the following options are accepted. Enables the use of block layer write barriers for writes into the journal and unwritten extent conversion. This allows for drive level write caching to be enabled, for devices that - support write barriers. + support write barriers. This is the default. + + nobarrier + Disables the use of block layer write barriers. dmapi Enable the DMAPI (Data Management API) event callouts. @@ -66,8 +69,16 @@ When mounting an XFS filesystem, the following options are accepted. Indicates that XFS is allowed to create inodes at any location in the filesystem, including those which will result in inode numbers occupying more than 32 bits of significance. This is - provided for backwards compatibility, but causes problems for - backup applications that cannot handle large inode numbers. + the default for 64-bit or CONFIG_LBDAF kernels as of 2.6.33. + + noinode64 + Indicates that XFS must create inodes in filesystem locations + which will not result in inode numbers occupying more than 32 + bits of significance. This is provided for backwards compatibility, + for 32-bit applications which may not use the 64-bit stat interface, + such as backup applications that cannot handle large inode numbers. + Note that this only affects new inode creation; existing 64-bit + inode locations are unaffected. largeio/nolargeio If "nolargeio" is specified, the optimal I/O reported in diff --git a/fs/xfs/linux-2.6/xfs_super.c b/fs/xfs/linux-2.6/xfs_super.c index 97c0f5a..7c74965 100644 --- a/fs/xfs/linux-2.6/xfs_super.c +++ b/fs/xfs/linux-2.6/xfs_super.c @@ -95,6 +95,7 @@ mempool_t *xfs_ioend_pool; #define MNTOPT_NOBARRIER "nobarrier" /* .. disable */ #define MNTOPT_OSYNCISOSYNC "osyncisosync" /* o_sync is REALLY o_sync */ #define MNTOPT_64BITINODE "inode64" /* inodes can be allocated anywhere */ +#define MNTOPT_32BITINODE "noinode64" /* inodes allocated in 32-bit range */ #define MNTOPT_IKEEP "ikeep" /* do not free empty inode clusters */ #define MNTOPT_NOIKEEP "noikeep" /* free empty inode clusters */ #define MNTOPT_LARGEIO "largeio" /* report large I/O sizes in stat() */ @@ -196,7 +197,9 @@ xfs_parseargs( */ mp->m_flags |= XFS_MOUNT_BARRIER; mp->m_flags |= XFS_MOUNT_COMPAT_IOSIZE; +#ifndef XFS_BIG_INUMS mp->m_flags |= XFS_MOUNT_SMALL_INUMS; +#endif /* * These can be overridden by the mount option parsing. @@ -317,6 +320,8 @@ xfs_parseargs( this_char); return EINVAL; #endif + } else if (!strcmp(this_char, MNTOPT_32BITINODE)) { + mp->m_flags |= XFS_MOUNT_SMALL_INUMS; } else if (!strcmp(this_char, MNTOPT_NOUUID)) { mp->m_flags |= XFS_MOUNT_NOUUID; } else if (!strcmp(this_char, MNTOPT_BARRIER)) { @@ -534,6 +539,7 @@ xfs_showargs( { XFS_MOUNT_FILESTREAMS, "," MNTOPT_FILESTREAM }, { XFS_MOUNT_DMAPI, "," MNTOPT_DMAPI }, { XFS_MOUNT_GRPID, "," MNTOPT_GRPID }, + { XFS_MOUNT_SMALL_INUMS, "," MNTOPT_32BITINODE }, { 0, NULL } }; static struct proc_xfs_info xfs_info_unset[] = { From eflorac@intellique.com Wed Feb 10 14:03:45 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AK3i27075668 for ; Wed, 10 Feb 2010 14:03:45 -0600 X-ASG-Debug-ID: 1265832295-37a903b10000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp5-g21.free.fr (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 104671CBE24F for ; Wed, 10 Feb 2010 12:04:57 -0800 (PST) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by cuda.sgi.com with ESMTP id UwOJzF23LgCMEmrW for ; Wed, 10 Feb 2010 12:04:57 -0800 (PST) Received: from smtp5-g21.free.fr (localhost [127.0.0.1]) by smtp5-g21.free.fr (Postfix) with ESMTP id 7C3C0D48015 for ; Wed, 10 Feb 2010 21:04:53 +0100 (CET) Received: from galadriel.home (pla78-1-82-235-234-79.fbx.proxad.net [82.235.234.79]) by smtp5-g21.free.fr (Postfix) with ESMTP id 81CA9D48147 for ; Wed, 10 Feb 2010 21:04:51 +0100 (CET) Date: Wed, 10 Feb 2010 21:04:48 +0100 From: Emmanuel Florac To: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] enable inode64 by default when possible Subject: Re: [PATCH] enable inode64 by default when possible Message-ID: <20100210210448.36f16aba@galadriel.home> In-Reply-To: <4B7309D7.5090800@sandeen.net> References: <4B7309D7.5090800@sandeen.net> Organization: Intellique X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.9; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: smtp5-g21.free.fr[212.27.42.5] X-Barracuda-Start-Time: 1265832299 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22194 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Le Wed, 10 Feb 2010 13:32:39 -0600 vous =E9criviez: > As such, this patch changes the default to inode64 whenever > XFS_BIG_INUMS is set, which in turn depends on either > CONFIG_LBDAF or 64-bit longs. But doesn't it cause problems specially for NFS sharing ? --=20 ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | | +33 1 78 94 84 02 ------------------------------------------------------------------------ From bpm@sgi.com Wed Feb 10 14:13:36 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AKDaDw076447 for ; Wed, 10 Feb 2010 14:13:36 -0600 Received: from whiskey.americas.sgi.com (whiskey.americas.sgi.com [128.162.233.19]) by relay1.corp.sgi.com (Postfix) with ESMTP id 7F3608F8066; Wed, 10 Feb 2010 12:14:47 -0800 (PST) Received: by whiskey.americas.sgi.com (Postfix, from userid 4600) id A0E164266A1; Wed, 10 Feb 2010 14:15:27 -0600 (CST) Date: Wed, 10 Feb 2010 14:15:27 -0600 From: bpm@sgi.com To: Christoph Hellwig Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Message-ID: <20100210201527.GJ23654@sgi.com> References: <20100210003220.6021.74943.stgit@case> <20100210003337.6021.10942.stgit@case> <20100210090750.GB21875@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100210090750.GB21875@infradead.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Howdy, On Wed, Feb 10, 2010 at 04:07:50AM -0500, Christoph Hellwig wrote: > On Tue, Feb 09, 2010 at 06:33:37PM -0600, Ben Myers wrote: > > hopefully > > the XFS gurus can continue to provide guidance. > > Ccing the xfs list would help with that :) I'll cross-post the next rev. ;) > > + if (xfs_ipincount(c_xip)) { > > + /* > > + * AFAICS the child is always modified after the parent > > + * in nfsd so should always have a larger lsn. > > + */ > > + if (c_xip->i_itemp->ili_last_lsn > force_lsn) { > > + force_lsn = c_xip->i_itemp->ili_last_lsn; > > + } else { > > + force_lsn = 0; /* whole thing */ > > + } > > I wouldn't rely on that and always take the larger one. I am concerned that ili_last_lsn will roll over at some point. I haven't figured out how to detect that yet. > Now with the simplification of always having a non-zero first argument > suggested in the previous mail this might be simplified down to: > > STATIC int > xfs_fs_nfs_commit_metadata( > struct dentry *parent, > struct dentry *child) > { > struct xfs_inode *dp = XFS_I(parent->d_inode); > struct xfs_inode *ip = NULL; > struct xfs_mount *mp = dp->i_mount; > xfs_lsn_t force_lsn = 0; > int error = 0; > > if (child) { > ip = XFS_I(child->d_inode); > xfs_lock_two_inodes(dp, ip, XFS_ILOCK_SHARED); > } else { > xfs_ilock(dp, XFS_ILOCK_SHARED); > } > > if (xfs_ipincount(dp)) > force_lsn = dp->i_itemp->ili_last_lsn; > if (ip && xfs_ipincount(ip)) > force_lsn = max(force_lsn, ip->i_itemp->ili_last_lsn); > + if (force_lsn) > error = _xfs_log_force_lsn(mp, force_lsn, NULL); > > if (ip) > xfs_iunlock(ip, XFS_ILOCK_SHARED); > xfs_iunlock(dp, XFS_ILOCK_SHARED); > > return error; > } That's nice! > Note that _xfs_log_force_lsn is new in the XFS tree, mainline still > has _xfs_log_force with an lsn argument. I've been working with Bruce's tree. Not sure how best to handle that. Thanks, Ben From sandeen@sandeen.net Wed Feb 10 14:14:10 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_00,FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AKE9eu076507 for ; Wed, 10 Feb 2010 14:14:09 -0600 X-ASG-Debug-ID: 1265832921-573500aa0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.sandeen.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 238D81396BB6 for ; Wed, 10 Feb 2010 12:15:21 -0800 (PST) Received: from mail.sandeen.net (64-131-60-146.usfamily.net [64.131.60.146]) by cuda.sgi.com with ESMTP id VGFhwAzeGfOri3aC for ; Wed, 10 Feb 2010 12:15:21 -0800 (PST) Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sandeen.net (Postfix) with ESMTP id 6FD9312AEC00; Wed, 10 Feb 2010 14:15:21 -0600 (CST) Message-ID: <4B7313D9.9020603@sandeen.net> Date: Wed, 10 Feb 2010 14:15:21 -0600 From: Eric Sandeen User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Emmanuel Florac CC: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] enable inode64 by default when possible Subject: Re: [PATCH] enable inode64 by default when possible References: <4B7309D7.5090800@sandeen.net> <20100210210448.36f16aba@galadriel.home> In-Reply-To: <20100210210448.36f16aba@galadriel.home> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Barracuda-Connect: 64-131-60-146.usfamily.net[64.131.60.146] X-Barracuda-Start-Time: 1265832923 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.92 X-Barracuda-Spam-Status: No, SCORE=-1.92 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=RDNS_DYNAMIC X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22195 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_DYNAMIC Delivered to trusted network by host with dynamic-looking rDNS X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Emmanuel Florac wrote: > Le Wed, 10 Feb 2010 13:32:39 -0600 vous écriviez: > >> As such, this patch changes the default to inode64 whenever >> XFS_BIG_INUMS is set, which in turn depends on either >> CONFIG_LBDAF or 64-bit longs. > > But doesn't it cause problems specially for NFS sharing ? > For some clients, yes - as do other NFS servers. That's what noinode64 is for... Also, newer nfs clients have an option: nfs.enable_ino64= [NFS] enable 64-bit inode numbers. If zero, the NFS client will fake up a 32-bit inode number for the readdir() and stat() syscalls instead of returning the full 64-bit number. The default is to return 64-bit inode numbers. (see Documentation/kernel-parameters.txt) At some point we have to drag people kicking and screaming out of 1988, I think! :) (I am mindful that this may manifest itself as "xfs is incompatible" but if we document this and advertise it a bit, I hope we can avoid that. At some point, apps just need to be fixed.) -Eric From eflorac@intellique.com Wed Feb 10 14:41:27 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1AKfRpS078500 for ; Wed, 10 Feb 2010 14:41:27 -0600 X-ASG-Debug-ID: 1265834554-246001360000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from smtp5-g21.free.fr (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 313641C32F9 for ; Wed, 10 Feb 2010 12:42:38 -0800 (PST) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by cuda.sgi.com with ESMTP id IAuqKAgI8Zy3NMk3 for ; Wed, 10 Feb 2010 12:42:38 -0800 (PST) Received: from smtp5-g21.free.fr (localhost [127.0.0.1]) by smtp5-g21.free.fr (Postfix) with ESMTP id 80344D48103; Wed, 10 Feb 2010 21:42:30 +0100 (CET) Received: from galadriel.home (pla78-1-82-235-234-79.fbx.proxad.net [82.235.234.79]) by smtp5-g21.free.fr (Postfix) with ESMTP id 662BFD48201; Wed, 10 Feb 2010 21:42:28 +0100 (CET) Date: Wed, 10 Feb 2010 21:42:16 +0100 From: Emmanuel Florac To: Eric Sandeen Cc: xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [PATCH] enable inode64 by default when possible Subject: Re: [PATCH] enable inode64 by default when possible Message-ID: <20100210214216.0ab263fd@galadriel.home> In-Reply-To: <4B7313D9.9020603@sandeen.net> References: <4B7309D7.5090800@sandeen.net> <20100210210448.36f16aba@galadriel.home> <4B7313D9.9020603@sandeen.net> Organization: Intellique X-Mailer: Claws Mail 3.0.2 (GTK+ 2.12.9; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: smtp5-g21.free.fr[212.27.42.5] X-Barracuda-Start-Time: 1265834561 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22196 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Le Wed, 10 Feb 2010 14:15:21 -0600 vous =E9criviez: > > But doesn't it cause problems specially for NFS sharing ? > > =20 >=20 > For some clients, yes - as do other NFS servers. It would be nice to mention which one... At least it would be good to know at least approximately which releases of the main Unix clients (Linux, Solaris, OS X, BSDs, HP/UX, AIX) are compatible. --=20 ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | | +33 1 78 94 84 02 ------------------------------------------------------------------------ From SRS0+Mqth+70+fromorbit.com=david@internode.on.net Wed Feb 10 15:28:06 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.6 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1ALS5Vn081992 for ; Wed, 10 Feb 2010 15:28:05 -0600 X-ASG-Debug-ID: 1265837357-54e9005f0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from mail.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 201E81C358C for ; Wed, 10 Feb 2010 13:29:17 -0800 (PST) Received: from mail.internode.on.net (bld-mail15.adl6.internode.on.net [150.101.137.100]) by cuda.sgi.com with ESMTP id N2k4M3sABXePSSr8 for ; Wed, 10 Feb 2010 13:29:17 -0800 (PST) Received: from discord (unverified [121.44.103.80]) by mail.internode.on.net (SurgeMail 3.8f2) with ESMTP id 1690883-1927428 for multiple; Thu, 11 Feb 2010 07:59:16 +1030 (CDT) Received: from dave by discord with local (Exim 4.69) (envelope-from ) id 1NfK7X-0002fn-12; Thu, 11 Feb 2010 08:29:15 +1100 Date: Thu, 11 Feb 2010 08:29:14 +1100 From: Dave Chinner To: bpm@sgi.com Cc: Christoph Hellwig , linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Subject: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Message-ID: <20100210212914.GT11483@discord.disaster> References: <20100210003220.6021.74943.stgit@case> <20100210003337.6021.10942.stgit@case> <20100210090750.GB21875@infradead.org> <20100210201527.GJ23654@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100210201527.GJ23654@sgi.com> User-Agent: Mutt/1.5.18 (2008-05-17) X-Barracuda-Connect: bld-mail15.adl6.internode.on.net[150.101.137.100] X-Barracuda-Start-Time: 1265837359 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22199 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 10, 2010 at 02:15:27PM -0600, bpm@sgi.com wrote: > Howdy, > > On Wed, Feb 10, 2010 at 04:07:50AM -0500, Christoph Hellwig wrote: > > On Tue, Feb 09, 2010 at 06:33:37PM -0600, Ben Myers wrote: > > > hopefully > > > the XFS gurus can continue to provide guidance. > > > > Ccing the xfs list would help with that :) > > I'll cross-post the next rev. ;) > > > > + if (xfs_ipincount(c_xip)) { > > > + /* > > > + * AFAICS the child is always modified after the parent > > > + * in nfsd so should always have a larger lsn. > > > + */ > > > + if (c_xip->i_itemp->ili_last_lsn > force_lsn) { > > > + force_lsn = c_xip->i_itemp->ili_last_lsn; > > > + } else { > > > + force_lsn = 0; /* whole thing */ > > > + } > > > > I wouldn't rely on that and always take the larger one. > > I am concerned that ili_last_lsn will roll over at some point. I > haven't figured out how to detect that yet. XFS_LSN_CMP() should be used for LSN comparisons - it handles rollover corectly. > > Now with the simplification of always having a non-zero first argument > > suggested in the previous mail this might be simplified down to: > > > > STATIC int > > xfs_fs_nfs_commit_metadata( > > struct dentry *parent, > > struct dentry *child) > > { > > struct xfs_inode *dp = XFS_I(parent->d_inode); > > struct xfs_inode *ip = NULL; > > struct xfs_mount *mp = dp->i_mount; > > xfs_lsn_t force_lsn = 0; > > int error = 0; > > > > if (child) { > > ip = XFS_I(child->d_inode); > > xfs_lock_two_inodes(dp, ip, XFS_ILOCK_SHARED); > > } else { > > xfs_ilock(dp, XFS_ILOCK_SHARED); > > } > > > > if (xfs_ipincount(dp)) > > force_lsn = dp->i_itemp->ili_last_lsn; > > if (ip && xfs_ipincount(ip)) > > force_lsn = max(force_lsn, ip->i_itemp->ili_last_lsn); So this should be: if (XFS_LSN_CMP(force_lsn, ip->i_itemp->ili_last_lsn) < 0) force_lsn = ip->i_itemp->ili_last_lsn; Cheers, Dave. -- Dave Chinner david@fromorbit.com From BATV+c92b07a41bedde9ca4f4+2362+infradead.org+hch@bombadil.srs.infradead.org Wed Feb 10 15:56:31 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.8 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1ALuUDi083837 for ; Wed, 10 Feb 2010 15:56:31 -0600 X-ASG-Debug-ID: 1265839064-19d701130000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from bombadil.infradead.org (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 2FA7613973F8; Wed, 10 Feb 2010 13:57:44 -0800 (PST) Received: from bombadil.infradead.org (bombadil.infradead.org [18.85.46.34]) by cuda.sgi.com with ESMTP id GfFo0zDixB9gAzXB; Wed, 10 Feb 2010 13:57:44 -0800 (PST) X-ASG-Whitelist: Client Received: from hch by bombadil.infradead.org with local (Exim 4.69 #1 (Red Hat Linux)) id 1NfKZ6-0001ST-JY; Wed, 10 Feb 2010 21:57:44 +0000 Date: Wed, 10 Feb 2010 16:57:44 -0500 From: Christoph Hellwig To: bpm@sgi.com Cc: linux-nfs@vger.kernel.org, xfs@oss.sgi.com X-ASG-Orig-Subj: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Subject: Re: [RFC PATCH 2/2] xfs_export_operations.commit_metadata Message-ID: <20100210215744.GB25004@infradead.org> References: <20100210003220.6021.74943.stgit@case> <20100210003337.6021.10942.stgit@case> <20100210090750.GB21875@infradead.org> <20100210201527.GJ23654@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100210201527.GJ23654@sgi.com> User-Agent: Mutt/1.5.19 (2009-01-05) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: bombadil.infradead.org[18.85.46.34] X-Barracuda-Start-Time: 1265839065 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean On Wed, Feb 10, 2010 at 02:15:27PM -0600, bpm@sgi.com wrote: > > Note that _xfs_log_force_lsn is new in the XFS tree, mainline still > > has _xfs_log_force with an lsn argument. > > I've been working with Bruce's tree. Not sure how best to handle that. Keep working against it for now, the merge fixup will be easy enough. From jpiszcz@lucidpixels.com Thu Feb 11 08:08:28 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1BE8SDj151398 for ; Thu, 11 Feb 2010 08:08:28 -0600 X-ASG-Debug-ID: 1265897380-5897007c0000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from lucidpixels.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id E028B1C5899 for ; Thu, 11 Feb 2010 06:09:40 -0800 (PST) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by cuda.sgi.com with ESMTP id FeALlbCRQp6GrcZl for ; Thu, 11 Feb 2010 06:09:40 -0800 (PST) Received: by lucidpixels.com (Postfix, from userid 1001) id C1E1C38DFB8; Thu, 11 Feb 2010 09:09:39 -0500 (EST) Date: Thu, 11 Feb 2010 09:09:39 -0500 (EST) From: Justin Piszcz To: linux-kernel@vger.kernel.org, linux-raid@vger.kernl.org, xfs@oss.sgi.com, Alan Piszcz X-ASG-Orig-Subj: 2.6.32.3 x86_64 - XFS hangs, all I/O to D-state bug (again) Subject: 2.6.32.3 x86_64 - XFS hangs, all I/O to D-state bug (again) Message-ID: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Barracuda-Connect: lucidpixels.com[75.144.35.66] X-Barracuda-Start-Time: 1265897381 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22254 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean Hello, While tarring and compressing (bzip2) a lot of files, the following error occurred, note the output is not clean because this was taken from netconsole. When this occurs, the host cannot be rebooted with reboot/proceses cannot be killed and the box locks up. There are no apparent hardware issues. Before, asterisk would trigger this bug, since asterisk no longer runs on this host, it ran for ~2-3 months without any problems, until now. Please cc me as I am not on the lists, thanks. Is this a md raid issue or XFS? From the trace it appears to be an XFS bug? Feb 11 07:38:14 l1 [ 20.270270] e100: eth1 NIC Link is Up 100 Mbps Full Duplex Feb 11 07:47:54 l1 [ 600.432165] INFO: task scp:4871 blocked for more than 120 seconds. Feb 11 07:47:54 l1 [ 600.432177] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 11 07:47:54 l1 [ 600.432182] scp D Feb 11 07:47:54 l1 ffff8801eee6f14c Feb 11 07:47:54 l1 0 4871 4870 0x00000000 Feb 11 07:47:54 l1 [ 600.432188] ffff880220c87950 Feb 11 07:47:54 l1 0000000000000082 Feb 11 07:47:54 l1 0000000000200200 Feb 11 07:47:54 l1 ffff88023eb71400 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432196] 000000000000c928 Feb 11 07:47:54 l1 ffff8801f7a55fd8 Feb 11 07:47:54 l1 ffff880205fc6150 Feb 11 07:47:54 l1 ffff880205fc63c8 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432203] 0000000000001000 Feb 11 07:47:54 l1 ffffffff8108d02a Feb 11 07:47:54 l1 ffff8801eee6f0c0 Feb 11 07:47:54 l1 ffff880205fc63c8 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432209] Call Trace: Feb 11 07:47:54 l1 [ 600.432230] [] ? generic_file_buffered_write+0x1aa/0x290 Feb 11 07:47:54 l1 [ 600.432236] [] ? __down_write_nested+0x7d/0xb0 Feb 11 07:47:54 l1 [ 600.432242] [] ? xfs_write+0x23d/0x950 Feb 11 07:47:54 l1 [ 600.432250] [] ? do_sync_write+0xe3/0x130 Feb 11 07:47:54 l1 [ 600.432260] [] ? autoremove_wake_function+0x0/0x30 Feb 11 07:47:54 l1 [ 600.432266] [] ? fsnotify+0x4/0x1a0 Feb 11 07:47:54 l1 [ 600.432271] [] ? common_interrupt+0xe/0x13 Feb 11 07:47:54 l1 [ 600.432276] [] ? vfs_write+0xcb/0x180 Feb 11 07:47:54 l1 [ 600.432280] [] ? sys_write+0x53/0xa0 Feb 11 07:47:54 l1 [ 600.432285] [] ? system_call_fastpath+0x16/0x1b Feb 11 07:47:54 l1 [ 600.432291] INFO: task flush-9:3:4874 blocked for more than 120 seconds. Feb 11 07:47:54 l1 [ 600.432294] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 11 07:47:54 l1 [ 600.432297] flush-9:3 D Feb 11 07:47:54 l1 ffff88021948d750 Feb 11 07:47:54 l1 0 4874 2 0x00000000 Feb 11 07:47:54 l1 [ 600.432309] ffff88023fa3c8c0 Feb 11 07:47:54 l1 0000000000000046 Feb 11 07:47:54 l1 000000001948d750 Feb 11 07:47:54 l1 ffffffff8163d788 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432323] 000000000000c928 Feb 11 07:47:54 l1 ffff8801f79fbfd8 Feb 11 07:47:54 l1 ffff88021948d750 Feb 11 07:47:54 l1 ffff88021948d9c8 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432329] 000000013f467bb0 Feb 11 07:47:54 l1 ffffffff8108c183 Feb 11 07:47:54 l1 0000000000000000 Feb 11 07:47:54 l1 ffff88021948d9c8 Feb 11 07:47:54 l1 Feb 11 07:47:54 l1 [ 600.432337] Call Trace: Feb 11 07:47:54 l1 [ 600.432341] [] ? find_lock_page+0x23/0x80 Feb 11 07:47:54 l1 [ 600.432344] [] ? find_or_create_page+0x41/0xc0 Feb 11 07:47:54 l1 [ 600.432349] [] ? schedule_timeout+0x195/0x1f0 Feb 11 07:47:54 l1 [ 600.432352] [] ? __down+0x61/0xa0 Feb 11 07:47:54 l1 [ 600.432356] [] ? down+0x46/0x50 Feb 11 07:47:54 l1 [ 600.432360] [] ? _xfs_buf_find+0x134/0x220 Feb 11 07:47:54 l1 [ 600.432363] [] ? xfs_buf_get_flags+0x6e/0x190 Feb 11 07:47:54 l1 [ 600.432366] [] ? xfs_buf_read_flags+0x12/0xa0 Feb 11 07:47:54 l1 [ 600.432371] [] ? xfs_trans_read_buf+0x1f6/0x350 Feb 11 07:47:54 l1 [ 600.432377] [] ? xfs_read_agf+0x68/0x190 Feb 11 07:47:54 l1 [ 600.432380] [] ? xfs_alloc_read_agf+0x30/0xd0 Feb 11 07:47:54 l1 [ 600.432383] [] ? xfs_alloc_fix_freelist+0x379/0x450 Feb 11 07:47:54 l1 [ 600.432388] [] ? xfs_iext_remove+0x35/0x80 Feb 11 07:47:54 l1 [ 600.432393] [] ? xfs_bmap_add_extent_delay_real+0x5ef/0x11a0 Feb 11 07:47:54 l1 [ 600.432396] [] ? xfs_trans_log_buf+0x63/0xa0 Feb 11 07:47:54 l1 [ 600.432401] [] ? xlog_state_get_iclog_space+0x60/0x2c0 Feb 11 07:47:54 l1 [ 600.432404] [] ? __down_read+0x17/0xae Feb 11 07:47:54 l1 [ 600.432408] [] ? xfs_alloc_vextent+0x310/0x4b0 Feb 11 07:47:54 l1 [ 600.432412] [] ? xfs_bmap_btalloc+0x598/0xa40 Feb 11 07:47:54 l1 [ 600.432417] [] ? xfs_bmapi+0x9e2/0x11a0 Feb 11 07:47:54 l1 [ 600.432422] [] ? xfs_trans_reserve+0x9f/0x210 Feb 11 07:47:54 l1 [ 600.432425] [] ? xfs_iomap_write_allocate+0x23e/0x3b0 Feb 11 07:47:54 l1 [ 600.432429] [] ? xfs_iomap+0x2c0/0x300 Feb 11 07:47:54 l1 [ 600.432433] [] ? xfs_map_blocks+0x25/0x30 Feb 11 07:47:54 l1 [ 600.432437] [] ? xfs_page_state_convert+0x414/0x6c0 Feb 11 07:47:54 l1 [ 600.432443] [] ? radix_tree_gang_lookup_tag_slot+0xc3/0xf0 Feb 11 07:47:54 l1 [ 600.432447] [] ? xfs_vm_writepage+0x77/0x130 Feb 11 07:47:54 l1 [ 600.432453] [] ? __writepage+0xa/0x40 Feb 11 07:47:54 l1 [ 600.432456] [] ? write_cache_pages+0x1df/0x3c0 Feb 11 07:47:54 l1 [ 600.432459] [] ? __writepage+0x0/0x40 Feb 11 07:47:54 l1 [ 600.432464] [] ? writeback_single_inode+0xd2/0x390 Feb 11 07:47:54 l1 [ 600.432468] [] ? writeback_inodes_wb+0x3ff/0x5e0 Feb 11 07:47:54 l1 [ 600.432473] [] ? wb_writeback+0x11e/0x1f0 Feb 11 07:47:54 l1 [ 600.432479] [] ? try_to_del_timer_sync+0x5e/0x90 Feb 11 07:47:54 l1 [ 600.432484] [] ? wb_do_writeback+0x17b/0x180 Feb 11 07:47:54 l1 [ 600.432487] [] ? bdi_writeback_task+0x5d/0xa0 Feb 11 07:47:54 l1 [ 600.432492] [] ? bdi_start_fn+0x0/0xf0 Feb 11 07:47:54 l1 [ 600.432496] [] ? bdi_start_fn+0x7e/0xf0 Feb 11 07:47:54 l1 [ 600.432499] [] ? bdi_start_fn+0x0/0xf0 Feb 11 07:47:54 l1 [ 600.432503] [] ? kthread+0x96/0xa0 Feb 11 07:47:54 l1 [ 600.432508] [] ? child_rip+0xa/0x20 Feb 11 07:47:54 l1 [ 600.432513] [] ? kthread+0x0/0xa0 Feb 11 07:47:54 l1 [ 600.432517] [] ? child_rip+0x0/0x20 Feb 11 07:49:54 l1 [ 720.432171] INFO: task scp:4871 blocked for more than 120 seconds. Feb 11 07:49:54 l1 [ 720.432187] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 11 07:49:54 l1 [ 720.432192] scp D Feb 11 07:49:54 l1 ffff8801eee6f14c Feb 11 07:49:54 l1 0 4871 4870 0x00000000 Feb 11 07:49:54 l1 [ 720.432198] ffff880220c87950 Feb 11 07:49:54 l1 0000000000000082 Feb 11 07:49:54 l1 0000000000200200 Feb 11 07:49:54 l1 ffff88023eb71400 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432206] 000000000000c928 Feb 11 07:49:54 l1 ffff8801f7a55fd8 Feb 11 07:49:54 l1 ffff880205fc6150 Feb 11 07:49:54 l1 ffff880205fc63c8 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432217] 0000000000001000 Feb 11 07:49:54 l1 ffffffff8108d02a Feb 11 07:49:54 l1 ffff8801eee6f0c0 Feb 11 07:49:54 l1 ffff880205fc63c8 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432224] Call Trace: Feb 11 07:49:54 l1 [ 720.432238] [] ? generic_file_buffered_write+0x1aa/0x290 Feb 11 07:49:54 l1 [ 720.432243] [] ? __down_write_nested+0x7d/0xb0 Feb 11 07:49:54 l1 [ 720.432249] [] ? xfs_write+0x23d/0x950 Feb 11 07:49:54 l1 [ 720.432254] [] ? do_sync_write+0xe3/0x130 Feb 11 07:49:54 l1 [ 720.432259] [] ? autoremove_wake_function+0x0/0x30 Feb 11 07:49:54 l1 [ 720.432265] [] ? fsnotify+0x4/0x1a0 Feb 11 07:49:54 l1 [ 720.432270] [] ? common_interrupt+0xe/0x13 Feb 11 07:49:54 l1 [ 720.432275] [] ? vfs_write+0xcb/0x180 Feb 11 07:49:54 l1 [ 720.432280] [] ? sys_write+0x53/0xa0 Feb 11 07:49:54 l1 [ 720.432285] [] ? system_call_fastpath+0x16/0x1b Feb 11 07:49:54 l1 [ 720.432291] INFO: task flush-9:3:4874 blocked for more than 120 seconds. Feb 11 07:49:54 l1 [ 720.432295] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Feb 11 07:49:54 l1 [ 720.432298] flush-9:3 D Feb 11 07:49:54 l1 ffff88021948d750 Feb 11 07:49:54 l1 0 4874 2 0x00000000 Feb 11 07:49:54 l1 [ 720.432306] ffff88023fa3c8c0 Feb 11 07:49:54 l1 0000000000000046 Feb 11 07:49:54 l1 000000001948d750 Feb 11 07:49:54 l1 ffffffff8163d788 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432322] 000000000000c928 Feb 11 07:49:54 l1 ffff8801f79fbfd8 Feb 11 07:49:54 l1 ffff88021948d750 Feb 11 07:49:54 l1 ffff88021948d9c8 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432329] 000000013f467bb0 Feb 11 07:49:54 l1 ffffffff8108c183 Feb 11 07:49:54 l1 0000000000000000 Feb 11 07:49:54 l1 ffff88021948d9c8 Feb 11 07:49:54 l1 Feb 11 07:49:54 l1 [ 720.432336] Call Trace: Feb 11 07:49:54 l1 [ 720.432340] [] ? find_lock_page+0x23/0x80 Feb 11 07:49:54 l1 [ 720.432343] [] ? find_or_create_page+0x41/0xc0 Feb 11 07:49:54 l1 [ 720.432348] [] ? schedule_timeout+0x195/0x1f0 Feb 11 07:49:54 l1 [ 720.432351] [] ? __down+0x61/0xa0 Feb 11 07:49:54 l1 [ 720.432356] [] ? down+0x46/0x50 Feb 11 07:49:54 l1 [ 720.432361] [] ? _xfs_buf_find+0x134/0x220 Feb 11 07:49:54 l1 [ 720.432365] [] ? xfs_buf_get_flags+0x6e/0x190 Feb 11 07:49:54 l1 [ 720.432368] [] ? xfs_buf_read_flags+0x12/0xa0 Feb 11 07:49:54 l1 [ 720.432373] [] ? xfs_trans_read_buf+0x1f6/0x350 Feb 11 07:49:54 l1 [ 720.432378] [] ? xfs_read_agf+0x68/0x190 Feb 11 07:49:54 l1 [ 720.432382] [] ? xfs_alloc_read_agf+0x30/0xd0 Feb 11 07:49:54 l1 [ 720.432386] [] ? xfs_alloc_fix_freelist+0x379/0x450 Feb 11 07:49:54 l1 [ 720.432391] [] ? xfs_iext_remove+0x35/0x80 Feb 11 07:49:54 l1 [ 720.432396] [] ? xfs_bmap_add_extent_delay_real+0x5ef/0x11a0 Feb 11 07:49:54 l1 [ 720.432399] [] ? xfs_trans_log_buf+0x63/0xa0 Feb 11 07:49:54 l1 [ 720.432404] [] ? xlog_state_get_iclog_space+0x60/0x2c0 Feb 11 07:49:54 l1 [ 720.432410] [] ? __down_read+0x17/0xae Feb 11 07:49:54 l1 [ 720.432415] [] ? xfs_alloc_vextent+0x310/0x4b0 Feb 11 07:49:54 l1 [ 720.432420] [] ? xfs_bmap_btalloc+0x598/0xa40 Feb 11 07:49:54 l1 [ 720.432426] [] ? xfs_bmapi+0x9e2/0x11a0 Feb 11 07:49:54 l1 [ 720.432430] [] ? xfs_trans_reserve+0x9f/0x210 Feb 11 07:49:54 l1 [ 720.432434] [] ? xfs_iomap_write_allocate+0x23e/0x3b0 Feb 11 07:49:54 l1 [ 720.432437] [] ? xfs_iomap+0x2c0/0x300 Feb 11 07:49:54 l1 [ 720.432441] [] ? xfs_map_blocks+0x25/0x30 Feb 11 07:49:54 l1 [ 720.432445] [] ? xfs_page_state_convert+0x414/0x6c0 Feb 11 07:49:54 l1 [ 720.432451] [] ? radix_tree_gang_lookup_tag_slot+0xc3/0xf0 Feb 11 07:49:54 l1 [ 720.432455] [] ? xfs_vm_writepage+0x77/0x130 Feb 11 07:49:54 l1 [ 720.432459] [] ? __writepage+0xa/0x40 Feb 11 07:49:54 l1 [ 720.432462] [] ? write_cache_pages+0x1df/0x3c0 Feb 11 07:49:54 l1 [ 720.432466] [] ? __writepage+0x0/0x40 Feb 11 07:49:54 l1 [ 720.432472] [] ? writeback_single_inode+0xd2/0x390 Feb 11 07:49:54 l1 [ 720.432476] [] ? writeback_inodes_wb+0x3ff/0x5e0 Feb 11 07:49:54 l1 [ 720.432480] [] ? wb_writeback+0x11e/0x1f0 Feb 11 07:49:54 l1 [ 720.432485] [] ? try_to_del_timer_sync+0x5e/0x90 Feb 11 07:49:54 l1 [ 720.432489] [] ? wb_do_writeback+0x17b/0x180 Feb 11 07:49:54 l1 [ 720.432493] [] ? bdi_writeback_task+0x5d/0xa0 Feb 11 07:49:54 l1 [ 720.432497] [] ? bdi_start_fn+0x0/0xf0 Feb 11 07:49:54 l1 [ 720.432500] [] ? bdi_start_fn+0x7e/0xf0 Feb 11 07:49:54 l1 [ 720.432503] [] ? bdi_start_fn+0x0/0xf0 Feb 11 07:49:54 l1 [ 720.432506] [] ? kthread+0x96/0xa0 Feb 11 07:49:54 l1 [ 720.432511] [] ? child_rip+0xa/0x20 Feb 11 07:49:54 l1 [ 720.432515] [] ? kthread+0x0/0xa0 Feb 11 07:49:54 l1 [ 720.432519] [] ? child_rip+0x0/0x20 Justin. From jpiszcz@lucidpixels.com Thu Feb 11 08:12:19 2010 X-Spam-Checker-Version: SpamAssassin 3.3.0-rupdated (updated) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=-0.1 required=5.0 tests=AWL,BAYES_00, FH_DATE_PAST_20XX autolearn=no version=3.3.0-rupdated Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id o1BECJoM151656 for ; Thu, 11 Feb 2010 08:12:19 -0600 X-ASG-Debug-ID: 1265897611-412501220000-NocioJ X-Barracuda-URL: http://cuda.sgi.com:80/cgi-bin/mark.cgi Received: from lucidpixels.com (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id EE0EE1399FF8 for ; Thu, 11 Feb 2010 06:13:31 -0800 (PST) Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by cuda.sgi.com with ESMTP id RTY2Bn7ap5GqYbmq for ; Thu, 11 Feb 2010 06:13:31 -0800 (PST) Received: by lucidpixels.com (Postfix, from userid 1001) id 58AE738DFB8; Thu, 11 Feb 2010 09:13:31 -0500 (EST) Date: Thu, 11 Feb 2010 09:13:31 -0500 (EST) From: Justin Piszcz To: linux-kernel@vger.kernel.org, linux-raid@vger.kernel.org, xfs@oss.sgi.com, Alan Piszcz X-ASG-Orig-Subj: Re: 2.6.32.3 x86_64 - XFS hangs, all I/O to D-state bug (again) Subject: Re: 2.6.32.3 x86_64 - XFS hangs, all I/O to D-state bug (again) In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Barracuda-Connect: lucidpixels.com[75.144.35.66] X-Barracuda-Start-Time: 1265897612 X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Virus-Scanned: by cuda.sgi.com at sgi.com X-Barracuda-Spam-Score: -1.52 X-Barracuda-Spam-Status: No, SCORE=-1.52 using per-user scores of TAG_LEVEL=2.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.1 tests=BSF_RULE7568M X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.22255 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.50 BSF_RULE7568M Custom Rule 7568M X-Virus-Scanned: ClamAV version 0.94.2, clamav-milter version 0.94.2 on oss.sgi.com X-Virus-Status: Clean The errors in the e-mail had bad formatting, please see the following: http://home.comcast.net/~jpiszcz/20100211/crash.txt In addition, the linux-raid list had a typo in it, fixed. On Thu, 11 Feb 2010, Justin Piszcz wrote: > Hello, > > While tarring and compressing (bzip2) a lot of files, the following error > occurred, note the output is not clean because this was taken from > netconsole. > > When this occurs, the host cannot be rebooted with reboot/proceses cannot > be killed and the box locks up. There are no apparent hardware issues. > > Before, asterisk would trigger this bug, since asterisk no lon