From fm316383@gmail.com Fri Jan 1 09:44:59 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM,HTML_MESSAGE autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 71BB57F5D for ; Fri, 1 Jan 2016 09:44:59 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 6134B304059 for ; Fri, 1 Jan 2016 07:44:56 -0800 (PST) X-ASG-Debug-ID: 1451663093-04cbb0101a1b0a30001-NocioJ Received: from nh604-vm9.bullet.mail.ssk.yahoo.co.jp (nh604-vm9.bullet.mail.ssk.yahoo.co.jp [182.22.90.66]) by cuda.sgi.com with ESMTP id KWW3eGgt0yF7VmSk for ; Fri, 01 Jan 2016 07:44:54 -0800 (PST) X-Barracuda-Envelope-From: fm316383@gmail.com X-Barracuda-Apparent-Source-IP: 182.22.90.66 Received: from [182.22.66.104] by nh604.bullet.mail.ssk.yahoo.co.jp with NNFMP; 01 Jan 2016 15:44:52 -0000 Received: from [182.22.91.132] by t602.bullet.mail.ssk.yahoo.co.jp with NNFMP; 01 Jan 2016 15:44:52 -0000 Received: from [127.0.0.1] by omp605.mail.ssk.yahoo.co.jp with NNFMP; 01 Jan 2016 15:44:52 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 522202.35882.bm@omp605.mail.ssk.yahoo.co.jp Received: (qmail 83860 invoked by uid 60001); 1 Jan 2016 15:44:51 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=yj20050223; d=yahoo.co.jp; h=Message-ID:X-YMail-OSG:Received:X-RocketYMMF:X-Mailer:X-YMail-JAS:Sender:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=L4eajkGMPwDRomRElhrBG35hI4T+i34zp/M2tkefJtSDMQ8UZzKSlmmxDYC2BpF66SgOMTxzLsHT6uCP2mSOO122SQ7AKCiKM9a8jf3glcLJ6GdnNxSR7Tk+D7WDNvDb ; Message-ID: <640672.76249.qm@web101604.mail.kks.yahoo.co.jp> X-YMail-OSG: gBRL.KMVM1mwxlasxO_TDXkTSoDvVGKNcf2xxbADtgRzn3FOiWu1qJtA9JWZZSL63mI9_y4voOTOW1JpAIHmx4I9ptsD5BoXoTkApV6MpDGhQuVHt7s6yuqslMNJ8MoZtntFR8h1fA760GZ5vznhWlJMTU2FV228suq8guC91aS48cZ8dPz.fCvhfFeXjR8FQTRfrAHAtw41NTm.OA5Dyf5uxAKjECKGhyj3d6BHSEzCNIXPpbwWL_XyqCsUTOYwODN5nfycVAVmW3RzFZd3cSEpq8z2NQQ6qcCvin3aQCTri8QCfRSxp37YBsRARtTkrkQPQoUwy8BbaScBCWRo2j.uE.Z1crG6SUPtqPXvJKMurDc4n0Y4dsxtmf05V0nMLKV79z1DeFF5zi4dJYWg959rxpko1VEm0s.ugGUABuG7GlNb8EMXwAGPVTycJaFHFjCCKWijZIz8Y3NHNJFQ1bGQ6n6Tmr.rkF2vJ_27_EUT6oTyc6eHohzPPiUQi3EaWfHK_1uu2gCFwH8pIEEla7LobfRqgx_xB..vkj6bwb3YzVzsr.v9vkl7doVwrwf27OsqsgNLAC5Amni0f2mFzo4OHVhZk2o.XoY5LAqDz9Xm6qoa_RoxXAUc9p3QmnZIBmit.rxNAsSKkmhv3DRBgMrOUONODYnCMcd1iDy73.DdWaynxML7tTMPV2.VsrL2c1_fTqc79PWZrsb1L6FADjPoxCQBCeg7miNZ7O2vf17268q6BkD1SLclhzNCvf0De8.nsd3dqOGDCaZu.C8Eztn1lpnMMvMo Received: from [197.148.123.121] by web101604.mail.kks.yahoo.co.jp via HTTP; Sat, 02 Jan 2016 00:44:51 JST X-RocketYMMF: mrglenfrank X-Mailer: YahooMailWebService/0.8.111_67 X-YMail-JAS: nvTofeEVM1mxKUXuA.0naHAuYMqg8imDHQ9yXAVastJ4NQPM8C2Pi70HoxHReGFUn1mHpgwxTNUFjqcbvfg6qnej55VE3d0tipldPTA9CF3QQf4eIUHXM.f7cTO5t4LX8Czi Sender: mrglenfrank@yahoo.co.jp Date: Sat, 2 Jan 2016 00:44:51 +0900 (JST) From: Fred Martins Reply-To: Fred Martins Subject: May Almighty God Bless You!!! To: undisclosed recipients: ; X-ASG-Orig-Subj: May Almighty God Bless You!!! MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="2024049926-1842152024-1451663091=:76249" X-Barracuda-Connect: nh604-vm9.bullet.mail.ssk.yahoo.co.jp[182.22.90.66] X-Barracuda-Start-Time: 1451663093 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.46 X-Barracuda-Spam-Status: No, SCORE=0.46 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE, PLING_PLING X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25749 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 0.00 HTML_MESSAGE BODY: HTML included in message 0.46 PLING_PLING Subject has lots of exclamation marks --2024049926-1842152024-1451663091=:76249 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable =A0May Almighty God Bless You!!!=0A=0AI have a Mutual/Beneficial Business P= roject that would be benificial to you. I only have two questions to ask of= you,=0Aif you are interested.=0A=0A1. Can you handle this project?=0A2. Ca= n I give you this trust ?=0A=0APlease note that the deal requires high leve= l of maturity, honesty and secrecy. This will involve moving=A0 some money = from my office, on trust to your hands or bank account.=0AAlso note that i = will do everything to make sure that the money is moved as a purely legitim= ate fund, so you will not be exposed to any risk.=0AI request for your full= co-operation. I will give you details and procedure when I recieve your re= ply.=0ATo commence this transaction, I require you to immediately indicate= =A0 your interest by a return reply.=0AI will be waiting for your response = in a timely manner.=0A=0ABest Regard,=0AMr. Fred Martins=0A --2024049926-1842152024-1451663091=:76249 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
 May Almighty God Bless You!!!
I have a Mutual/Beneficial Business Project that would be benificial to y= ou. I only have two questions to ask of you,
if you are interested.
<= br>1. Can you handle this project?
2. Can I give you this trust ?
Please note that the deal requires high level of maturity, honesty and sec= recy. This will involve moving  some money from my office, on trust to= your hands or bank account.
Also note that i will do everything to make= sure that the money is moved as a purely legitimate fund, so you will not = be exposed to any risk.
I request for your full co-operation. I will giv= e you details and procedure when I recieve your reply.
To commence this = transaction, I require you to immediately indicate  your interest by a= return reply.
I will be waiting for your response in a timely manner.

Best Regard,
Mr. Fred Martins
--2024049926-1842152024-1451663091=:76249-- From sandeen@sandeen.net Fri Jan 1 17:40:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,MIME_QP_LONG_LINE autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D842529DF6 for ; Fri, 1 Jan 2016 17:40:17 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 695D2AC002 for ; Fri, 1 Jan 2016 15:40:13 -0800 (PST) X-ASG-Debug-ID: 1451691607-04cbb0101d1b61c0001-NocioJ Received: from sandeen.net (sandeen.net [63.231.237.45]) by cuda.sgi.com with ESMTP id JRn8duPdcVVeiUxD for ; Fri, 01 Jan 2016 15:40:07 -0800 (PST) X-Barracuda-Envelope-From: sandeen@sandeen.net X-Barracuda-Apparent-Source-IP: 63.231.237.45 Received: from [30.34.82.18] (unknown [172.56.16.88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by sandeen.net (Postfix) with ESMTPSA id EC8C463C5FF0; Fri, 1 Jan 2016 17:40:06 -0600 (CST) From: Eric Sandeen Content-Type: multipart/alternative; boundary=Apple-Mail-92929011-3B27-4518-AAA8-7F1F5F943123 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Subject: Re: xfs_growfs can expand capacity,then how to reduce xfs filesystem capacity? Message-Id: X-ASG-Orig-Subj: Re: xfs_growfs can expand capacity,then how to reduce xfs filesystem capacity? Date: Fri, 1 Jan 2016 15:40:06 -0800 References: <1CB94550540AE44E9CAAE8CD5AF40BF846D18C5E@H3CMLB12-EX.srv.huawei-3com.com> In-Reply-To: <1CB94550540AE44E9CAAE8CD5AF40BF846D18C5E@H3CMLB12-EX.srv.huawei-3com.com> To: "zhao.mingyue@h3c.com" , xfs@oss.sgi.com X-Mailer: iPhone Mail (13C75) X-Barracuda-Connect: sandeen.net[63.231.237.45] X-Barracuda-Start-Time: 1451691607 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.92 X-Barracuda-Spam-Status: No, SCORE=0.92 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA085, HTML_MESSAGE, MIME_QP_LONG_LINE, MIME_QP_LONG_LINE_2 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25757 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 HTML_MESSAGE BODY: HTML included in message 0.00 MIME_QP_LONG_LINE RAW: Quoted-printable line longer than 76 chars 0.82 MIME_QP_LONG_LINE_2 RAW: Quoted-printable line longer than 76 chars 0.10 BSF_SC0_SA085 Custom Rule SA085 --Apple-Mail-92929011-3B27-4518-AAA8-7F1F5F943123 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable http://xfs.org/index.php/XFS_FAQ#Q:_Is_there_a_way_to_make_a_XFS_filesystem_= larger_or_smaller.3F > On Dec 2, 2015, at 6:11 PM, "zhao.mingyue@h3c.com" w= rote: >=20 > =20 > --------------------------------------------------------------------------= ----------------------------------------------------------- > =E6=9C=AC=E9=82=AE=E4=BB=B6=E5=8F=8A=E5=85=B6=E9=99=84=E4=BB=B6=E5=90=AB=E6= =9C=89=E6=9D=AD=E5=B7=9E=E5=8D=8E=E4=B8=89=E9=80=9A=E4=BF=A1=E6=8A=80=E6=9C=AF= =E6=9C=89=E9=99=90=E5=85=AC=E5=8F=B8=E7=9A=84=E4=BF=9D=E5=AF=86=E4=BF=A1=E6=81= =AF=EF=BC=8C=E4=BB=85=E9=99=90=E4=BA=8E=E5=8F=91=E9=80=81=E7=BB=99=E4=B8=8A=E9= =9D=A2=E5=9C=B0=E5=9D=80=E4=B8=AD=E5=88=97=E5=87=BA > =E7=9A=84=E4=B8=AA=E4=BA=BA=E6=88=96=E7=BE=A4=E7=BB=84=E3=80=82=E7=A6=81=E6= =AD=A2=E4=BB=BB=E4=BD=95=E5=85=B6=E4=BB=96=E4=BA=BA=E4=BB=A5=E4=BB=BB=E4=BD=95= =E5=BD=A2=E5=BC=8F=E4=BD=BF=E7=94=A8=EF=BC=88=E5=8C=85=E6=8B=AC=E4=BD=86=E4=B8= =8D=E9=99=90=E4=BA=8E=E5=85=A8=E9=83=A8=E6=88=96=E9=83=A8=E5=88=86=E5=9C=B0=E6= =B3=84=E9=9C=B2=E3=80=81=E5=A4=8D=E5=88=B6=E3=80=81 > =E6=88=96=E6=95=A3=E5=8F=91=EF=BC=89=E6=9C=AC=E9=82=AE=E4=BB=B6=E4=B8=AD=E7= =9A=84=E4=BF=A1=E6=81=AF=E3=80=82=E5=A6=82=E6=9E=9C=E6=82=A8=E9=94=99=E6=94=B6= =E4=BA=86=E6=9C=AC=E9=82=AE=E4=BB=B6=EF=BC=8C=E8=AF=B7=E6=82=A8=E7=AB=8B=E5=8D= =B3=E7=94=B5=E8=AF=9D=E6=88=96=E9=82=AE=E4=BB=B6=E9=80=9A=E7=9F=A5=E5=8F=91=E4= =BB=B6=E4=BA=BA=E5=B9=B6=E5=88=A0=E9=99=A4=E6=9C=AC > =E9=82=AE=E4=BB=B6=EF=BC=81 > This e-mail and its attachments contain confidential information from H3C,= which is=20 > intended only for the person or entity whose address is listed above. Any u= se of the=20 > information contained herein in any way (including, but not limited to, to= tal or partial=20 > disclosure, reproduction, or dissemination) by persons other than the inte= nded=20 > recipient(s) is prohibited. If you receive this e-mail in error, please no= tify the sender=20 > by phone or email immediately and delete it! > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs --Apple-Mail-92929011-3B27-4518-AAA8-7F1F5F943123 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

On Dec 2, 2015, at 6:11 PM, "zhao.mingyue@h3c.com" <zhao.mingyue@h3c.com> wrote:

=

 

--------------------------------------= ----------------------------------------------------------------------------= -------------------
=E6=9C=AC=E9=82=AE=E4=BB=B6=E5=8F=8A=E5=85=B6=E9=99=84=E4=BB=B6=E5=90= =AB=E6=9C=89=E6=9D=AD=E5=B7=9E=E5=8D=8E=E4=B8=89=E9=80=9A=E4=BF=A1=E6=8A=80=E6= =9C=AF=E6=9C=89=E9=99=90=E5=85=AC=E5=8F=B8=E7=9A=84=E4=BF=9D=E5=AF=86=E4=BF=A1= =E6=81=AF=EF=BC=8C=E4=BB=85=E9=99=90=E4=BA=8E=E5=8F=91=E9=80=81=E7=BB=99=E4=B8= =8A=E9=9D=A2=E5=9C=B0=E5=9D=80=E4=B8=AD=E5=88=97=E5=87=BA
=E7=9A=84=E4=B8=AA=E4=BA=BA=E6=88=96=E7=BE=A4=E7=BB=84=E3=80=82=E7=A6= =81=E6=AD=A2=E4=BB=BB=E4=BD=95=E5=85=B6=E4=BB=96=E4=BA=BA=E4=BB=A5=E4=BB=BB=E4= =BD=95=E5=BD=A2=E5=BC=8F=E4=BD=BF=E7=94=A8=EF=BC=88=E5=8C=85=E6=8B=AC=E4=BD=86= =E4=B8=8D=E9=99=90=E4=BA=8E=E5=85=A8=E9=83=A8=E6=88=96=E9=83=A8=E5=88=86=E5=9C= =B0=E6=B3=84=E9=9C=B2=E3=80=81=E5=A4=8D=E5=88=B6=E3=80=81
=E6=88=96=E6=95=A3=E5=8F=91=EF=BC=89=E6=9C=AC=E9=82=AE=E4=BB=B6=E4=B8= =AD=E7=9A=84=E4=BF=A1=E6=81=AF=E3=80=82=E5=A6=82=E6=9E=9C=E6=82=A8=E9=94=99=E6= =94=B6=E4=BA=86=E6=9C=AC=E9=82=AE=E4=BB=B6=EF=BC=8C=E8=AF=B7=E6=82=A8=E7=AB=8B= =E5=8D=B3=E7=94=B5=E8=AF=9D=E6=88=96=E9=82=AE=E4=BB=B6=E9=80=9A=E7=9F=A5=E5=8F= =91=E4=BB=B6=E4=BA=BA=E5=B9=B6=E5=88=A0=E9=99=A4=E6=9C=AC
=E9=82=AE=E4=BB=B6=EF=BC=81
This e-mail and its attac= hments contain confidential information from H3C, which is
intended only for the person or entity whose address is listed above. Any us= e of the
information contained herein in any way (including, but not limited to, tota= l or partial
disclosure, reproduction, or dissemination) by persons other than the intend= ed
recipient(s) is prohibited. If you receive this e-mail in error, please noti= fy the sender
by phone or email immediately and delete it!
____________________= ___________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/l= istinfo/xfs
= --Apple-Mail-92929011-3B27-4518-AAA8-7F1F5F943123-- From s_w9gc9x@student.uni-kassel.de Sat Jan 2 18:26:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=LOTS_OF_MONEY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 774237F5D for ; Sat, 2 Jan 2016 18:26:18 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 187B6AC005 for ; Sat, 2 Jan 2016 16:26:15 -0800 (PST) X-ASG-Debug-ID: 1451780767-04cbb0101b1ca9b0001-NocioJ Received: from mx4.its.uni-kassel.de (mx4.its.uni-kassel.de [141.51.12.63]) by cuda.sgi.com with ESMTP id VtOYFZXVplPKhw3S for ; Sat, 02 Jan 2016 16:26:08 -0800 (PST) X-Barracuda-Envelope-From: s_w9gc9x@student.uni-kassel.de X-Barracuda-Apparent-Source-IP: 141.51.12.63 X-IPAS-Result: A2Ao5wAYaohW/5GqM40bQ4MkCDAwHEIBAQ6DN3SELaopi1EOgwiCXgEWgUwBAQEBAQECgQkYAQGCGIF+LkMoIAImAkkWLYUvB4F3bAqfBYddiBKEDIJJikKBAY56EQGDAQwuE4E2BYJwhnKDV3eIVgGBNYNPiRSBFIcLPAQjhTETijSDc2SBSgEKAQEBAYJPIIQCgUIBAQE X-IronPort-AV: E=Sophos;i="5.20,514,1444687200"; d="scan'208";a="25896876" X-UKS-Mail: fzgewiuhfgadjshfgeijrgzaiwefghieuwfgehjfgiw54567456757fguhklghfrg57gtftjhjtd X-CGP-ClamAV-Result: CLEAN X-VirusScanner: Niversoft's CGPClamav Helper v1.18.7a (ClamAV engine v0.98.7) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Charity Donation To: Me X-ASG-Orig-Subj: Charity Donation From: "Jeff Skoll" Date: Sat, 02 Jan 2016 18:49:39 -0500 Reply-To: jeff.skoll@skollcharity.org X-Priority: 1 (High) Sensitivity: Company-Confidential Message-ID: X-Barracuda-Connect: mx4.its.uni-kassel.de[141.51.12.63] X-Barracuda-Start-Time: 1451780767 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25782 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) Hi, My name is Jeffrey Skoll, a philanthropist and the founder of one of the la= rgest private foundations in the world. I believe strongly in =E2=80=98givi= ng while living.=E2=80=99 I had one idea that never changed in my mind =E2= =80=94 that you should use your wealth to help people and I have decided to= secretly give USD2.498 Million to a randomly selected individual. On recei= pt of this email, you should count yourself as the individual. Kindly get b= ack to me at your earliest convenience, so I know your email address is val= id. Visit the web page to know more about me: http://www.theglobeandmail.com/ne= ws/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article42= 09888/ or you can read an article of me on Wikipedia. Regards, Jeffrey Skoll. From s_w9gc9x@student.uni-kassel.de Sat Jan 2 18:59:45 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=LOTS_OF_MONEY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 892707F5D for ; Sat, 2 Jan 2016 18:59:45 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5A7C38F8033 for ; Sat, 2 Jan 2016 16:59:45 -0800 (PST) X-ASG-Debug-ID: 1451782779-04cbb0101a1cb020001-NocioJ Received: from mx4.its.uni-kassel.de (mx4.its.uni-kassel.de [141.51.12.63]) by cuda.sgi.com with ESMTP id 1AI2p5QEgmgauh2b for ; Sat, 02 Jan 2016 16:59:40 -0800 (PST) X-Barracuda-Envelope-From: s_w9gc9x@student.uni-kassel.de X-Barracuda-Apparent-Source-IP: 141.51.12.63 X-IPAS-Result: A2Ao5wAycYhW/5CqM40bQ4MkCDAwHEIBAQ6DN3SELaopi1EOgwiCXgEWgUwBAQEBAQECgQkYAQGCGIF+LkMoIAImAkkWLYUvB4F3bAqfAYddiBKEDIJJikSBAY56EQGDAQwuE4E2BYJwhnKDV3eIVgGBNYNPiRSBFIcLPAQjhTETijSDc2SBSgEKAQEBAYJPIIQCgUIBAQE X-IronPort-AV: E=Sophos;i="5.20,514,1444687200"; d="scan'208";a="25969998" X-UKS-Mail: fzgewiuhfgadjshfgeijrgzaiwefghieuwfgehjfgiw54567456757fguhklghfrg57gtftjhjtd X-CGP-ClamAV-Result: CLEAN X-VirusScanner: Niversoft's CGPClamav Helper v1.18.7a (ClamAV engine v0.98.7) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Charity Donation To: Me X-ASG-Orig-Subj: Charity Donation From: "Jeff Skoll" Date: Sat, 02 Jan 2016 19:36:28 -0500 Reply-To: jeff.skoll@skollcharity.org X-Priority: 1 (High) Sensitivity: Company-Confidential Message-ID: X-Barracuda-Connect: mx4.its.uni-kassel.de[141.51.12.63] X-Barracuda-Start-Time: 1451782779 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25782 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) Hi, My name is Jeffrey Skoll, a philanthropist and the founder of one of the la= rgest private foundations in the world. I believe strongly in =E2=80=98givi= ng while living.=E2=80=99 I had one idea that never changed in my mind =E2= =80=94 that you should use your wealth to help people and I have decided to= secretly give USD2.498 Million to a randomly selected individual. On recei= pt of this email, you should count yourself as the individual. Kindly get b= ack to me at your earliest convenience, so I know your email address is val= id. Visit the web page to know more about me: http://www.theglobeandmail.com/ne= ws/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article42= 09888/ or you can read an article of me on Wikipedia. Regards, Jeffrey Skoll. From s_w9gc9x@student.uni-kassel.de Sun Jan 3 02:00:03 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=LOTS_OF_MONEY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 20DB47F5D for ; Sun, 3 Jan 2016 02:00:03 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 02206304032 for ; Sun, 3 Jan 2016 00:00:03 -0800 (PST) X-ASG-Debug-ID: 1451807990-04cb6c3032407b40002-NocioJ Received: from mx4.its.uni-kassel.de (mx4.its.uni-kassel.de [141.51.12.63]) by cuda.sgi.com with ESMTP id 4UDV6KH7mAXsW3Ik for ; Sat, 02 Jan 2016 23:59:59 -0800 (PST) X-Barracuda-Envelope-From: s_w9gc9x@student.uni-kassel.de X-Barracuda-Apparent-Source-IP: 141.51.12.63 X-IPAS-Result: A2BhqQBK1IhW/5GqM40bQ4MkCDAwHAFBAQEOgzd0hC2qKYtRDoMIgl4BFoFMAQEBAQEBAoEJGAEBghiBfi5DKCACJgJJFi2FLweBd2wKnnCHXYgShAyCSYpEgQGOehEBgwEMLhOBNgWCcIZyg1d3iFYBgTWDT4kUgRSHCzwEI4UxE4o0g3NkgUoBCgEBAQGCTyCEAoFCAQEB X-IronPort-AV: E=Sophos;i="5.20,515,1444687200"; d="scan'208";a="26079350" X-UKS-Mail: fzgewiuhfgadjshfgeijrgzaiwefghieuwfgehjfgiw54567456757fguhklghfrg57gtftjhjtd X-CGP-ClamAV-Result: CLEAN X-VirusScanner: Niversoft's CGPClamav Helper v1.18.7a (ClamAV engine v0.98.7) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Charity Donation To: Me X-ASG-Orig-Subj: Charity Donation From: "Jeff Skoll" Date: Sun, 03 Jan 2016 02:59:54 -0500 Reply-To: jeff.skoll@skollcharity.org X-Priority: 1 (High) Sensitivity: Company-Confidential Message-ID: X-Barracuda-Connect: mx4.its.uni-kassel.de[141.51.12.63] X-Barracuda-Start-Time: 1451807994 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25789 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) Hi, My name is Jeffrey Skoll, a philanthropist and the founder of one of the la= rgest private foundations in the world. I believe strongly in =E2=80=98givi= ng while living.=E2=80=99 I had one idea that never changed in my mind =E2= =80=94 that you should use your wealth to help people and I have decided to= secretly give USD2.498 Million to a randomly selected individual. On recei= pt of this email, you should count yourself as the individual. Kindly get b= ack to me at your earliest convenience, so I know your email address is val= id. Visit the web page to know more about me: http://www.theglobeandmail.com/ne= ws/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article42= 09888/ or you can read an article of me on Wikipedia. Regards, Jeffrey Skoll. From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:09:41 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 7DE1E7F37 for ; Sun, 3 Jan 2016 06:09:41 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 2D4FEAC004 for ; Sun, 3 Jan 2016 04:09:41 -0800 (PST) X-ASG-Debug-ID: 1451822978-04bdf0264e42fb40001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id mdBisgMlHur88DXF (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:09:38 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from [83.175.99.196] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhTd-0007fZ-Ru; Sun, 03 Jan 2016 12:09:38 +0000 From: Christoph Hellwig To: darrick.wong@oracle.com Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: [PATCH 1/3] xfs: pass inode instead of file to xfs_reflink_dirty_range Date: Sun, 3 Jan 2016 13:07:51 +0100 X-ASG-Orig-Subj: [PATCH 1/3] xfs: pass inode instead of file to xfs_reflink_dirty_range Message-Id: <1451822873-12969-2-git-send-email-hch@lst.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1451822873-12969-1-git-send-email-hch@lst.de> References: <1451822873-12969-1-git-send-email-hch@lst.de> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451822978 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 We don't actually need a file for write_begin/end, those can be passed as NULL for disk based file systems. This is important as we won't even have a file pointer during a truncate operation, which gives a guaranteed NULL pointer dererference with the current code. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_reflink.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index da4a715..4a3f0ee 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -1316,7 +1316,7 @@ out_error: */ STATIC int xfs_reflink_dirty_range( - struct file *filp, + struct inode *inode, xfs_off_t pos, xfs_off_t len) { @@ -1330,14 +1330,14 @@ xfs_reflink_dirty_range( unsigned long bytes; /* Bytes to write to page */ void *fsdata; - mapping = filp->f_mapping; + mapping = inode->i_mapping; a_ops = mapping->a_ops; flags = AOP_FLAG_UNINTERRUPTIBLE; do { offset = (pos & (PAGE_CACHE_SIZE - 1)); bytes = min_t(unsigned long, len, PAGE_CACHE_SIZE) - offset; - rpage = xfs_get_page(file_inode(filp), pos); + rpage = xfs_get_page(inode, pos); if (IS_ERR(rpage)) { error = PTR_ERR(rpage); break; @@ -1346,24 +1346,24 @@ xfs_reflink_dirty_range( break; } - error = a_ops->write_begin(filp, mapping, pos, bytes, flags, + error = a_ops->write_begin(NULL, mapping, pos, bytes, flags, &page, &fsdata); page_cache_release(rpage); if (error < 0) break; - trace_xfs_reflink_unshare_page(file_inode(filp), page, + trace_xfs_reflink_unshare_page(inode, page, pos, bytes); if (!PageUptodate(page)) { - pr_err("%s: STALE? ino=%lu pos=%llu\n", - __func__, filp->f_inode->i_ino, pos); + pr_err("%s: STALE? ino=%llu pos=%llu\n", + __func__, XFS_I(inode)->i_ino, pos); WARN_ON(1); } if (mapping_writably_mapped(mapping)) flush_dcache_page(page); - error = a_ops->write_end(filp, mapping, pos, bytes, bytes, + error = a_ops->write_end(NULL, mapping, pos, bytes, bytes, page, fsdata); if (error < 0) break; @@ -1454,7 +1454,7 @@ xfs_reflink_dirty_extents( flen = XFS_FSB_TO_B(mp, rlen); if (fpos + flen > isize) flen = isize - fpos; - error = xfs_reflink_dirty_range(filp, fpos, flen); + error = xfs_reflink_dirty_range(VFS_I(ip), fpos, flen); xfs_ilock(ip, XFS_ILOCK_EXCL); if (error) goto out; -- 1.9.1 From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:09:43 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id ADEE37F47 for ; Sun, 3 Jan 2016 06:09:43 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 3EA59AC005 for ; Sun, 3 Jan 2016 04:09:39 -0800 (PST) X-ASG-Debug-ID: 1451822976-04cbb0101a1d42b0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id e4t8MBBEmU0DfEIx (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:09:36 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from [83.175.99.196] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhTa-0007fK-Vq; Sun, 03 Jan 2016 12:09:35 +0000 From: Christoph Hellwig To: darrick.wong@oracle.com Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: reflink fixes Date: Sun, 3 Jan 2016 13:07:50 +0100 X-ASG-Orig-Subj: reflink fixes Message-Id: <1451822873-12969-1-git-send-email-hch@lst.de> X-Mailer: git-send-email 1.9.1 X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451822976 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Hi Darrick, below are three patches I need to xfstests to pass reliably for the reflink code. I actually still see a lot of issues when testing NFS clones on XFS due to extents hanging on the COW for too long, but that's a separate issue I still need to investigate. From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:09:44 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 14A247F51 for ; Sun, 3 Jan 2016 06:09:44 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 9A826AC001 for ; Sun, 3 Jan 2016 04:09:43 -0800 (PST) X-ASG-Debug-ID: 1451822981-04cbb0101b1d42d0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id eyFl5Gq20cYFZpMR (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:09:42 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from [83.175.99.196] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhTg-0007hN-Ma; Sun, 03 Jan 2016 12:09:41 +0000 From: Christoph Hellwig To: darrick.wong@oracle.com Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: [PATCH 2/3] xfs: only end a COW operation in xfs_zero_remaining_bytes if we started one Date: Sun, 3 Jan 2016 13:07:52 +0100 X-ASG-Orig-Subj: [PATCH 2/3] xfs: only end a COW operation in xfs_zero_remaining_bytes if we started one Message-Id: <1451822873-12969-3-git-send-email-hch@lst.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1451822873-12969-1-git-send-email-hch@lst.de> References: <1451822873-12969-1-git-send-email-hch@lst.de> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451822981 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Without these we can see NULL pointer dereferences due to a non-existing COW fork during xfstests runs. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_bmap_util.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 7bee3c7..e777095 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -1226,13 +1226,15 @@ xfs_zero_remaining_bytes( error = xfs_bwrite(bp); xfs_buf_relse(bp); - if (error) { - err2 = xfs_reflink_end_cow_failed(ip, offset, + if (should_fork) { + if (error) { + err2 = xfs_reflink_end_cow_failed(ip, offset, + lastoffset - offset + 1); + return error; + } + error = xfs_reflink_end_cow(ip, offset, lastoffset - offset + 1); - return error; } - error = xfs_reflink_end_cow(ip, offset, - lastoffset - offset + 1); if (error) return error; } -- 1.9.1 From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:09:50 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 974DE7F47 for ; Sun, 3 Jan 2016 06:09:50 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 86FBE8F8037 for ; Sun, 3 Jan 2016 04:09:50 -0800 (PST) X-ASG-Debug-ID: 1451822984-04cb6c303140acf0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id uHuhLrmfm9B97UfG (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:09:44 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from [83.175.99.196] (helo=localhost) by bombadil.infradead.org with esmtpsa (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhTj-0007hV-HS; Sun, 03 Jan 2016 12:09:43 +0000 From: Christoph Hellwig To: darrick.wong@oracle.com Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend Date: Sun, 3 Jan 2016 13:07:53 +0100 X-ASG-Orig-Subj: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend Message-Id: <1451822873-12969-4-git-send-email-hch@lst.de> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1451822873-12969-1-git-send-email-hch@lst.de> References: <1451822873-12969-1-git-send-email-hch@lst.de> X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451822984 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Otherwise we leak COW allocations done earlier in writepage. This can be reproduced fairly easily when we hit the non-blocking writeback EAGAIN case. Signed-off-by: Christoph Hellwig --- fs/xfs/xfs_aops.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 185415a..9c69dc3 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -588,6 +588,7 @@ xfs_cancel_ioend( { xfs_ioend_t *next; struct buffer_head *bh, *next_bh; + int error; do { next = ioend->io_list; @@ -605,6 +606,12 @@ xfs_cancel_ioend( unlock_buffer(bh); } while ((bh = next_bh) != NULL); + if (ioend->io_flags & XFS_IOEND_COW) { + error = xfs_reflink_end_cow_failed( + XFS_I(ioend->io_inode), + ioend->io_offset, ioend->io_size); + WARN_ON_ONCE(error); + } mempool_free(ioend, xfs_ioend_pool); } while ((ioend = next) != NULL); } -- 1.9.1 From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:13:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 6527D7F4E for ; Sun, 3 Jan 2016 06:13:56 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5354B8F8035 for ; Sun, 3 Jan 2016 04:13:56 -0800 (PST) X-ASG-Debug-ID: 1451823234-04cbb0101c1d44b0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id gRCgcEdkOPJ91XKQ (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:13:55 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhXl-00011s-So; Sun, 03 Jan 2016 12:13:53 +0000 Date: Sun, 3 Jan 2016 04:13:53 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 02/76] xfs: fix log ticket type printing Message-ID: <20160103121353.GA28429@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 02/76] xfs: fix log ticket type printing References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085636.12713.59155.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219085636.12713.59155.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823234 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Sat, Dec 19, 2015 at 12:56:36AM -0800, Darrick J. Wong wrote: > Update the log ticket reservation type printing code to reflect > all the types of log tickets, to avoid incorrect debug output and > avoid running off the end of the array. This looks okay, but I think we should use name array initializers to make sure it doesn't got out of sync ever. Also I don't think we'd ever run past the array as the users does a size check. From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:15:31 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 649737F4E for ; Sun, 3 Jan 2016 06:15:31 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 471B4304032 for ; Sun, 3 Jan 2016 04:15:28 -0800 (PST) X-ASG-Debug-ID: 1451823325-04bdf0265042fca0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id HP63iPaVZujyrl1p (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:15:25 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhZF-0002Jo-AV; Sun, 03 Jan 2016 12:15:25 +0000 Date: Sun, 3 Jan 2016 04:15:25 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct Message-ID: <20160103121525.GB28429@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085655.12713.60317.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219085655.12713.60317.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823325 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Sat, Dec 19, 2015 at 12:56:55AM -0800, Darrick J. Wong wrote: > Because struct xfs_agfl is 36 bytes long and has a 64-bit integer > inside it, gcc will quietly round the structure size up to the nearest > 64 bits -- in this case, 40 bytes. This results in the XFS_AGFL_SIZE > macro returning incorrect results for v5 filesystems on 64-bit > machines (118 items instead of 119). As a result, a 32-bit xfs_repair > will see garbage in AGFL item 119 and complain. > > Therefore, tell gcc not to pad the structure so that the AGFL size > calculation is correct. Do you have a testcase for this? From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:17:31 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CE1CA7F37 for ; Sun, 3 Jan 2016 06:17:31 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 544E2AC003 for ; Sun, 3 Jan 2016 04:17:31 -0800 (PST) X-ASG-Debug-ID: 1451823449-04cb6c303240aea0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id PMltMAXhjqORWQKd (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:17:29 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhbE-0002P3-Rp; Sun, 03 Jan 2016 12:17:28 +0000 Date: Sun, 3 Jan 2016 04:17:28 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled Message-ID: <20160103121728.GC28429@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090008.12713.25755.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219090008.12713.25755.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823449 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Sat, Dec 19, 2015 at 01:00:08AM -0800, Darrick J. Wong wrote: > Implement extent swapping when reverse-mapping is enabled. Can you just fold this into the previous patch? There seem to be a lot of patches in the series that would benefit from merging, but this is the one that really screams for it :) From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:18:59 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id DAFF37F37 for ; Sun, 3 Jan 2016 06:18:59 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7A883AC002 for ; Sun, 3 Jan 2016 04:18:59 -0800 (PST) X-ASG-Debug-ID: 1451823512-04cbb0101d1d45c0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id f5X7FpIVXZfUUBd6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:18:33 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhcG-0002Sf-30; Sun, 03 Jan 2016 12:18:32 +0000 Date: Sun, 3 Jan 2016 04:18:32 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 35/76] libxfs: refactor short btree block verification Message-ID: <20160103121832.GD28429@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 35/76] libxfs: refactor short btree block verification References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090014.12713.75657.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219090014.12713.75657.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823512 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS > +/** > + * xfs_btree_sblock_v5hdr_verify() -- verify the v5 fields of a short-format > + * btree block > + * > + * @bp: buffer containing the btree block > + * @max_recs: pointer to the m_*_mxr max records field in the xfs mount > + * @pag_max_level: pointer to the per-ag max level field > + */ We're so far avoided using kerneldoc comments and they silly super verbosity in XFS. It might be good to tone this down to the normal XFS level (same for some other patches). Otherwise looks good: Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:21:01 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 29BA07F58 for ; Sun, 3 Jan 2016 06:21:01 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id F19E5304053 for ; Sun, 3 Jan 2016 04:21:00 -0800 (PST) X-ASG-Debug-ID: 1451823658-04bdf0264f42fe30001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id gCDpCC5IoRGQQe3a (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:20:59 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhec-0003i2-9i; Sun, 03 Jan 2016 12:20:58 +0000 Date: Sun, 3 Jan 2016 04:20:58 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 57/76] xfs: allocate delayed extents in CoW fork Message-ID: <20160103122058.GE28429@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 57/76] xfs: allocate delayed extents in CoW fork References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090248.12713.32309.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219090248.12713.32309.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823659 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS I really don't like the separate XFS_IOEND_COW flag. This should be an ioend type just like XFS_IO_DELALLOC, XFS_IO_UNWRITTEN and XFS_IO_OVERWRITE. We might need some work in the direct I/O completions for that to really work, though. From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:22:13 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 14D9D7F5A for ; Sun, 3 Jan 2016 06:22:13 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id A7A7CAC003 for ; Sun, 3 Jan 2016 04:22:12 -0800 (PST) X-ASG-Debug-ID: 1451823730-04cbb0101b1d4710001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id gXQiAyWowKdBHDCb (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:22:11 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhfm-0003kQ-1l; Sun, 03 Jan 2016 12:22:10 +0000 Date: Sun, 3 Jan 2016 04:22:10 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 01/22] xfsdocs: fix asciidoc confusion w.r.t. include file newline at EOF Message-ID: <20160103122210.GA14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 01/22] xfsdocs: fix asciidoc confusion w.r.t. include file newline at EOF References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091215.15558.79820.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091215.15558.79820.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823730 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Sat, Dec 19, 2015 at 01:12:15AM -0800, Darrick J. Wong wrote: > Space each include one line apart so that asciidoc knows that each > section has ended at included file's EOF. This prevents it from > spewing horribly confused XML that blows up xmllint if each included > file doesn't itself end with a blank line. Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:22:46 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 6FDAA7F51 for ; Sun, 3 Jan 2016 06:22:46 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 0C45BAC001 for ; Sun, 3 Jan 2016 04:22:45 -0800 (PST) X-ASG-Debug-ID: 1451823762-04bdf02650430000001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id LJnRI2HtCmuWBrEv (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:22:42 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhgH-0003nj-OO; Sun, 03 Jan 2016 12:22:41 +0000 Date: Sun, 3 Jan 2016 04:22:41 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 02/22] xfsdocs: update book subtitles and headings Message-ID: <20160103122241.GB14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 02/22] xfsdocs: update book subtitles and headings References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091221.15558.28333.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091221.15558.28333.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823762 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:23:07 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D6F287F51 for ; Sun, 3 Jan 2016 06:23:07 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 6BD37AC001 for ; Sun, 3 Jan 2016 04:23:07 -0800 (PST) X-ASG-Debug-ID: 1451823785-04cb6c302f40afe0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id EQ4ZHmdL17Ybqihi (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:23:06 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhgf-0003od-1X; Sun, 03 Jan 2016 12:23:05 +0000 Date: Sun, 3 Jan 2016 04:23:05 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 03/22] xfsdocs: restore the old rendering of structure field definitions Message-ID: <20160103122305.GC14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 03/22] xfsdocs: restore the old rendering of structure field definitions References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091227.15558.73975.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091227.15558.73975.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823786 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:23:37 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id A26607F51 for ; Sun, 3 Jan 2016 06:23:37 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 324BAAC002 for ; Sun, 3 Jan 2016 04:23:37 -0800 (PST) X-ASG-Debug-ID: 1451823815-04cbb0101a1d4760001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id AySEF8f8E1qyq3Qo (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:23:35 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhh8-0003qI-Gd; Sun, 03 Jan 2016 12:23:34 +0000 Date: Sun, 3 Jan 2016 04:23:34 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 04/22] xfsdocs: convert html entities to regular characters Message-ID: <20160103122334.GD14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 04/22] xfsdocs: convert html entities to regular characters References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091234.15558.35406.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091234.15558.35406.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823815 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:23:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 26BC27F51 for ; Sun, 3 Jan 2016 06:23:56 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id AD074AC001 for ; Sun, 3 Jan 2016 04:23:55 -0800 (PST) X-ASG-Debug-ID: 1451823833-04cbb0101d1d4780001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id JOcxjRPcr8OytxVE (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:23:54 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhhR-0003sq-23; Sun, 03 Jan 2016 12:23:53 +0000 Date: Sun, 3 Jan 2016 04:23:53 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 05/22] xfsdocs: convert images to text Message-ID: <20160103122353.GE14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 05/22] xfsdocs: convert images to text References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091242.15558.50391.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091242.15558.50391.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823833 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:24:41 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 893177F51 for ; Sun, 3 Jan 2016 06:24:41 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 2CBA18F8035 for ; Sun, 3 Jan 2016 04:24:41 -0800 (PST) X-ASG-Debug-ID: 1451823879-04cb6c302f40b040001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id X765gceaKNfb2pr6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:24:40 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhiA-000469-RQ; Sun, 03 Jan 2016 12:24:38 +0000 Date: Sun, 3 Jan 2016 04:24:38 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 07/22] xfsdocs: update types Message-ID: <20160103122438.GF14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 07/22] xfsdocs: update types References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091259.15558.84471.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091259.15558.84471.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823880 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks fine, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:25:08 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 89B7A7F51 for ; Sun, 3 Jan 2016 06:25:08 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 390F1AC001 for ; Sun, 3 Jan 2016 04:25:08 -0800 (PST) X-ASG-Debug-ID: 1451823906-04bdf02651430330001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 9no1YcSSIQsQ1Fi2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:25:06 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhic-00059j-5b; Sun, 03 Jan 2016 12:25:06 +0000 Date: Sun, 3 Jan 2016 04:25:06 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 08/22] xfsdocs: make example section titles consistent Message-ID: <20160103122506.GG14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 08/22] xfsdocs: make example section titles consistent References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091305.15558.97231.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091305.15558.97231.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823906 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:25:42 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 821717F51 for ; Sun, 3 Jan 2016 06:25:42 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 631BF304032 for ; Sun, 3 Jan 2016 04:25:42 -0800 (PST) X-ASG-Debug-ID: 1451823940-04bdf02650430360001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id Vj003xTeCOm8PNUb (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:25:41 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhjA-0005SW-I1; Sun, 03 Jan 2016 12:25:40 +0000 Date: Sun, 3 Jan 2016 04:25:40 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 09/22] xfsdocs: fix various errors and missing bits in the text Message-ID: <20160103122540.GH14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 09/22] xfsdocs: fix various errors and missing bits in the text References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091311.15558.80832.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091311.15558.80832.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823941 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:26:12 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 2FD317F51 for ; Sun, 3 Jan 2016 06:26:11 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7B3E6AC001 for ; Sun, 3 Jan 2016 04:26:11 -0800 (PST) X-ASG-Debug-ID: 1451823970-04bdf0264e430370001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id hQ1F0KLA8Wq8PXBc (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:26:10 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhje-0005YZ-0A; Sun, 03 Jan 2016 12:26:10 +0000 Date: Sun, 3 Jan 2016 04:26:09 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 10/22] xfsdocs: add missing structure field definitions Message-ID: <20160103122609.GI14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 10/22] xfsdocs: add missing structure field definitions References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091318.15558.52742.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091318.15558.52742.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823970 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:26:35 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B64CB7F55 for ; Sun, 3 Jan 2016 06:26:35 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 57BA0AC004 for ; Sun, 3 Jan 2016 04:26:35 -0800 (PST) X-ASG-Debug-ID: 1451823993-04cbb0101a1d4810001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id Zj6SaqljADYYi5lh (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:26:34 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhk1-0005c1-34; Sun, 03 Jan 2016 12:26:33 +0000 Date: Sun, 3 Jan 2016 04:26:33 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 11/22] xfsdocs: add missing xfs_db examples Message-ID: <20160103122633.GJ14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 11/22] xfsdocs: add missing xfs_db examples References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091325.15558.37906.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091325.15558.37906.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451823994 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:26:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 234867F7B for ; Sun, 3 Jan 2016 06:26:56 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id B67D2AC004 for ; Sun, 3 Jan 2016 04:26:55 -0800 (PST) X-ASG-Debug-ID: 1451824014-04bdf0264e430380001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 8gHwUk0M7TkYAKp2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:26:55 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhkM-0005gg-FL; Sun, 03 Jan 2016 12:26:54 +0000 Date: Sun, 3 Jan 2016 04:26:54 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 12/22] xfsdocs: fix quoted text markings Message-ID: <20160103122654.GK14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 12/22] xfsdocs: fix quoted text markings References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091331.15558.64509.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091331.15558.64509.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451824014 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:28:02 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 46DD67F7C for ; Sun, 3 Jan 2016 06:28:02 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 15F228F8033 for ; Sun, 3 Jan 2016 04:28:02 -0800 (PST) X-ASG-Debug-ID: 1451824080-04bdf026504303d0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 9C9JmrWdF8ylvuPA (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:28:01 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhlQ-0005n8-FZ; Sun, 03 Jan 2016 12:28:00 +0000 Date: Sun, 3 Jan 2016 04:28:00 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 13/22] xfsdocs: introduce XFS at a high level Message-ID: <20160103122800.GL14361@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 13/22] xfsdocs: introduce XFS at a high level References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091338.15558.24376.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091338.15558.24376.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451824081 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25794 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks great! Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 06:31:39 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 4B55D7F6B for ; Sun, 3 Jan 2016 06:31:39 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 392788F8033 for ; Sun, 3 Jan 2016 04:31:39 -0800 (PST) X-ASG-Debug-ID: 1451824297-04cb6c303240b1d0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id XhVHNYsPRYzuAROV (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 04:31:38 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFhou-0007L1-Vl; Sun, 03 Jan 2016 12:31:36 +0000 Date: Sun, 3 Jan 2016 04:31:36 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 06/22] xfsdocs: delete images of hex dumps Message-ID: <20160103123136.GA27913@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 06/22] xfsdocs: delete images of hex dumps References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091249.15558.18497.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091249.15558.18497.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451824298 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25793 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:40:52 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id CFC6E7F81 for ; Sun, 3 Jan 2016 07:40:52 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id BED7B8F8033 for ; Sun, 3 Jan 2016 05:40:49 -0800 (PST) X-ASG-Debug-ID: 1451828443-04cbb0101a1d5be0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id fXoR2VmqVBKMSvpo (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:40:44 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFitm-0004ZX-K7; Sun, 03 Jan 2016 13:40:42 +0000 Date: Sun, 3 Jan 2016 05:40:42 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 14/22] xfsdocs: document the xfs log format Message-ID: <20160103134042.GA9144@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 14/22] xfsdocs: document the xfs log format References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091344.15558.83896.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091344.15558.83896.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451828444 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:41:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id D03DF7F7B for ; Sun, 3 Jan 2016 07:41:56 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id B1B4B304032 for ; Sun, 3 Jan 2016 05:41:53 -0800 (PST) X-ASG-Debug-ID: 1451828512-04bdf0264e431490001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 4USlGAfHahtiruWm (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:41:52 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFiut-0004yF-JA; Sun, 03 Jan 2016 13:41:51 +0000 Date: Sun, 3 Jan 2016 05:41:51 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes Message-ID: <20160103134151.GB9144@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091351.15558.51417.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091351.15558.51417.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451828512 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks generally good, although I wonder the take on the realtime device is a bit too positive given that we generally advise people not to use it. Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:46:25 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 9432C7F7B for ; Sun, 3 Jan 2016 07:46:25 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 25BA9AC002 for ; Sun, 3 Jan 2016 05:46:25 -0800 (PST) X-ASG-Debug-ID: 1451828782-04cbb0101d1d5d90001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 6iaGTqVT0KcjIbam (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:46:23 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFizG-0006PK-1B; Sun, 03 Jan 2016 13:46:22 +0000 Date: Sun, 3 Jan 2016 05:46:22 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 16/22] xfsdocs: add documentation of v5 fields Message-ID: <20160103134621.GA19752@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 16/22] xfsdocs: add documentation of v5 fields References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091358.15558.23715.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091358.15558.23715.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451828783 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:47:01 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 734667F7B for ; Sun, 3 Jan 2016 07:47:01 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 43F02304032 for ; Sun, 3 Jan 2016 05:47:01 -0800 (PST) X-ASG-Debug-ID: 1451828819-04bdf0264e4315e0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id NKW2EkepUgiVtElU (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:46:59 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFizr-0006QI-BO; Sun, 03 Jan 2016 13:46:59 +0000 Date: Sun, 3 Jan 2016 05:46:59 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 22/22] xfsdocs: move directory chapter before xattr chapter Message-ID: <20160103134659.GB19752@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 22/22] xfsdocs: move directory chapter before xattr chapter References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091437.15558.91782.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091437.15558.91782.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451828819 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:54:03 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E4A3E7F7B for ; Sun, 3 Jan 2016 07:54:03 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 73C6FAC003 for ; Sun, 3 Jan 2016 05:54:03 -0800 (PST) X-ASG-Debug-ID: 1451829241-04cb6c303140cd80001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id 4eIXdxzrT3UFz4GG (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:54:02 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFj6e-0008Bx-W7; Sun, 03 Jan 2016 13:54:01 +0000 Date: Sun, 3 Jan 2016 05:54:00 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 17/22] xfsdocs: add a chapter discussing v5 disk format Message-ID: <20160103135400.GA30378@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 17/22] xfsdocs: add a chapter discussing v5 disk format References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091405.15558.33731.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091405.15558.33731.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451829241 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:54:34 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 0D3187F7B for ; Sun, 3 Jan 2016 07:54:34 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id F07CE304039 for ; Sun, 3 Jan 2016 05:54:33 -0800 (PST) X-ASG-Debug-ID: 1451829269-04cb6c303040cd90001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id B9CJAikAAJ8lRjEO (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:54:30 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFj76-0008P2-SS; Sun, 03 Jan 2016 13:54:28 +0000 Date: Sun, 3 Jan 2016 05:54:28 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: hch@infradead.org, david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 18/22] xfsdocs: document magic numbers and theoretical limits Message-ID: <20160103135428.GB30378@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 18/22] xfsdocs: document magic numbers and theoretical limits References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091411.15558.9135.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091411.15558.9135.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451829269 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, : Christoph Hellwig From BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org Sun Jan 3 07:55:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 1D4657F7B for ; Sun, 3 Jan 2016 07:55:18 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id EE3658F8035 for ; Sun, 3 Jan 2016 05:55:17 -0800 (PST) X-ASG-Debug-ID: 1451829316-04bdf026504317b0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id qcr3chfB929PFHu8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 05:55:16 -0800 (PST) X-Barracuda-Envelope-From: BATV+79bc6ce57c55c66c7247+4515+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aFj7r-0001Mb-K0; Sun, 03 Jan 2016 13:55:15 +0000 Date: Sun, 3 Jan 2016 05:55:15 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: david@fromorbit.com, xfs@oss.sgi.com Subject: Re: [PATCH 19/22] xfsdocs: document the sparse inodes feature Message-ID: <20160103135515.GC30378@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 19/22] xfsdocs: document the sparse inodes feature References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091418.15558.27198.stgit@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151219091418.15558.27198.stgit@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451829316 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25795 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS Looks good, Reviewed-by: Christoph Hellwig From dan.j.williams@intel.com Sun Jan 3 12:13:15 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 6702D7F37 for ; Sun, 3 Jan 2016 12:13:15 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 588428F8033 for ; Sun, 3 Jan 2016 10:13:12 -0800 (PST) X-ASG-Debug-ID: 1451844786-04cbb0101c1da030001-NocioJ Received: from mail-yk0-f179.google.com (mail-yk0-f179.google.com [209.85.160.179]) by cuda.sgi.com with ESMTP id UCdaEiGMFC4qW2kO (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 10:13:07 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 209.85.160.179 Received: by mail-yk0-f179.google.com with SMTP id v14so136130211ykd.3 for ; Sun, 03 Jan 2016 10:13:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=THPfaZpE7+X0uM6y+NRIvVhHaCeXdS1lxSOT2ZjsP6I=; b=wSBA0euLFETAt/lRVPUj8s8y3wIIQ1/VTZRsK/MzWmLrg1Qwfi5KS0x2hyiqpe5vCk hfJJOLnO8oc6cOwuBJVZYlhFGVb8O5aqbnv3mVZJUJRnbO2El2Wn4P6yubS66KqgXM04 7Klkn9QM59Rs47DWxt1Wr7AAQizQK6wLxZnbCCvxRqeABSMQTL8guCk6Bn7XH4k7VAU+ qBwJtz/1VMVMAcYUpY99CGe+T2w9c8oTSHQZ1A579yVWW4PnqD/IrBN0SN/nCiSqWfcJ YFxiWx6NNs8dtbgggAV+jQ9GfwAxLdV4V9sRVxkQFBjxvlwbs1rrOhWnkkI0ft0DSB6k dHhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=THPfaZpE7+X0uM6y+NRIvVhHaCeXdS1lxSOT2ZjsP6I=; b=B47BLl08VZrIx1hnCzbpgVRtZ46EcMZhKFmbnupb/9vaubJ0Kk1WlCZSActtctGwCa ZequP2Sjwt07OjldBQFgIyvsQ9U4iDTBi6VKC8apq2LEjwX4lR/l5MX9kCiV1nhn9+Dj qquMBWmLk17uwRr8TOVT+secCs9HDbbQIXRozXNrUv0cliRKbRGB7cgEXt4k+lB9HsHW EwUBMwF55jK232vhCXldWmGowdTf4JmXjQ4he1q6eO0cqBkeYMAKXzH9OPT3yp/13bKm BzFybn09vs1BdhGQmU333YLchYtnVjtHRhTpisPhYpWb4aFYHCsuybNAIThPBlq1STwY pu2Q== X-Gm-Message-State: ALoCoQnjDgECfC1c5DgXMn6C8p81ckxI62SIo/ESso9dupRyD1D1OE/ZvO0d2LQ/WxY9QK/E3PukRnxzJmbwOmhrF8U6+F+KMnVeI8AOvUhFavgDhnsFbkY= MIME-Version: 1.0 X-Received: by 10.129.85.83 with SMTP id j80mr60067671ywb.28.1451844786282; Sun, 03 Jan 2016 10:13:06 -0800 (PST) Received: by 10.37.202.20 with HTTP; Sun, 3 Jan 2016 10:13:06 -0800 (PST) In-Reply-To: <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> Date: Sun, 3 Jan 2016 10:13:06 -0800 Message-ID: Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync From: Dan Williams X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync To: Ross Zwisler Cc: "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , "Theodore Ts'o" , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen Content-Type: text/plain; charset=UTF-8 X-Barracuda-Connect: mail-yk0-f179.google.com[209.85.160.179] X-Barracuda-Start-Time: 1451844786 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25799 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature On Wed, Dec 23, 2015 at 11:39 AM, Ross Zwisler wrote: > To properly handle fsync/msync in an efficient way DAX needs to track dirty > pages so it is able to flush them durably to media on demand. > > The tracking of dirty pages is done via the radix tree in struct > address_space. This radix tree is already used by the page writeback > infrastructure for tracking dirty pages associated with an open file, and > it already has support for exceptional (non struct page*) entries. We > build upon these features to add exceptional entries to the radix tree for > DAX dirty PMD or PTE pages at fault time. > > Signed-off-by: Ross Zwisler I'm hitting the following report with the ndctl dax test [1] on next-20151231. I bisected it to commit 3cb108f941de "dax-add-support-for-fsync-sync-v6". I'll take a closer look tomorrow, but in case someone can beat me to it, here's the back-trace: ------------[ cut here ]------------ kernel BUG at fs/inode.c:497! [..] CPU: 1 PID: 3001 Comm: umount Tainted: G O 4.4.0-rc7+ #2412 Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 task: ffff8800da2a5a00 ti: ffff880307794000 task.ti: ffff880307794000 RIP: 0010:[] [] clear_inode+0x71/0x80 RSP: 0018:ffff880307797d50 EFLAGS: 00010002 RAX: ffff8800da2a5a00 RBX: ffff8800ca2e7328 RCX: ffff8800da2a5a28 RDX: 0000000000000001 RSI: 0000000000000005 RDI: ffff8800ca2e7530 RBP: ffff880307797d60 R08: ffffffff82900ae0 R09: 0000000000000000 R10: ffff8800ca2e7548 R11: 0000000000000000 R12: ffff8800ca2e7530 R13: ffff8800ca2e7328 R14: ffff8800da2e88d0 R15: ffff8800da2e88d0 FS: 00007f2b22f4a880(0000) GS:ffff88031fc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00005648abd933e8 CR3: 000000007f3fc000 CR4: 00000000000006e0 Stack: ffff8800ca2e7328 ffff8800ca2e7000 ffff880307797d88 ffffffffa01c18af ffff8800ca2e7328 ffff8800ca2e74d0 ffffffffa01ec740 ffff880307797db0 ffffffff81281038 ffff8800ca2e74c0 ffff880307797e00 ffff8800ca2e7328 Call Trace: [] xfs_fs_evict_inode+0x5f/0x110 [xfs] [] evict+0xb8/0x180 [] dispose_list+0x3b/0x50 [] evict_inodes+0x144/0x170 [] generic_shutdown_super+0x3f/0xf0 [] kill_block_super+0x27/0x70 [] deactivate_locked_super+0x43/0x70 [] deactivate_super+0x5c/0x60 [] cleanup_mnt+0x3f/0x90 [] __cleanup_mnt+0x12/0x20 [] task_work_run+0x76/0x90 [] syscall_return_slowpath+0x20a/0x280 [] int_ret_from_sys_call+0x25/0x9f Code: 48 8d 93 30 03 00 00 48 39 c2 75 23 48 8b 83 d0 00 00 00 a8 20 74 1a a8 40 75 18 48 c7 8 3 d0 00 00 00 60 00 00 00 5b 41 5c 5d c3 <0f> 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 1f 44 00 00 0f 1f 44 00 00 55 RIP [] clear_inode+0x71/0x80 RSP ---[ end trace 3b1d8898a94a4fc1 ]--- [1]: git://git@github.com:pmem/ndctl.git pending make TESTS="test/dax.sh" check From david@fromorbit.com Sun Jan 3 14:48:08 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 687967F37 for ; Sun, 3 Jan 2016 14:48:08 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 01C60AC002 for ; Sun, 3 Jan 2016 12:48:04 -0800 (PST) X-ASG-Debug-ID: 1451854079-04bdf0264e4372e0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id Gl29kEOnwgNIhorr for ; Sun, 03 Jan 2016 12:48:00 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CCCwDjh4lWPGu7LXleKAECgw9SbYJihXegOBIGigaBT4k+HoVrBAICgRJNAQEBAQEBBwEBAQFBP4Q1AQEEOhwjEAgDGAklDwUlAwcaE4guv1wBCyEZhXWFR4JcgVsBAYUDAQSXBoVAiAiBZUqMVUSFFIEKD4dJhQIqNINOgUIBAQE Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 07:17:59 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFpZG-00008J-Ar; Mon, 04 Jan 2016 07:47:58 +1100 Date: Mon, 4 Jan 2016 07:47:58 +1100 From: Dave Chinner To: Andrea Gelmini Cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 Message-ID: <20160103204758.GW19802@dastard> X-ASG-Orig-Subj: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 References: <20151214180048.GA15690@glen> <20151214195422.GM26718@dastard> <20151214201526.GA25152@glen> <20151214212220.GO26718@dastard> <20151215091145.GA19282@glen> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151215091145.GA19282@glen> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451854079 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-BRTS-Evidence: gelma.net X-Barracuda-Spam-Score: 2.60 X-Barracuda-Spam-Status: No, SCORE=2.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0249, MARKETING_SUBJECT X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25802 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 2.00 BSF_SC0_MV0249 Custom rule MV0249 On Tue, Dec 15, 2015 at 10:11:45AM +0100, Andrea Gelmini wrote: > On Tue, Dec 15, 2015 at 08:22:20AM +1100, Dave Chinner wrote: > > $ gdb vmlinux > > .... > > (gdb) l *(xfs_iflush_cluster+0x9d7) > > It's not working. Shame on me, I forgot to set "Compile the kernel with debug info". > > I'm recompiling, to try it again. > Maybe, in the meanwhile, you can do something with my files. You can find 'em here: > http://mail.gelma.net/xfs_kasan Any update on this problem, Andrea? Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Sun Jan 3 15:06:36 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 92CB97F37 for ; Sun, 3 Jan 2016 15:06:36 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 18190AC002 for ; Sun, 3 Jan 2016 13:06:33 -0800 (PST) X-ASG-Debug-ID: 1451855185-04cbb0101a1dc490001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id pFYdkyxhgBUJmYPq for ; Sun, 03 Jan 2016 13:06:25 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2C9EACBjIlWPGu7LXleKAECgw+BP4Jig36BeaBKBoFliXCJPoYJAgIBAQKBEk0BAQEBAQEHAQEBAUE/hDUBAQQnExwYCQIQCAMYCSUPBSUDBxoTiC6/VwEBAQcCASAZhXWFR4QkhRgBBJMKg3yNSIFlhEaIWY46gnUcgXEqNAGDRIFLAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 07:35:45 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFpqS-0000A0-Q7; Mon, 04 Jan 2016 08:05:44 +1100 Date: Mon, 4 Jan 2016 08:05:44 +1100 From: Dave Chinner To: Dan Williams Cc: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, xfs@oss.sgi.com Subject: Re: [PATCH 2/3] xfs: handle shutdown notifications Message-ID: <20160103210544.GX19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 2/3] xfs: handle shutdown notifications References: <20151201235836.36836.90416.stgit@dwillia2-desk3.jf.intel.com> <20151201235846.36836.1476.stgit@dwillia2-desk3.jf.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151201235846.36836.1476.stgit@dwillia2-desk3.jf.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451855185 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25802 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Tue, Dec 01, 2015 at 03:58:46PM -0800, Dan Williams wrote: > Force a filesystem shutdown when the backing device is known to be dead. > I.e. blk_queue_enter() permanently returns -ENODEV. > > Cc: xfs@oss.sgi.com > Suggested-by: Dave Chinner > Signed-off-by: Dan Williams > --- > fs/block_dev.c | 3 ++- > fs/xfs/xfs_super.c | 9 +++++++++ > include/linux/fs.h | 2 ++ > 3 files changed, 13 insertions(+), 1 deletion(-) > > diff --git a/fs/block_dev.c b/fs/block_dev.c > index 39989e990df9..dfe9a53a7c53 100644 > --- a/fs/block_dev.c > +++ b/fs/block_dev.c > @@ -1802,7 +1802,7 @@ int __invalidate_device(struct block_device *bdev, bool kill_dirty) > } > EXPORT_SYMBOL(__invalidate_device); > > -static void generic_bdi_gone(struct super_block *sb) > +void generic_bdi_gone(struct super_block *sb) > { > struct inode *inode, *_inode = NULL; > > @@ -1828,6 +1828,7 @@ static void generic_bdi_gone(struct super_block *sb) > spin_unlock(&sb->s_inode_list_lock); > iput(_inode); > } > +EXPORT_SYMBOL(generic_bdi_gone); > > void shutdown_partition(struct gendisk *disk, int partno) > { > diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c > index 36bd8825bfb0..63c36508e9db 100644 > --- a/fs/xfs/xfs_super.c > +++ b/fs/xfs/xfs_super.c > @@ -1618,6 +1618,14 @@ xfs_fs_free_cached_objects( > return xfs_reclaim_inodes_nr(XFS_M(sb), sc->nr_to_scan); > } > > +static void > +xfs_fs_bdi_gone( > + struct super_block *sb) > +{ > + xfs_force_shutdown(XFS_M(sb), SHUTDOWN_DEVICE_REQ); > + generic_bdi_gone(sb); > +} You haven't cc'd me or the XFS list on the other patches in this series, so I cannot comment as to whether this will work or not. As I've asked before, please do not do single patch CC's like this as it does not give the reviewer the context to be able to review the change, nor is it possible to *apply and test* the patch they are being asked to comment on. Please resend the series with complete CC's to the relevant lists. Sure, go ahead an CC individual people on certain patches, but you need to make sure the whole patch series hit each mailing list.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Sun Jan 3 15:29:37 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 6EA377F37 for ; Sun, 3 Jan 2016 15:29:37 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 0A6C9AC002 for ; Sun, 3 Jan 2016 13:29:36 -0800 (PST) X-ASG-Debug-ID: 1451856574-04bdf02651437e00001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id ppdbFuMyZlC81sgz for ; Sun, 03 Jan 2016 13:29:34 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AWEAB3kolWPGu7LXleKAECgw+BP4Jig36BeaBQi1WJPoYJBAICgRJNAQEBAQEBBwEBAQFBP4Q1AQEEOhwjEAgDDgoJJQ8FJQMHGhOILr9OAQEIAiEZhXWFR4E8AYMHhHgFjTmJTY1IgWWNH4VYiGKDAoIAKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 07:59:06 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFqD3-0000Bg-Hp; Mon, 04 Jan 2016 08:29:05 +1100 Date: Mon, 4 Jan 2016 08:29:05 +1100 From: Dave Chinner To: Christoph Hellwig Cc: "Darrick J. Wong" , xfs@oss.sgi.com Subject: Re: [PATCH 02/76] xfs: fix log ticket type printing Message-ID: <20160103212905.GY19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 02/76] xfs: fix log ticket type printing References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085636.12713.59155.stgit@birch.djwong.org> <20160103121353.GA28429@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103121353.GA28429@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451856574 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25803 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Sun, Jan 03, 2016 at 04:13:53AM -0800, Christoph Hellwig wrote: > On Sat, Dec 19, 2015 at 12:56:36AM -0800, Darrick J. Wong wrote: > > Update the log ticket reservation type printing code to reflect > > all the types of log tickets, to avoid incorrect debug output and > > avoid running off the end of the array. > > This looks okay, but I think we should use name array initializers > to make sure it doesn't got out of sync ever. Good idea, separate patch is fine. Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Sun Jan 3 15:31:08 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 49AAB7F37 for ; Sun, 3 Jan 2016 15:31:08 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 3A417304032 for ; Sun, 3 Jan 2016 13:31:05 -0800 (PST) X-ASG-Debug-ID: 1451856662-04cb6c30314139c0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id Q4QSwc3BbGlokUrC for ; Sun, 03 Jan 2016 13:31:02 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AZEAB3kolWPGu7LXleKAECgw+BP4Jig36BeaBKBotViT6GCQQCAoESTQEBAQEBAQcBAQEBQT+ENQEBBDocIxAIAw4KCSUPBSUDBxoTiC6/TgEBCAIhGYV1hUeJPAWXBo1IjwSOOoUCKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 08:00:38 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFqEY-0000Br-CB; Mon, 04 Jan 2016 08:30:38 +1100 Date: Mon, 4 Jan 2016 08:30:38 +1100 From: Dave Chinner To: Christoph Hellwig Cc: "Darrick J. Wong" , xfs@oss.sgi.com Subject: Re: [PATCH 35/76] libxfs: refactor short btree block verification Message-ID: <20160103213038.GZ19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 35/76] libxfs: refactor short btree block verification References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090014.12713.75657.stgit@birch.djwong.org> <20160103121832.GD28429@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103121832.GD28429@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451856662 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25802 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Sun, Jan 03, 2016 at 04:18:32AM -0800, Christoph Hellwig wrote: > > +/** > > + * xfs_btree_sblock_v5hdr_verify() -- verify the v5 fields of a short-format > > + * btree block > > + * > > + * @bp: buffer containing the btree block > > + * @max_recs: pointer to the m_*_mxr max records field in the xfs mount > > + * @pag_max_level: pointer to the per-ag max level field > > + */ > > We're so far avoided using kerneldoc comments and they silly super > verbosity in XFS. It might be good to tone this down to the normal > XFS level (same for some other patches). I'll do that before I commit it. Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Sun Jan 3 21:58:44 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 9D7F529DF5 for ; Sun, 3 Jan 2016 21:58:44 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 7228D304032 for ; Sun, 3 Jan 2016 19:58:44 -0800 (PST) X-ASG-Debug-ID: 1451879921-04bdf0265143df00001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id MIKYD4waopokCJJQ for ; Sun, 03 Jan 2016 19:58:41 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CtCwAc7YlWPGu7LXlWCCgBAoMPUm2CYoV3oEYBAQEBAQEGi1WJPhgKhWcEAgKBF00BAQEBAQEHAQEBAUE/hDQBAQEDAQEBASQTHBgLBQsIAxgJJQ8FExIDBxoTiCcHD79ZAQEBAQEBBAEBAQEfGYV1hUeCbYFFg2+BGwWXBoVAiAiPBI46gnUcgXEqNIUQAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 14:28:41 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFwI4-0000hQ-AP; Mon, 04 Jan 2016 14:58:40 +1100 Date: Mon, 4 Jan 2016 14:58:40 +1100 From: Dave Chinner To: Eric Sandeen Cc: xfs@oss.sgi.com Subject: Re: [PATCH V2] xfs: create helper for bmap finish & trans join in attr code Message-ID: <20160104035840.GA19802@dastard> X-ASG-Orig-Subj: Re: [PATCH V2] xfs: create helper for bmap finish & trans join in attr code References: <56441B8E.6070603@redhat.com> <5644BEF8.6070201@sandeen.net> <20151112165801.GA14854@infradead.org> <20151112201231.GS19199@dastard> <20151113090840.GA16423@infradead.org> <56465643.5070504@sandeen.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <56465643.5070504@sandeen.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451879921 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25809 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Fri, Nov 13, 2015 at 03:29:39PM -0600, Eric Sandeen wrote: > On 11/13/15 3:08 AM, Christoph Hellwig wrote: > > On Fri, Nov 13, 2015 at 07:12:31AM +1100, Dave Chinner wrote: > >> On Thu, Nov 12, 2015 at 08:58:01AM -0800, Christoph Hellwig wrote: > >>> I think the problem here is simply that our interfaces suck. > >>> xfs_trans_roll really needs to rejoin any inode to the new transaction > >>> to that was joined to the previous one. Once we've fixed that we can > >>> get rid of the silly committed arguments and everyone will be happy. > >> > >> xfs_trans_roll is not specifically for rolling transactions with > >> locked inodes in them. We could use it for any object that needs > >> multiple transactions to modify. e.g. we could roll transactions > >> across an AGF (using hold+join) so that it remains locked across > >> multiple allocation/free transactions. > > > > xfs_trans_roll already logs the inode core, which requires the > > inode to be attached to the transaction. While I could see the > > point of moving this out of the core __xfs_trans_roll into an > > xfs_trans_roll_inode helper we might as well follow the current > > interface for now. > > Trying to follow you guys ;) > > Something like this? yes, something like that ;) (better late than never!) -Dave. > diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c > index dbae649..d23bce8 100644 > --- a/fs/xfs/xfs_bmap_util.c > +++ b/fs/xfs/xfs_bmap_util.c > @@ -91,32 +91,32 @@ xfs_zero_extent( > * last due to locking considerations. We never free any extents in > * the first transaction. > * > - * Return 1 if the given transaction was committed and a new one > - * started, and 0 otherwise in the committed parameter. > + * If an inode *ip is provided, rejoin it to the transaction if > + * the transaction was committed. > */ > int /* error */ > xfs_bmap_finish( > struct xfs_trans **tp, /* transaction pointer addr */ > struct xfs_bmap_free *flist, /* i/o: list extents to free */ > - int *committed)/* xact committed or not */ > + xfs_inode_t *ip) > { > struct xfs_efd_log_item *efd; /* extent free data */ > struct xfs_efi_log_item *efi; /* extent free intention */ > int error; /* error return value */ > + int committed;/* xact committed or not */ > struct xfs_bmap_free_item *free; /* free extent item */ > struct xfs_bmap_free_item *next; /* next item on free list */ > > ASSERT((*tp)->t_flags & XFS_TRANS_PERM_LOG_RES); > - if (flist->xbf_count == 0) { > - *committed = 0; > + if (flist->xbf_count == 0) > return 0; > - } > + > efi = xfs_trans_get_efi(*tp, flist->xbf_count); > for (free = flist->xbf_first; free; free = free->xbfi_next) > xfs_trans_log_efi_extent(*tp, efi, free->xbfi_startblock, > free->xbfi_blockcount); > > - error = __xfs_trans_roll(tp, NULL, committed); > + error = __xfs_trans_roll(tp, ip, &committed); > if (error) { > /* > * If the transaction was committed, drop the EFD reference > @@ -128,16 +128,13 @@ xfs_bmap_finish( > * transaction so we should return committed=1 even though we're > * returning an error. > */ > - if (*committed) { > + if (committed) { > xfs_efi_release(efi); > xfs_force_shutdown((*tp)->t_mountp, > (error == -EFSCORRUPTED) ? > SHUTDOWN_CORRUPT_INCORE : > SHUTDOWN_META_IO_ERROR); > - } else { > - *committed = 1; > } > - > return error; > } > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs > -- Dave Chinner david@fromorbit.com From debbugs@buxtehude.debian.org Sun Jan 3 22:57:28 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id E4D0429DF5 for ; Sun, 3 Jan 2016 22:57:28 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id C38AA304032 for ; Sun, 3 Jan 2016 20:57:25 -0800 (PST) X-ASG-Debug-ID: 1451883437-04cb6c302f41a750001-NocioJ Received: from buxtehude.debian.org (buxtehude.debian.org [206.12.19.147]) by cuda.sgi.com with ESMTP id bUKQLa6FEMKTmsaV (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 03 Jan 2016 20:57:18 -0800 (PST) X-Barracuda-Envelope-From: debbugs@buxtehude.debian.org X-Barracuda-Apparent-Source-IP: 206.12.19.147 Received: from debbugs by buxtehude.debian.org with local (Exim 4.84) (envelope-from ) id 1aFxCk-0002mj-2M; Mon, 04 Jan 2016 04:57:14 +0000 X-Loop: owner@bugs.debian.org Subject: Bug#766811: spurious library links Reply-To: Marco d'Itri , 766811@bugs.debian.org X-ASG-Orig-Subj: Bug#766811: spurious library links Resent-From: Marco d'Itri Resent-To: debian-bugs-dist@lists.debian.org Resent-CC: XFS Development Team X-Loop: owner@bugs.debian.org Resent-Date: Mon, 04 Jan 2016 04:57:11 +0000 Resent-Message-ID: X-Debian-PR-Message: followup 766811 X-Debian-PR-Package: xfslibs-dev X-Debian-PR-Keywords: X-Debian-PR-Source: xfsprogs Received: via spool by 766811-submit@bugs.debian.org id=B766811.14518831228992 (code B ref 766811); Mon, 04 Jan 2016 04:57:11 +0000 Received: (at 766811) by bugs.debian.org; 4 Jan 2016 04:52:02 +0000 Received: from attila.bofh.it ([85.94.204.146] ident=postfix) by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84) (envelope-from ) id 1aFx7h-0002K8-Rz; Mon, 04 Jan 2016 04:52:02 +0000 Received: by attila.bofh.it (Postfix, from userid 10) id E8D542E4DA; Mon, 4 Jan 2016 05:51:59 +0100 (CET) Received: by bongo.bofh.it (Postfix, from userid 1000) id A0B1E2C0039; Mon, 4 Jan 2016 05:41:36 +0100 (CET) Date: Mon, 4 Jan 2016 05:41:36 +0100 From: Marco d'Itri To: 766811@bugs.debian.org Cc: 766811-submitter@bugs.debian.org Message-ID: <20160104044136.GD11738@bongo.bofh.it> References: <20141026004825.GA27863@bongo.bofh.it> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="a2FkP9tdjPU2nyhF" Content-Disposition: inline In-Reply-To: <20141026004825.GA27863@bongo.bofh.it> User-Agent: Mutt/1.5.24 (2015-08-30) X-CrossAssassin-Score: 2 X-Barracuda-Connect: buxtehude.debian.org[206.12.19.147] X-Barracuda-Start-Time: 1451883438 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25810 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header --a2FkP9tdjPU2nyhF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Oct 26, Marco d'Itri wrote: > These links do not appear to have any purpose and should be removed: >=20 > /lib/libhandle.a -> /usr/lib/libhandle.a > /lib/libhandle.la -> /usr/lib/libhandle.la > /usr/lib/libhandle.so -> /lib/libhandle.so >=20 > Also, policy forbids to thip the .la files at all for normal libraries. This is an obvious bug and I have not heard back from you in over one year, what are your plans about fixing this? Do you need help? --=20 ciao, Marco --a2FkP9tdjPU2nyhF Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQGcBAABCAAGBQJWifgAAAoJECv1pcEixPek0JAL/ivJ8g5KklMSzitaMhhBqkw2 I2BJ5hb3PIyBxqRWqzwrdHyGUJX8rkyJz+bFxYcUi9a0+/CYx7KYkqjkHGPiXiA7 WWOQdCp8IPMdmR4QpY3KkamoGODvXJplSJvYeZd7wKWNfvznU+XsoREaXMm48vu4 Xvyi76vYXxN9Gt0fRJ12K1Tsw5wSV2bxZollaHSyShIDEW8aR955O77gYQmnc6xW RTeM79ZkRBEEPhOgFw3wu1pjORUVAIPnNTIwXnMGxOgLLnxuoN6wfkcVMuYhCBK+ 8vUEgo+YezaMFxVEU/5+ew0C55RobZoSNTVTcNwrTSuSc974Y+W+Nxvb8/U6buDi JEVLea/aIzugUT9NmQ/UwtcdDClX2QY5G5XaFP8oniLI4NCQBPM3WRoi4kpjtkbc FVtdn/kjOIKVwIe5xYzwhczfp6YP1mm6Cr8a8ZJdK+o/LhUi41H/O4PyR6e+spoI 2TV8F5j4guNpVCdTJzpzij1K7kzC3HcZQVDreCGwcA== =p1lD -----END PGP SIGNATURE----- --a2FkP9tdjPU2nyhF-- From dave@fromorbit.com Sun Jan 3 23:55:03 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id E98057F37 for ; Sun, 3 Jan 2016 23:55:02 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id B843E304032 for ; Sun, 3 Jan 2016 21:55:02 -0800 (PST) X-ASG-Debug-ID: 1451886898-04cb6c303241b3e0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id GTeZtjwCCR1bXnBZ for ; Sun, 03 Jan 2016 21:54:58 -0800 (PST) X-Barracuda-Envelope-From: dave@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2C/DQAYCIpWPGu7LXleKAECgw9SbYZggXmgLwIKEgaBDJQhhw9NAQEBAQEBBwEBAQFBP4URIxiBAgMHiFvAEoYOigJohBkFlwaFQIltjR9EhRSIYoIuAQEIAgE6HIFxKjSDRoFKAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:24:57 +1030 Received: from disappointment.disaster.area ([192.168.1.110] helo=disappointment) by dastard with esmtp (Exim 4.80) (envelope-from ) id 1aFy6Z-0000tY-SW; Mon, 04 Jan 2016 16:54:55 +1100 Received: from dave by disappointment with local (Exim 4.86) (envelope-from ) id 1aFy6Z-000473-RU; Mon, 04 Jan 2016 16:54:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, ext4@vger.kernel.org Subject: [PATCH 0/3] fs: FS_IOC_FS[GS]ETXATTR promotion Date: Mon, 4 Jan 2016 16:54:49 +1100 X-ASG-Orig-Subj: [PATCH 0/3] fs: FS_IOC_FS[GS]ETXATTR promotion Message-Id: <1451886892-15548-1-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 2.5.0 X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451886898 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Hi folks, As I mentioned on -fsdevel this morning, the XFS_IOC_FS[SG]SETXATTR API promotion patch for ext4 project quota really needs to be split up into two patches: one for the API promotion and the other for the ext4 implementation of the API. This short series has the isolated API promotion patch, as well as a followup cleanup patch in XFS to get rid of the XFS definitions from the kernel code completely, followed by the first conflicting patch I had against the ext4-all-in-one patch for a per-inode DAX enablement flag. I've pushed these patches out to the following branch: git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git xfs-setxattr-promotion And I'll update them and merge them into the for-next branch according to feedback and review. Cheers, Dave. From dave@fromorbit.com Sun Jan 3 23:55:04 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 29D0E7F37 for ; Sun, 3 Jan 2016 23:55:04 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 19C88304059 for ; Sun, 3 Jan 2016 21:55:03 -0800 (PST) X-ASG-Debug-ID: 1451886901-04cb6c303141b3e0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id vLKH841x3GCNaIFr for ; Sun, 03 Jan 2016 21:55:01 -0800 (PST) X-Barracuda-Envelope-From: dave@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2B/DQAYCIpWPGu7LXleKAECgw+BP4ZggXmgTQacPE0BAQEBAQEHAQEBAUE/hDUBBVYjEAgYMTkDBxQZiC7AEoYOjwMFh16PKI8th26FMUSFFIhigi4BglMqNIUQAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:24:57 +1030 Received: from disappointment.disaster.area ([192.168.1.110] helo=disappointment) by dastard with esmtp (Exim 4.80) (envelope-from ) id 1aFy6Z-0000tb-T2; Mon, 04 Jan 2016 16:54:55 +1100 Received: from dave by disappointment with local (Exim 4.86) (envelope-from ) id 1aFy6Z-000476-SI; Mon, 04 Jan 2016 16:54:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, ext4@vger.kernel.org Subject: [PATCH 1/3] fs: XFS_IOC_FS[SG]SETXATTR to FS_IOC_FS[SG]ETXATTR promotion Date: Mon, 4 Jan 2016 16:54:50 +1100 X-ASG-Orig-Subj: [PATCH 1/3] fs: XFS_IOC_FS[SG]SETXATTR to FS_IOC_FS[SG]ETXATTR promotion Message-Id: <1451886892-15548-2-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451886892-15548-1-git-send-email-david@fromorbit.com> References: <1451886892-15548-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451886901 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- From: Dave Chinner Hoist the ioctl definitions for the XFS_IOC_FS[SG]SETXATTR API from fs/xfs/libxfs/xfs_fs.h to include/uapi/linux/fs.h so that the ioctls can be used by all filesystems, not just XFS. This enables (initially) ext4 to use the ioctl to set project IDs on inodes. Based-on-patch-from: Li Xi Signed-off-by: Dave Chinner --- fs/xfs/libxfs/xfs_fs.h | 51 +++++++++++++++++-------------------------------- include/uapi/linux/fs.h | 32 +++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+), 33 deletions(-) diff --git a/fs/xfs/libxfs/xfs_fs.h b/fs/xfs/libxfs/xfs_fs.h index b2b73a9..dd29d0a 100644 --- a/fs/xfs/libxfs/xfs_fs.h +++ b/fs/xfs/libxfs/xfs_fs.h @@ -36,38 +36,23 @@ struct dioattr { #endif /* - * Structure for XFS_IOC_FSGETXATTR[A] and XFS_IOC_FSSETXATTR. + * Flags for the bs_xflags/fsx_xflags field in FS_IOC_FS[GS]ETXATTR[A] */ -#ifndef HAVE_FSXATTR -struct fsxattr { - __u32 fsx_xflags; /* xflags field value (get/set) */ - __u32 fsx_extsize; /* extsize field value (get/set)*/ - __u32 fsx_nextents; /* nextents field value (get) */ - __u32 fsx_projid; /* project identifier (get/set) */ - unsigned char fsx_pad[12]; -}; -#endif - -/* - * Flags for the bs_xflags/fsx_xflags field - * There should be a one-to-one correspondence between these flags and the - * XFS_DIFLAG_s. - */ -#define XFS_XFLAG_REALTIME 0x00000001 /* data in realtime volume */ -#define XFS_XFLAG_PREALLOC 0x00000002 /* preallocated file extents */ -#define XFS_XFLAG_IMMUTABLE 0x00000008 /* file cannot be modified */ -#define XFS_XFLAG_APPEND 0x00000010 /* all writes append */ -#define XFS_XFLAG_SYNC 0x00000020 /* all writes synchronous */ -#define XFS_XFLAG_NOATIME 0x00000040 /* do not update access time */ -#define XFS_XFLAG_NODUMP 0x00000080 /* do not include in backups */ -#define XFS_XFLAG_RTINHERIT 0x00000100 /* create with rt bit set */ -#define XFS_XFLAG_PROJINHERIT 0x00000200 /* create with parents projid */ -#define XFS_XFLAG_NOSYMLINKS 0x00000400 /* disallow symlink creation */ -#define XFS_XFLAG_EXTSIZE 0x00000800 /* extent size allocator hint */ -#define XFS_XFLAG_EXTSZINHERIT 0x00001000 /* inherit inode extent size */ -#define XFS_XFLAG_NODEFRAG 0x00002000 /* do not defragment */ -#define XFS_XFLAG_FILESTREAM 0x00004000 /* use filestream allocator */ -#define XFS_XFLAG_HASATTR 0x80000000 /* no DIFLAG for this */ +#define XFS_XFLAG_REALTIME FS_XFLAG_REALTIME +#define XFS_XFLAG_PREALLOC FS_XFLAG_PREALLOC +#define XFS_XFLAG_IMMUTABLE FS_XFLAG_IMMUTABLE +#define XFS_XFLAG_APPEND FS_XFLAG_APPEND +#define XFS_XFLAG_SYNC FS_XFLAG_SYNC +#define XFS_XFLAG_NOATIME FS_XFLAG_NOATIME +#define XFS_XFLAG_NODUMP FS_XFLAG_NODUMP +#define XFS_XFLAG_RTINHERIT FS_XFLAG_RTINHERIT +#define XFS_XFLAG_PROJINHERIT FS_XFLAG_PROJINHERIT +#define XFS_XFLAG_NOSYMLINKS FS_XFLAG_NOSYMLINKS +#define XFS_XFLAG_EXTSIZE FS_XFLAG_EXTSIZE +#define XFS_XFLAG_EXTSZINHERIT FS_XFLAG_EXTSZINHERIT +#define XFS_XFLAG_NODEFRAG FS_XFLAG_NODEFRAG +#define XFS_XFLAG_FILESTREAM FS_XFLAG_FILESTREAM +#define XFS_XFLAG_HASATTR FS_XFLAG_HASATTR /* * Structure for XFS_IOC_GETBMAP. @@ -514,8 +499,8 @@ typedef struct xfs_swapext #define XFS_IOC_ALLOCSP _IOW ('X', 10, struct xfs_flock64) #define XFS_IOC_FREESP _IOW ('X', 11, struct xfs_flock64) #define XFS_IOC_DIOINFO _IOR ('X', 30, struct dioattr) -#define XFS_IOC_FSGETXATTR _IOR ('X', 31, struct fsxattr) -#define XFS_IOC_FSSETXATTR _IOW ('X', 32, struct fsxattr) +#define XFS_IOC_FSGETXATTR FS_IOC_FSGETXATTR +#define XFS_IOC_FSSETXATTR FS_IOC_FSSETXATTR #define XFS_IOC_ALLOCSP64 _IOW ('X', 36, struct xfs_flock64) #define XFS_IOC_FREESP64 _IOW ('X', 37, struct xfs_flock64) #define XFS_IOC_GETBMAP _IOWR('X', 38, struct getbmap) diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index f15d980..df175dd 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -110,6 +110,36 @@ struct inodes_stat_t { #define MS_MGC_VAL 0xC0ED0000 #define MS_MGC_MSK 0xffff0000 +/* + * Structure for FS_IOC_FSGETXATTR[A] and FS_IOC_FSSETXATTR. + */ +struct fsxattr { + __u32 fsx_xflags; /* xflags field value (get/set) */ + __u32 fsx_extsize; /* extsize field value (get/set)*/ + __u32 fsx_nextents; /* nextents field value (get) */ + __u32 fsx_projid; /* project identifier (get/set) */ + unsigned char fsx_pad[12]; +}; + +/* + * Flags for the fsx_xflags field + */ +#define FS_XFLAG_REALTIME 0x00000001 /* data in realtime volume */ +#define FS_XFLAG_PREALLOC 0x00000002 /* preallocated file extents */ +#define FS_XFLAG_IMMUTABLE 0x00000008 /* file cannot be modified */ +#define FS_XFLAG_APPEND 0x00000010 /* all writes append */ +#define FS_XFLAG_SYNC 0x00000020 /* all writes synchronous */ +#define FS_XFLAG_NOATIME 0x00000040 /* do not update access time */ +#define FS_XFLAG_NODUMP 0x00000080 /* do not include in backups */ +#define FS_XFLAG_RTINHERIT 0x00000100 /* create with rt bit set */ +#define FS_XFLAG_PROJINHERIT 0x00000200 /* create with parents projid */ +#define FS_XFLAG_NOSYMLINKS 0x00000400 /* disallow symlink creation */ +#define FS_XFLAG_EXTSIZE 0x00000800 /* extent size allocator hint */ +#define FS_XFLAG_EXTSZINHERIT 0x00001000 /* inherit inode extent size */ +#define FS_XFLAG_NODEFRAG 0x00002000 /* do not defragment */ +#define FS_XFLAG_FILESTREAM 0x00004000 /* use filestream allocator */ +#define FS_XFLAG_HASATTR 0x80000000 /* no DIFLAG for this */ + /* the read-only stuff doesn't really belong here, but any other place is probably as bad and I don't want to create yet another include file. */ @@ -169,6 +199,8 @@ struct inodes_stat_t { #define FS_IOC32_SETFLAGS _IOW('f', 2, int) #define FS_IOC32_GETVERSION _IOR('v', 1, int) #define FS_IOC32_SETVERSION _IOW('v', 2, int) +#define FS_IOC_FSGETXATTR _IOR ('X', 31, struct fsxattr) +#define FS_IOC_FSSETXATTR _IOW ('X', 32, struct fsxattr) /* * Inode flags (FS_IOC_GETFLAGS / FS_IOC_SETFLAGS) -- 2.5.0 From dave@fromorbit.com Sun Jan 3 23:55:06 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id B0C877F54 for ; Sun, 3 Jan 2016 23:55:06 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id A1BF38F8035 for ; Sun, 3 Jan 2016 21:55:03 -0800 (PST) X-ASG-Debug-ID: 1451886898-04cb6c303241b3e0002-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id RAhxPNMP4VwXHou2 for ; Sun, 03 Jan 2016 21:55:01 -0800 (PST) X-Barracuda-Envelope-From: dave@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CCDQAYCIpWPGu7LXleKAECgw+BP4ZggXmgTQabIoEaTQEBAQEBAQcBAQEBQT+ENQEFJy8jEAgYMTkDBxQZiC7AEoYOikSEPwWXBo8th26FMY46gi4BQQUcgXEqNIUQAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:24:57 +1030 Received: from disappointment.disaster.area ([192.168.1.110] helo=disappointment) by dastard with esmtp (Exim 4.80) (envelope-from ) id 1aFy6Z-0000td-U2; Mon, 04 Jan 2016 16:54:55 +1100 Received: from dave by disappointment with local (Exim 4.86) (envelope-from ) id 1aFy6Z-00047G-TH; Mon, 04 Jan 2016 16:54:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, ext4@vger.kernel.org Subject: [PATCH 3/3] xfs: introduce per-inode DAX enablement Date: Mon, 4 Jan 2016 16:54:52 +1100 X-ASG-Orig-Subj: [PATCH 3/3] xfs: introduce per-inode DAX enablement Message-Id: <1451886892-15548-4-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451886892-15548-1-git-send-email-david@fromorbit.com> References: <1451886892-15548-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451886900 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- From: Dave Chinner Rather than just being able to turn DAX on and off via a mount option, some applications may only want to enable DAX for certain performance critical files in a filesystem. This patch introduces a new inode flag to enable DAX in the v3 inode di_flags2 field. It adds support for setting and clearing flags in the di_flags2 field via the XFS_IOC_FSSETXATTR ioctl, and sets the S_DAX inode flag appropriately when it is seen. When this flag is set on a directory, it acts as an "inherit flag". That is, inodes created in the directory will automatically inherit the on-disk inode DAX flag, enabling administrators to set up directory heirarchies that automatically use DAX. Setting this flag on an empty root directory will make the entire filesystem use DAX by default. Signed-off-by: Dave Chinner --- fs/xfs/libxfs/xfs_format.h | 9 +++++++++ fs/xfs/xfs_inode.c | 32 +++++++++++++++++++++++--------- fs/xfs/xfs_ioctl.c | 18 +++++++++++++++++- fs/xfs/xfs_iops.c | 4 ++-- include/uapi/linux/fs.h | 1 + 5 files changed, 52 insertions(+), 12 deletions(-) diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h index f28eeab..b4ae7ce 100644 --- a/fs/xfs/libxfs/xfs_format.h +++ b/fs/xfs/libxfs/xfs_format.h @@ -1024,6 +1024,15 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev) XFS_DIFLAG_EXTSZINHERIT | XFS_DIFLAG_NODEFRAG | XFS_DIFLAG_FILESTREAM) /* + * Values for di_flags2 These start by being exposed to userspace in the upper + * 16 bits of the XFS_XFLAG_s range. + */ +#define XFS_DIFLAG2_DAX_BIT 0 /* use DAX for this inode */ +#define XFS_DIFLAG2_DAX (1 << XFS_DIFLAG2_DAX_BIT) + +#define XFS_DIFLAG2_ANY (XFS_DIFLAG2_DAX) + +/* * Inode number format: * low inopblog bits - offset in block * next agblklog bits - block number in ag diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index ca9ca5a..8929908 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -610,7 +610,9 @@ __xfs_iflock( STATIC uint _xfs_dic2xflags( - __uint16_t di_flags) + __uint16_t di_flags, + uint64_t di_flags2, + bool has_attr) { uint flags = 0; @@ -645,25 +647,32 @@ _xfs_dic2xflags( flags |= FS_XFLAG_FILESTREAM; } + if (di_flags2 & XFS_DIFLAG2_ANY) { + if (di_flags2 & XFS_DIFLAG2_DAX) + flags |= FS_XFLAG_DAX; + } + + if (has_attr) + flags |= FS_XFLAG_HASATTR; + return flags; } uint xfs_ip2xflags( - xfs_inode_t *ip) + struct xfs_inode *ip) { - xfs_icdinode_t *dic = &ip->i_d; + struct xfs_icdinode *dic = &ip->i_d; - return _xfs_dic2xflags(dic->di_flags) | - (XFS_IFORK_Q(ip) ? FS_XFLAG_HASATTR : 0); + return _xfs_dic2xflags(dic->di_flags, dic->di_flags2, XFS_IFORK_Q(ip)); } uint xfs_dic2xflags( - xfs_dinode_t *dip) + struct xfs_dinode *dip) { - return _xfs_dic2xflags(be16_to_cpu(dip->di_flags)) | - (XFS_DFORK_Q(dip) ? FS_XFLAG_HASATTR : 0); + return _xfs_dic2xflags(be16_to_cpu(dip->di_flags), + be64_to_cpu(dip->di_flags2), XFS_DFORK_Q(dip)); } /* @@ -862,7 +871,8 @@ xfs_ialloc( case S_IFREG: case S_IFDIR: if (pip && (pip->i_d.di_flags & XFS_DIFLAG_ANY)) { - uint di_flags = 0; + uint64_t di_flags2 = 0; + uint di_flags = 0; if (S_ISDIR(mode)) { if (pip->i_d.di_flags & XFS_DIFLAG_RTINHERIT) @@ -898,7 +908,11 @@ xfs_ialloc( di_flags |= XFS_DIFLAG_NODEFRAG; if (pip->i_d.di_flags & XFS_DIFLAG_FILESTREAM) di_flags |= XFS_DIFLAG_FILESTREAM; + if (pip->i_d.di_flags2 & XFS_DIFLAG2_DAX) + di_flags2 |= XFS_DIFLAG2_DAX; + ip->i_d.di_flags |= di_flags; + ip->i_d.di_flags2 |= di_flags2; } /* FALLTHROUGH */ case S_IFLNK: diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c index 94b35eb3..478d04e 100644 --- a/fs/xfs/xfs_ioctl.c +++ b/fs/xfs/xfs_ioctl.c @@ -945,6 +945,7 @@ xfs_set_diflags( unsigned int xflags) { unsigned int di_flags; + uint64_t di_flags2; /* can't set PREALLOC this way, just preserve it */ di_flags = (ip->i_d.di_flags & XFS_DIFLAG_PREALLOC); @@ -977,8 +978,18 @@ xfs_set_diflags( if (xflags & FS_XFLAG_EXTSIZE) di_flags |= XFS_DIFLAG_EXTSIZE; } - ip->i_d.di_flags = di_flags; + + /* diflags2 only valid for v3 inodes. */ + if (ip->i_d.di_version < 3) + return; + + di_flags2 = 0; + if (xflags & FS_XFLAG_DAX) + di_flags2 |= XFS_DIFLAG2_DAX; + + ip->i_d.di_flags2 = di_flags2; + } STATIC void @@ -1004,6 +1015,11 @@ xfs_diflags_to_linux( inode->i_flags |= S_NOATIME; else inode->i_flags &= ~S_NOATIME; + if (xflags & FS_XFLAG_DAX) + inode->i_flags |= S_DAX; + else + inode->i_flags &= ~S_DAX; + } static int diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index 245268a..a1b8af1 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -1201,8 +1201,8 @@ xfs_diflags_to_iflags( inode->i_flags |= S_SYNC; if (flags & XFS_DIFLAG_NOATIME) inode->i_flags |= S_NOATIME; - /* XXX: Also needs an on-disk per inode flag! */ - if (ip->i_mount->m_flags & XFS_MOUNT_DAX) + if (ip->i_mount->m_flags & XFS_MOUNT_DAX || + ip->i_d.di_flags2 & XFS_DIFLAG2_DAX) inode->i_flags |= S_DAX; } diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h index df175dd..4cad4c8 100644 --- a/include/uapi/linux/fs.h +++ b/include/uapi/linux/fs.h @@ -138,6 +138,7 @@ struct fsxattr { #define FS_XFLAG_EXTSZINHERIT 0x00001000 /* inherit inode extent size */ #define FS_XFLAG_NODEFRAG 0x00002000 /* do not defragment */ #define FS_XFLAG_FILESTREAM 0x00004000 /* use filestream allocator */ +#define FS_XFLAG_DAX 0x00008000 /* use DAX for IO */ #define FS_XFLAG_HASATTR 0x80000000 /* no DIFLAG for this */ /* the read-only stuff doesn't really belong here, but any other place is -- 2.5.0 From dave@fromorbit.com Sun Jan 3 23:55:10 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D7D177F3F for ; Sun, 3 Jan 2016 23:55:09 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 4E1D4AC002 for ; Sun, 3 Jan 2016 21:55:06 -0800 (PST) X-ASG-Debug-ID: 1451886898-04cb6c303241b3e0003-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id tUBGl2Gxqcps4pVJ for ; Sun, 03 Jan 2016 21:55:03 -0800 (PST) X-Barracuda-Envelope-From: dave@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CCDQAYCIpWPGu7LXleKAECgw+BP4ZggXmgTQabIoEaTQEBAQEBAQcBAQEBQT+ENQEFJy8jEAgYMTkDBxQZiC7AEoYOikSEPwWHXo8onExEjXaCLgFGHIFxKjSDRyWBJAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:24:57 +1030 Received: from disappointment.disaster.area ([192.168.1.110] helo=disappointment) by dastard with esmtp (Exim 4.80) (envelope-from ) id 1aFy6Z-0000tc-TY; Mon, 04 Jan 2016 16:54:55 +1100 Received: from dave by disappointment with local (Exim 4.86) (envelope-from ) id 1aFy6Z-00047B-Sl; Mon, 04 Jan 2016 16:54:55 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, ext4@vger.kernel.org Subject: [PATCH 2/3] xfs: use FS_XFLAG definitions directly Date: Mon, 4 Jan 2016 16:54:51 +1100 X-ASG-Orig-Subj: [PATCH 2/3] xfs: use FS_XFLAG definitions directly Message-Id: <1451886892-15548-3-git-send-email-david@fromorbit.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451886892-15548-1-git-send-email-david@fromorbit.com> References: <1451886892-15548-1-git-send-email-david@fromorbit.com> X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451886902 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- From: Dave Chinner Now that the ioctls have been hoisted up to the VFS level, use the VFs definitions directly and remove the XFS specific definitions completely. Userspace is going to have to handle the change of this interface separately, so removing the definitions from xfs_fs.h is not an issue here at all. Signed-off-by: Dave Chinner --- fs/xfs/libxfs/xfs_format.h | 2 -- fs/xfs/libxfs/xfs_fs.h | 19 ------------ fs/xfs/xfs_inode.c | 32 ++++++++++---------- fs/xfs/xfs_ioctl.c | 74 +++++++++++++++++++++++----------------------- 4 files changed, 53 insertions(+), 74 deletions(-) diff --git a/fs/xfs/libxfs/xfs_format.h b/fs/xfs/libxfs/xfs_format.h index 8774498..f28eeab 100644 --- a/fs/xfs/libxfs/xfs_format.h +++ b/fs/xfs/libxfs/xfs_format.h @@ -984,8 +984,6 @@ static inline void xfs_dinode_put_rdev(struct xfs_dinode *dip, xfs_dev_t rdev) /* * Values for di_flags - * There should be a one-to-one correspondence between these flags and the - * XFS_XFLAG_s. */ #define XFS_DIFLAG_REALTIME_BIT 0 /* file's blocks come from rt area */ #define XFS_DIFLAG_PREALLOC_BIT 1 /* file space has been preallocated */ diff --git a/fs/xfs/libxfs/xfs_fs.h b/fs/xfs/libxfs/xfs_fs.h index dd29d0a..fffe3d0 100644 --- a/fs/xfs/libxfs/xfs_fs.h +++ b/fs/xfs/libxfs/xfs_fs.h @@ -36,25 +36,6 @@ struct dioattr { #endif /* - * Flags for the bs_xflags/fsx_xflags field in FS_IOC_FS[GS]ETXATTR[A] - */ -#define XFS_XFLAG_REALTIME FS_XFLAG_REALTIME -#define XFS_XFLAG_PREALLOC FS_XFLAG_PREALLOC -#define XFS_XFLAG_IMMUTABLE FS_XFLAG_IMMUTABLE -#define XFS_XFLAG_APPEND FS_XFLAG_APPEND -#define XFS_XFLAG_SYNC FS_XFLAG_SYNC -#define XFS_XFLAG_NOATIME FS_XFLAG_NOATIME -#define XFS_XFLAG_NODUMP FS_XFLAG_NODUMP -#define XFS_XFLAG_RTINHERIT FS_XFLAG_RTINHERIT -#define XFS_XFLAG_PROJINHERIT FS_XFLAG_PROJINHERIT -#define XFS_XFLAG_NOSYMLINKS FS_XFLAG_NOSYMLINKS -#define XFS_XFLAG_EXTSIZE FS_XFLAG_EXTSIZE -#define XFS_XFLAG_EXTSZINHERIT FS_XFLAG_EXTSZINHERIT -#define XFS_XFLAG_NODEFRAG FS_XFLAG_NODEFRAG -#define XFS_XFLAG_FILESTREAM FS_XFLAG_FILESTREAM -#define XFS_XFLAG_HASATTR FS_XFLAG_HASATTR - -/* * Structure for XFS_IOC_GETBMAP. * On input, fill in bmv_offset and bmv_length of the first structure * to indicate the area of interest in the file, and bmv_entries with diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 8ee3939..ca9ca5a 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -616,33 +616,33 @@ _xfs_dic2xflags( if (di_flags & XFS_DIFLAG_ANY) { if (di_flags & XFS_DIFLAG_REALTIME) - flags |= XFS_XFLAG_REALTIME; + flags |= FS_XFLAG_REALTIME; if (di_flags & XFS_DIFLAG_PREALLOC) - flags |= XFS_XFLAG_PREALLOC; + flags |= FS_XFLAG_PREALLOC; if (di_flags & XFS_DIFLAG_IMMUTABLE) - flags |= XFS_XFLAG_IMMUTABLE; + flags |= FS_XFLAG_IMMUTABLE; if (di_flags & XFS_DIFLAG_APPEND) - flags |= XFS_XFLAG_APPEND; + flags |= FS_XFLAG_APPEND; if (di_flags & XFS_DIFLAG_SYNC) - flags |= XFS_XFLAG_SYNC; + flags |= FS_XFLAG_SYNC; if (di_flags & XFS_DIFLAG_NOATIME) - flags |= XFS_XFLAG_NOATIME; + flags |= FS_XFLAG_NOATIME; if (di_flags & XFS_DIFLAG_NODUMP) - flags |= XFS_XFLAG_NODUMP; + flags |= FS_XFLAG_NODUMP; if (di_flags & XFS_DIFLAG_RTINHERIT) - flags |= XFS_XFLAG_RTINHERIT; + flags |= FS_XFLAG_RTINHERIT; if (di_flags & XFS_DIFLAG_PROJINHERIT) - flags |= XFS_XFLAG_PROJINHERIT; + flags |= FS_XFLAG_PROJINHERIT; if (di_flags & XFS_DIFLAG_NOSYMLINKS) - flags |= XFS_XFLAG_NOSYMLINKS; + flags |= FS_XFLAG_NOSYMLINKS; if (di_flags & XFS_DIFLAG_EXTSIZE) - flags |= XFS_XFLAG_EXTSIZE; + flags |= FS_XFLAG_EXTSIZE; if (di_flags & XFS_DIFLAG_EXTSZINHERIT) - flags |= XFS_XFLAG_EXTSZINHERIT; + flags |= FS_XFLAG_EXTSZINHERIT; if (di_flags & XFS_DIFLAG_NODEFRAG) - flags |= XFS_XFLAG_NODEFRAG; + flags |= FS_XFLAG_NODEFRAG; if (di_flags & XFS_DIFLAG_FILESTREAM) - flags |= XFS_XFLAG_FILESTREAM; + flags |= FS_XFLAG_FILESTREAM; } return flags; @@ -655,7 +655,7 @@ xfs_ip2xflags( xfs_icdinode_t *dic = &ip->i_d; return _xfs_dic2xflags(dic->di_flags) | - (XFS_IFORK_Q(ip) ? XFS_XFLAG_HASATTR : 0); + (XFS_IFORK_Q(ip) ? FS_XFLAG_HASATTR : 0); } uint @@ -663,7 +663,7 @@ xfs_dic2xflags( xfs_dinode_t *dip) { return _xfs_dic2xflags(be16_to_cpu(dip->di_flags)) | - (XFS_DFORK_Q(dip) ? XFS_XFLAG_HASATTR : 0); + (XFS_DFORK_Q(dip) ? FS_XFLAG_HASATTR : 0); } /* diff --git a/fs/xfs/xfs_ioctl.c b/fs/xfs/xfs_ioctl.c index d42738d..94b35eb3 100644 --- a/fs/xfs/xfs_ioctl.c +++ b/fs/xfs/xfs_ioctl.c @@ -859,25 +859,25 @@ xfs_merge_ioc_xflags( unsigned int xflags = start; if (flags & FS_IMMUTABLE_FL) - xflags |= XFS_XFLAG_IMMUTABLE; + xflags |= FS_XFLAG_IMMUTABLE; else - xflags &= ~XFS_XFLAG_IMMUTABLE; + xflags &= ~FS_XFLAG_IMMUTABLE; if (flags & FS_APPEND_FL) - xflags |= XFS_XFLAG_APPEND; + xflags |= FS_XFLAG_APPEND; else - xflags &= ~XFS_XFLAG_APPEND; + xflags &= ~FS_XFLAG_APPEND; if (flags & FS_SYNC_FL) - xflags |= XFS_XFLAG_SYNC; + xflags |= FS_XFLAG_SYNC; else - xflags &= ~XFS_XFLAG_SYNC; + xflags &= ~FS_XFLAG_SYNC; if (flags & FS_NOATIME_FL) - xflags |= XFS_XFLAG_NOATIME; + xflags |= FS_XFLAG_NOATIME; else - xflags &= ~XFS_XFLAG_NOATIME; + xflags &= ~FS_XFLAG_NOATIME; if (flags & FS_NODUMP_FL) - xflags |= XFS_XFLAG_NODUMP; + xflags |= FS_XFLAG_NODUMP; else - xflags &= ~XFS_XFLAG_NODUMP; + xflags &= ~FS_XFLAG_NODUMP; return xflags; } @@ -948,33 +948,33 @@ xfs_set_diflags( /* can't set PREALLOC this way, just preserve it */ di_flags = (ip->i_d.di_flags & XFS_DIFLAG_PREALLOC); - if (xflags & XFS_XFLAG_IMMUTABLE) + if (xflags & FS_XFLAG_IMMUTABLE) di_flags |= XFS_DIFLAG_IMMUTABLE; - if (xflags & XFS_XFLAG_APPEND) + if (xflags & FS_XFLAG_APPEND) di_flags |= XFS_DIFLAG_APPEND; - if (xflags & XFS_XFLAG_SYNC) + if (xflags & FS_XFLAG_SYNC) di_flags |= XFS_DIFLAG_SYNC; - if (xflags & XFS_XFLAG_NOATIME) + if (xflags & FS_XFLAG_NOATIME) di_flags |= XFS_DIFLAG_NOATIME; - if (xflags & XFS_XFLAG_NODUMP) + if (xflags & FS_XFLAG_NODUMP) di_flags |= XFS_DIFLAG_NODUMP; - if (xflags & XFS_XFLAG_NODEFRAG) + if (xflags & FS_XFLAG_NODEFRAG) di_flags |= XFS_DIFLAG_NODEFRAG; - if (xflags & XFS_XFLAG_FILESTREAM) + if (xflags & FS_XFLAG_FILESTREAM) di_flags |= XFS_DIFLAG_FILESTREAM; if (S_ISDIR(ip->i_d.di_mode)) { - if (xflags & XFS_XFLAG_RTINHERIT) + if (xflags & FS_XFLAG_RTINHERIT) di_flags |= XFS_DIFLAG_RTINHERIT; - if (xflags & XFS_XFLAG_NOSYMLINKS) + if (xflags & FS_XFLAG_NOSYMLINKS) di_flags |= XFS_DIFLAG_NOSYMLINKS; - if (xflags & XFS_XFLAG_EXTSZINHERIT) + if (xflags & FS_XFLAG_EXTSZINHERIT) di_flags |= XFS_DIFLAG_EXTSZINHERIT; - if (xflags & XFS_XFLAG_PROJINHERIT) + if (xflags & FS_XFLAG_PROJINHERIT) di_flags |= XFS_DIFLAG_PROJINHERIT; } else if (S_ISREG(ip->i_d.di_mode)) { - if (xflags & XFS_XFLAG_REALTIME) + if (xflags & FS_XFLAG_REALTIME) di_flags |= XFS_DIFLAG_REALTIME; - if (xflags & XFS_XFLAG_EXTSIZE) + if (xflags & FS_XFLAG_EXTSIZE) di_flags |= XFS_DIFLAG_EXTSIZE; } @@ -988,19 +988,19 @@ xfs_diflags_to_linux( struct inode *inode = VFS_I(ip); unsigned int xflags = xfs_ip2xflags(ip); - if (xflags & XFS_XFLAG_IMMUTABLE) + if (xflags & FS_XFLAG_IMMUTABLE) inode->i_flags |= S_IMMUTABLE; else inode->i_flags &= ~S_IMMUTABLE; - if (xflags & XFS_XFLAG_APPEND) + if (xflags & FS_XFLAG_APPEND) inode->i_flags |= S_APPEND; else inode->i_flags &= ~S_APPEND; - if (xflags & XFS_XFLAG_SYNC) + if (xflags & FS_XFLAG_SYNC) inode->i_flags |= S_SYNC; else inode->i_flags &= ~S_SYNC; - if (xflags & XFS_XFLAG_NOATIME) + if (xflags & FS_XFLAG_NOATIME) inode->i_flags |= S_NOATIME; else inode->i_flags &= ~S_NOATIME; @@ -1016,11 +1016,11 @@ xfs_ioctl_setattr_xflags( /* Can't change realtime flag if any extents are allocated. */ if ((ip->i_d.di_nextents || ip->i_delayed_blks) && - XFS_IS_REALTIME_INODE(ip) != (fa->fsx_xflags & XFS_XFLAG_REALTIME)) + XFS_IS_REALTIME_INODE(ip) != (fa->fsx_xflags & FS_XFLAG_REALTIME)) return -EINVAL; /* If realtime flag is set then must have realtime device */ - if (fa->fsx_xflags & XFS_XFLAG_REALTIME) { + if (fa->fsx_xflags & FS_XFLAG_REALTIME) { if (mp->m_sb.sb_rblocks == 0 || mp->m_sb.sb_rextsize == 0 || (ip->i_d.di_extsize % mp->m_sb.sb_rextsize)) return -EINVAL; @@ -1031,7 +1031,7 @@ xfs_ioctl_setattr_xflags( * we have appropriate permission. */ if (((ip->i_d.di_flags & (XFS_DIFLAG_IMMUTABLE | XFS_DIFLAG_APPEND)) || - (fa->fsx_xflags & (XFS_XFLAG_IMMUTABLE | XFS_XFLAG_APPEND))) && + (fa->fsx_xflags & (FS_XFLAG_IMMUTABLE | FS_XFLAG_APPEND))) && !capable(CAP_LINUX_IMMUTABLE)) return -EPERM; @@ -1095,8 +1095,8 @@ out_cancel: * extent size hint validation is somewhat cumbersome. Rules are: * * 1. extent size hint is only valid for directories and regular files - * 2. XFS_XFLAG_EXTSIZE is only valid for regular files - * 3. XFS_XFLAG_EXTSZINHERIT is only valid for directories. + * 2. FS_XFLAG_EXTSIZE is only valid for regular files + * 3. FS_XFLAG_EXTSZINHERIT is only valid for directories. * 4. can only be changed on regular files if no extents are allocated * 5. can be changed on directories at any time * 6. extsize hint of 0 turns off hints, clears inode flags. @@ -1112,10 +1112,10 @@ xfs_ioctl_setattr_check_extsize( { struct xfs_mount *mp = ip->i_mount; - if ((fa->fsx_xflags & XFS_XFLAG_EXTSIZE) && !S_ISREG(ip->i_d.di_mode)) + if ((fa->fsx_xflags & FS_XFLAG_EXTSIZE) && !S_ISREG(ip->i_d.di_mode)) return -EINVAL; - if ((fa->fsx_xflags & XFS_XFLAG_EXTSZINHERIT) && + if ((fa->fsx_xflags & FS_XFLAG_EXTSZINHERIT) && !S_ISDIR(ip->i_d.di_mode)) return -EINVAL; @@ -1132,7 +1132,7 @@ xfs_ioctl_setattr_check_extsize( return -EINVAL; if (XFS_IS_REALTIME_INODE(ip) || - (fa->fsx_xflags & XFS_XFLAG_REALTIME)) { + (fa->fsx_xflags & FS_XFLAG_REALTIME)) { size = mp->m_sb.sb_rextsize << mp->m_sb.sb_blocklog; } else { size = mp->m_sb.sb_blocksize; @@ -1143,7 +1143,7 @@ xfs_ioctl_setattr_check_extsize( if (fa->fsx_extsize % size) return -EINVAL; } else - fa->fsx_xflags &= ~(XFS_XFLAG_EXTSIZE | XFS_XFLAG_EXTSZINHERIT); + fa->fsx_xflags &= ~(FS_XFLAG_EXTSIZE | FS_XFLAG_EXTSZINHERIT); return 0; } @@ -1168,7 +1168,7 @@ xfs_ioctl_setattr_check_projid( if (xfs_get_projid(ip) != fa->fsx_projid) return -EINVAL; - if ((fa->fsx_xflags & XFS_XFLAG_PROJINHERIT) != + if ((fa->fsx_xflags & FS_XFLAG_PROJINHERIT) != (ip->i_d.di_flags & XFS_DIFLAG_PROJINHERIT)) return -EINVAL; -- 2.5.0 From david@fromorbit.com Mon Jan 4 00:22:37 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id D699C7F37 for ; Mon, 4 Jan 2016 00:22:37 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id C6347304032 for ; Sun, 3 Jan 2016 22:22:34 -0800 (PST) X-ASG-Debug-ID: 1451888552-04cb6c303241ba00001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id RAZnCcMAw4YDrPAG for ; Sun, 03 Jan 2016 22:22:32 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CMDQBKD4pWPGu7LXleKAECgw+BP4ZggXmgTQaLVYk+hgkEAoEZTQEBAQEBAQcBAQEBQT+EdRwjGCQ0BSUDBw4fiC7AChmFdYpqgn6BGwWTCoN8jUiBZY0fhViIYoUCKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:52:20 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFyX5-0000xF-Ep; Mon, 04 Jan 2016 17:22:19 +1100 Date: Mon, 4 Jan 2016 17:22:19 +1100 From: Dave Chinner To: xfs@oss.sgi.com Cc: ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org Subject: lazytime implementation questions Message-ID: <20160104062219.GB19802@dastard> X-ASG-Orig-Subj: lazytime implementation questions MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451888552 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Hi folks, I've been looking at implementing the lazytime mount option for XFS, and I'm struggling to work out what it is supposed to mean. AFAICT, on ext4, lazytime means that pure timestamp updates are not journalled and they are only ever written back when the inode is otherwise dirtied and written, or they are timestamp dirty for 24 hours which triggers writeback. This poses a couple of problems for XFS: 1. we log every timestamp change, so there is no mechanism for delayed/deferred update. 2. we track dirty metadata in the journal, not via the VFS dirty inode lists, so all the infrastructure written for ext4 to do periodic flushing is useless to us. These are solvable problems, but what I'm not sure about is exactly what the intended semantics of lazytime durability are. That is, exactly what guaranteed are we giving userspace about timestamp updates when lazytime is used? The guarantees we have to give will greatly influence the XFS implementation, so I really need to nail down what we are expected to provide userspace. Can we: a) just ignore all durability concerns? b) if not, do we only need to care about the 24 hour writeback and unmount? c) if not, are fsync/sync/syncfs/freeze/unmount supposed to provide durability of all metadata changes? d) do we have to care about ordering - if we fsync one inode with 1 hour old timestamps, do we also need to guarantee that all the inodes with older dirty timestamps also get made durable? I really want to completely ignore all ordering and forced durability requirements for lazytime (i.e. implement only periodic, optimistic and freeze/unmount writeback), but I haven't found any documentation of what durability lazytime is supposed to provide and can really only guess what was intended from the ext4 implementation.... Clarity would be appreciated. Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Jan 4 00:29:14 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 349297F37 for ; Mon, 4 Jan 2016 00:29:14 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 15149304032 for ; Sun, 3 Jan 2016 22:29:13 -0800 (PST) X-ASG-Debug-ID: 1451888948-04cb6c302f41bb40001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id tqEDSj6IPAk95EjE for ; Sun, 03 Jan 2016 22:29:09 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQDQBNEIpWPGu7LXleKAECgw8iMG2GYIF5oC8CChIGgQyKSYk+GAKFbwSBG00BAQEBAQEHAQEBAUE/hGIvOyQ0BSUDBy2ILp5noRoJGYV1igIJgnUMQRuBGwWNOYVRg3yFQIgIgWWHboUvAo46gjqCSCo0g0aBSgEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 04 Jan 2016 16:59:07 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aFydf-0000z8-DB for xfs@oss.sgi.com; Mon, 04 Jan 2016 17:29:07 +1100 Date: Mon, 4 Jan 2016 17:29:07 +1100 From: Dave Chinner To: xfs@oss.sgi.com Subject: [ANNOUNCE] xfs: for-next branch updated to 887a9f2 Message-ID: <20160104062907.GC19802@dastard> X-ASG-Orig-Subj: [ANNOUNCE] xfs: for-next branch updated to 887a9f2 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="s2ZSL+KKDSLx8OML" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451888948 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25811 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- --s2ZSL+KKDSLx8OML Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi folks, The for-next branch of the xfs kernel repository at git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git has just been updated. The new head of the for-next branch is commit: 887a9f2 Merge branch 'xfs-dax-fixes-for-4.5' into for-next New Commits: Alexander Kuleshov (1): [211fe1a] xfs: make xfs_buf_ioend_async() static Brian Foster (8): [a70f9fe] xfs: detect and handle invalid iclog size set by mkfs [9d94901] xfs: refactor log record unpack and data processing [b94fb2d] xfs: refactor and open code log record crc check [d7f3769] xfs: return start block of first bad log record during reco= very [6528250] xfs: support a crc verification only log record pass [eed6b46] xfs: refactor log record start detection into a new helper [841af8e] xfs: detect and trim torn writes during log recovery [35de070] xfs: debug mode log record crc error injection Darrick J. Wong (5): [2e9101d] libxfs: make xfs_alloc_fix_freelist non-static [9b434a3] xfs: fix log ticket type printing [6d3eb1e] libxfs: use a convenience variable instead of open-coding t= he fork [96f859d] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is = correct [c5ab131] libxfs: refactor short btree block verification Dave Chinner (4): [289542c] Merge branch 'xfs-misc-fixes-for-4.5' into for-next [3b0fe47] xfs: Don't use reserved blocks for data blocks with DAX [a6d7636] xfs: fix recursive splice read locking with DAX [887a9f2] Merge branch 'xfs-dax-fixes-for-4.5' into for-next Eric Sandeen (2): [233135b] xfs: print name of verifier if it fails [f1f96c4] xfs: get mp from bma->ip in xfs_bmap code Jia He (1): [1d4292b] libxfs: Optimize the loop for xfs_bitmap_empty Markus Elfring (1): [a841b64] XFS: Use a signed return type for suffix_kstrtoint() Masatake YAMATO (1): [ffc671f] xfs: send warning of project quota to userspace via netlink Code Diffstat: fs/xfs/libxfs/xfs_alloc.c | 4 +- fs/xfs/libxfs/xfs_alloc.h | 1 + fs/xfs/libxfs/xfs_alloc_btree.c | 35 +-- fs/xfs/libxfs/xfs_attr_leaf.c | 1 + fs/xfs/libxfs/xfs_attr_remote.c | 1 + fs/xfs/libxfs/xfs_bit.c | 6 +- fs/xfs/libxfs/xfs_bmap.c | 27 +- fs/xfs/libxfs/xfs_bmap_btree.c | 1 + fs/xfs/libxfs/xfs_btree.c | 58 ++++ fs/xfs/libxfs/xfs_btree.h | 3 + fs/xfs/libxfs/xfs_da_btree.c | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 1 + fs/xfs/libxfs/xfs_dir2_data.c | 2 + fs/xfs/libxfs/xfs_dir2_leaf.c | 2 + fs/xfs/libxfs/xfs_dir2_node.c | 1 + fs/xfs/libxfs/xfs_dquot_buf.c | 1 + fs/xfs/libxfs/xfs_format.h | 2 +- fs/xfs/libxfs/xfs_ialloc.c | 1 + fs/xfs/libxfs/xfs_ialloc_btree.c | 27 +- fs/xfs/libxfs/xfs_inode_buf.c | 2 + fs/xfs/libxfs/xfs_log_recover.h | 1 + fs/xfs/libxfs/xfs_sb.c | 2 + fs/xfs/libxfs/xfs_symlink_remote.c | 1 + fs/xfs/xfs_buf.c | 2 +- fs/xfs/xfs_buf.h | 1 + fs/xfs/xfs_error.c | 4 +- fs/xfs/xfs_file.c | 25 +- fs/xfs/xfs_iomap.c | 11 +- fs/xfs/xfs_log.c | 51 +++- fs/xfs/xfs_log_priv.h | 3 + fs/xfs/xfs_log_recover.c | 569 ++++++++++++++++++++++++++++++---= ---- fs/xfs/xfs_super.c | 2 +- fs/xfs/xfs_sysfs.c | 36 +++ fs/xfs/xfs_trans_dquot.c | 14 +- 34 files changed, 691 insertions(+), 208 deletions(-) --=20 Dave Chinner david@fromorbit.com --s2ZSL+KKDSLx8OML Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJWihEyAAoJEK3oKUf0dfodH6sP/0MHR5pfBQZM+ug4YiZDWr4F PNs1VrgWyq/D4vLYXq9Qn8lLnsLf0x+hk+wcbXLFDJzhg4vR7ypUtjkMC8QblMIj H/3/djztjSUrm36cBBWhHEL/awqGAFus3ll/RDC+F0qvbdeOmWsOYmXlVzXhOtmB e42HzZBCU9Rb5cjP2AAViCj4qDBj9mD73XXznJPVFPNKsCknerOvz5YZejOqTfjJ 5iz0eeZ+u+W2ydVoqTB9LVv+IyJ200w8LK7yHyT5asbEp5wjwt0f7Fd7ODjZamLg 6j4BuKJpofDlOHW6EQ+COfNwz0TtBeEG0T7AhgXMh/YpJ0k9lAu5TGmYwmNYxFzw vGETWkPZxdwOld355bskQrtXUriUfLr0GBJpjc+GjipjzGAA474B6M8qQssT7p1e ehmE3HT3TTH93ULjCNjHriru4uZiTk5hDituWxZR6M1j2U0rd+DRobkuMgr1P+Za vW9s/VhLIlup02U4MruTMSSHbxqdxMka3EFRUWGA2m/EGQGSo0/5pFG+zf4Kh75B v5GxzfKVTUOncLrrv3c4Yatrk1QHNzwT5j2gYpbH8nvEISnUgDWt2ji+g3T3YRpM CmIC6t/yBsl8GlftPthxgoc7CK9bkK2ge/DARZtCsPkmF8nCFQvf/VAzqpymcrW4 Je+7A3NhQUnVCpfi9Tzx =yEBf -----END PGP SIGNATURE----- --s2ZSL+KKDSLx8OML-- From fengguang.wu@intel.com Mon Jan 4 02:00:27 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 220627F37 for ; Mon, 4 Jan 2016 02:00:27 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 130A7304032 for ; Mon, 4 Jan 2016 00:00:24 -0800 (PST) X-ASG-Debug-ID: 1451894419-04cb6c303241ce50001-NocioJ Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by cuda.sgi.com with ESMTP id cRFKgPuu4o99dekP for ; Mon, 04 Jan 2016 00:00:19 -0800 (PST) X-Barracuda-Envelope-From: fengguang.wu@intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.20 X-ASG-Whitelist: EmailCat (corporate) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga101.jf.intel.com with ESMTP; 04 Jan 2016 00:00:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,519,1444719600"; d="gz'50?scan'50,208,50";a="23149670" Received: from bee.sh.intel.com (HELO bee) ([10.239.97.14]) by fmsmga004.fm.intel.com with ESMTP; 04 Jan 2016 00:00:16 -0800 Received: from kbuild by bee with local (Exim 4.83) (envelope-from ) id 1aG03p-0007lp-2Y; Mon, 04 Jan 2016 16:00:13 +0800 Date: Mon, 4 Jan 2016 15:59:28 +0800 From: kbuild test robot To: Brian Foster Cc: kbuild-all@01.org, xfs@oss.sgi.com, Dave Chinner Subject: [xfs:for-next 7/10] xfs_log_recover.c:undefined reference to `__moddi3' Message-ID: <201601041527.YSzP8A8J%fengguang.wu@intel.com> X-ASG-Orig-Subj: [xfs:for-next 7/10] xfs_log_recover.c:undefined reference to `__moddi3' MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="ew6BAiZeqk4r7MaW" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: fengguang.wu@intel.com X-SA-Exim-Scanned: No (on bee); SAEximRunCond expanded to false X-Barracuda-Connect: mga02.intel.com[134.134.136.20] X-Barracuda-Start-Time: 1451894419 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 --ew6BAiZeqk4r7MaW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline tree: https://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git for-next head: 887a9f293305476865fdd1d5258c0dce3d96104f commit: 841af8ebef885af1db95eb538dde51c439e174e1 [7/10] xfs: detect and trim torn writes during log recovery config: i386-randconfig-r0-01040711 (attached as .config) reproduce: git checkout 841af8ebef885af1db95eb538dde51c439e174e1 # save the attached .config to linux build tree make ARCH=i386 All errors (new ones prefixed by >>): fs/built-in.o: In function `xlog_find_tail': >> xfs_log_recover.c:(.text+0x1c5db5): undefined reference to `__moddi3' xfs_log_recover.c:(.text+0x1c5e9d): undefined reference to `__moddi3' --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation --ew6BAiZeqk4r7MaW Content-Type: application/octet-stream Content-Disposition: attachment; filename=".config.gz" Content-Transfer-Encoding: base64 H4sICFMlilYAAy5jb25maWcAjDzbdtu2su/7K7TS87D3Qxrf4qTrLD9AICihIgkaAGXLL1yq raRedewcS+5O//7MALwA5FBtX1LNDG6DuWPon/7104y9HV6+bQ+P99unp79mX3fPu9ftYfcw +/L4tPvfWaJmhbIzkUj7MxBnj89vPz48nn++nF38fPHzyfvX+8+z1e71efc04y/PXx6/vsHo x5fnf/0E1FwVqVzUlxdzaWeP+9nzy2G23x3+1cBvP1/W52dXfwW/+x+yMFZX3EpV1IngKhG6 R5ZCp7VYi8IaILQiq6uCKy16ClXZsrJ1qnTO7NW73dOX87P3uO93LQXTfAkzp/7n1bvt6/3v H358vvxw786xd6esH3Zf/O9uXKb4KhFlbaqyVNr2SxrL+MpqxsUYt2RrUWfMioJvrCIG53nV /yiESOokZ3XOSpzWigHOLBw6E8XCLnvcQhRCS15LwxA/RsyrxRi4vBFysQz24jiTs43fdcnr NOE9Vt8Ykde3fLlgSVKzbKG0tMt8PC9nmZxr2DxwOWObwfxLZmpeVrUG3C2FY3wJLJMFcFPe iQErjbBViWLg5mBasAGHWpTI5/ArldrYmi+rYjVBV7KFoMn8juRc6II5aSyVMXKeiQGJqUwp imQKfcMKWy8rWKXM4QKXTJMUjnksc5Q2m4/WcGJjalVamQNbEhB/4JEsFlOUiYBLd8djGYju 8Fz+7mueZmxhrt69/4Jm4P1+++fu4f3u/scsBjz8eEevUpVazYXpZ0/lbS2Yzjbwu85FIB7l wjJgDwjvWmTm6ryFd6oIl25AZT88Pf724dvLw9vTbv/hf6qC5QKFRTAjPvw80Empr+sbpYNb m1cyS4BHoha3fj3j9c0ZpoWzck9ojN6+A6QdpNVKFDXs2ORlaIqkrUWxhjPj5nJpr87PWiTX cN81V3kp4c7fvevNXgOrrTCU9YPLYNlaaAMyheMIcM0qqwaSvwI5BHu3uJMljZkD5oxGZXeh UQgxt3dTIybWz+4uekS8p44B4YZCBgwJcFvH8Ld3x0er4+gLgvkgbKzKQCGVsShZV+/+/fzy vPtPdw3mhgX8NRuzliUfAfBfbrPw0KD+IPr5dSUqQe7LCwyohNKbmllwGktig+mSFUloRCoj wJyGK7EqId2quyanno4Ctwiq34o+qMps//bb/q/9YfetF/3OdYAmOV0mvAqgzFLd0Bi+DAUS IYnKmSzCHfdQb5aIzSMJOG0Ols0uwawnkWkzJdNGIFEP4+iQjapgDJhQy5eJGhrDkCRhltGD 1+CvEnRXGUMvsOEZwQRnUtY9T4c+D+fzgclRZJ0DD1jya2UsQZcrtKiJ9/vu1uzjt93rnro4 K/kKLJaAmwmmKlS9vEMLlKvoBgAIHk+qRHKC+X6U9HLXjXFQSs7Ac4BFNo4n2rRbBW/6wW73 f8wOsOfZ9vlhtj9sD/vZ9v7+5e358Pj8NYwN3fadD+dcVYWF656QCsf5nirc4dwkKLRcgGIB BaUUlpkVRlLBtSDIhyZu0ABxS8CkijfgDqJ5NTPUvRSbGnBB4MQhHLkF9ocxo6fo2YHbRErS dOAMcIgsa26WJLJaCEfpQlGCFYhbeeUCrkl1dRKvgHsEuyHquVIUK51rreeyOAsMolw1cXSo 7qvWJsLdkB4QJ0vBqMjUXp1+igxYBfG/99sQByZedwK9XWhVlSZczYPGhmVIkAJ/7oQ+RlLK xBzDJ2It+YR1L8FQ26OjgWQopB0JuAV+dPdNXHh095AcHcM7hpIE82zVENHjl4KvSgUJF6q9 hXyLsiHgTcFM8zAYrMDgFcFv9JxFdHnANA0gYj64iwFtIeyAtEN5WcGoafoUYLtTDItLLThY 2ITSkDhfQbbAlbs4UCdxXKhZDrN5DxJEcToZRGgAGARmAInjMQCEYZjDq8Hvi0jieZcHoM90 eRJxmi5GaV1iAZGlLCCjDq7E65xMToMU3A8Ee8BF6TIfZ1EGY0puypWuS0huMdkOuFam4Wa9 5SO2N1g0h7BMojgE+wCRz9H+jtyuv8seHF4ybr3BEKuuAGw2ecCBFlJHK4CBLGyUVwTBhcjS Oi47THMD0pY6rcKp08qKIPMVpYqOJhcFy9JA3JybDQEulAgBwHKCR8so/WIykCmWrCXsqxkz 0kgXI6eUhpRc1teV1KuAgbDMnGkt3dX1944JeEKqmZefvpQzqO/ADup13matztc2laZy9/rl 5fXb9vl+NxN/7p4humAQZ3CMLyBM6p0wOXmTEY+XaPDr3A+pnaf3kU3Ly6ya+/GBK4I0j1nI HVeRAGZsTukiTDCQUytyF5TWkPTJVHJXZCAtF7jRVGZ0hORU09nmoVIrPyysjTWQusill6oo a/OpPLHGr1VeQnA8F6F0QUwFIetKbEAPQR8w2Y3EaDxbH4zi9lxxEHQOxB1NN8cgbkpWRAr8 kXg1VRGPGAT8eK8YpkAQBsFhlMo5T6OFrXQBka0Fjoemxi0jQaWx8gZbH5bwVsM6h4fCfCQC zDM9wEOx0pBSRtUdwiGWSq0GSKz8Qd6oh5MiHH5buahURaQfBm4PY/smsSKqT+AWN+C/Mc1x 5tYVYAeraLEAE1kkvtzZsL9mpaR2Wcqhtjjc8gbURTAfSwxwubyFW+3Rxq04dFR/f4lhiRjk nMISE7dWQTfHS6p8WGZx3KJUoSldrr1eGZYKyORKLJQOZ2hE1/PX1eYGFM04XwyawCWqmqgy YnjlE+W2vEScwAiOlqsGQ2BHzIPIwp0fNUBwiPQCc0egwogkRsLtFFRMMiaEy6gy9jezAccU af683pJZIqVXBRYSRFOQJfjvrxKLteAdAgHIVVJloLdoX9D5a+L+jceAiqh8XLjmqtw0Cljb LBA/SHAKMFpw3BumkwChINmCYKApL5+PEMy9kbQecsHV+v1v2/3uYfaHd5bfX1++PD4N0m4k aypkBDe7kziy1ltEUYU7TGtZvOVZCmRp6CwZJIlpGMJaCO8g6AntoQuMDPpgyENjNoey4EG+ NARqwah4oqGpCsRPDvZoOvtSSaMxtMNq5oGkvSvQxsHliFLSyVyDRgXUtL9rJc1l8xk4gSoQ w3mcDGfzhKXjdGVuFiTQlxFHuY0VCy3tIO3heeKeYJw50a2QldvXwyO+8s3sX993+1CwgNBK lyxAbMkKTgZ+uUmU6UmDiDaVEdhXLNXM3P++w4eAMLaTymd7hVJh4bGBJqBguPMxhqfXV9/6 WNDXgQPy7igtDhc4UkJu5r1697DbPoCe7bokEO5W5KXtPFQYpZjiNIhJCveKA/JSQsCCAsqH D1DdSwuzCh20zm8GFGiTXNU5cdO4wuY0ib5pCbrzEoURf9uvL/e7/f7ldXaA23YVvS+77eHt dRfcxh1aL//m2Ae3OcU3fPVNBQPfLXyxoN8kosStBTOHL3N9DtXNiAQLsHupNEtSsZDAPwhn paGVGElY3s9PVHN6iUnrfC57cWkhXVQTzaoTfn52ejv50g0OSkYc9wIJ92mB+fi+5SKe+Apa e7uB0ASSNXCCi0qEVWPQF7aWOoq6W9hkmX0FSU47Tx/5r/MmOUpp1mVuiB9IZyjtugM3TGWQ LemgslcoV3f0OWtvMC8+X9KW9OMRhDV8EpfntzTucmpCcL9WVrmUf4M+js+PYi9o7GpiS6tP E/DPNJzryii6cJm75EpMpJ75jSzwhYdPbKRBn5PWXmQseoTIF0IlYnF7Ss/lsHU2cT18o+Xt JJPXkvHzmn5tdMgJhmEVZ2IUGtxJQ9JEQBMq75QZC21NN4MvdF+EJNnpNM7lCTmGqWGBDDFN DnB5EYNzWci8yt17Sgp5XLa5+hjinXJzm+UmDOiBGLyLN5tjMJjKMZDDplhFTOJi61xYFjX2 LEthu0JD6z7DnMzcSBV1O0iV51W9FFkZjilcr4a5Og1rIq2PJfOMFr1WGVgipjeRlfTII8Oc /RqaeWRKOXnneZjMu1ssfMlihAAAMP1XEabYTiQExPQbyDzCdqj4F5KdXs7lYKQwZSpvQ85b BXI1j7yy/LyaFGct0PbCHFVJBqWSa4WtWEFG1II6+ek1p0OBDB2bzWUyTldSXwqJ9gTCOrlf kAZJh/KFwudJ8CJ05cnjLshU0uMuLxahj4VUUKWpEfbq5Ac/8f8NRsR3UbJhbl4uNxC+JYmu ra96DfCuYkKiXaeSsyOwjOsJixJLNzqFcAawtSgY0Zvk4vdptMhACtueBEg9QkMgMxTIrI1N 8Jm8En2uRo/tg8pmWzkrKkblmv3WPElgFFrMsELkl8LMKc43u5l8UXI8bJA3ReCGv2G2729F Gg4JeTg8LmA20UqNqa6bZOphDze9VLbMyIjMlBmEgaV1e3FW7iLah7+4lgzjVzt4EsU6NI/z qVwuNItBR6SwjeHx0hahkZ2DGSQjZB/rQcQW1hvRco+LcisT3GObOzlh9q0Qib66OPnlsjNc R4s0FBbc5Q3bRPE1SZb7Rw/q6TJsCFwF2+WZACuObjrOl1RhsbhMP5ZO9DTdlUpRenA3rxJI NHo6498lCNK27OL669pq9VS+CKwWWsfVRvdyGWg4FoUdHEvOq0FzhTNmKODeL03a09LSku/s NL4c13NIn/C9TFfl8I0j8kEGMh0setxcXXY6AGEY6EBeZQNxzq3W8a/aMDgdGMhJeKtLjRQG daeYzIkOFiYx8GiJT2MjPzTikGKbusSU3snLsMruy4KxpzDR7fUZLMR44T2IlI55m0IufS93 9enJyRTq7OMJ5f/u6vOTk8jMuVlo2qvz3g/6dHapsacnyivFraDkmGtmloMSO5onycHdgA3X 6GxPG1/bt6gIrIq73hAqcmvHu+AZxp9Frrp5rFsnRkWV5qbMNad1E7yaTDd1ltjx82ponGMj 31XLXv67e5192z5vv+6+7Z4ProLCeClnL9+xiBbVz5oy74SWdU3IdPadE3vHhYLdwq82wHQs NKPiou/+dl3BvhaOQ8qwA9xBmpfLUt34xxILU/Vd7X2TIm/fgBYTPTd+fogEU+NnmzgEXPy6 VmswZjIRYZd1PJPgfrmUCmAdBRseZc4sBDabIbSyNm6ec+A1rK6mpk7ZeEACXnqK3iVqWlzX ZfSE2XJEGKz3yKgPNEaOVuuHscUCTBdkr5MMtUuhc5aN5uCVgZy3TkxCV3kckW+G8ULUXcs0 +XRblt82B4nKyHYz73/G79R+q+CCGWjuEdEyc7qY5ZBTnVAhHyCdXaojZOBcK+xXXUKYeIMe QxUZ7SYdOfzfdNeuk89SDJ8rO3jzlBnPiIgJgyCxRQjEgHa3JpVXfdfmLH3d/d/b7vn+r9n+ ftu8GLUGshHUOON3ortQa6LRtEOjaZjqlPMUbTCIE+H7K36yU0y31xGD0GoY4NM/H4JPvK5L 7Z8PUUUCcVZBZbIkPeAwkAFzvRYTbAtOO0XRHm0C350jDB0jin+ybWK7nVB8GQrF7OH18c/o JaYPWcr2Y6U4euQc18KlpivyjTkcErXBT8aKFh1kfPzj2ckY7NZExhXqpl5d9uX7GPFpasSn 1mnGpb5b55zzCVPgIrkSMghwi75ApmVBf5kQk0o+/ZTRU5mcVnG38wtfRB9sLYy/PYvA9uAT aPzJHQQAxUJXRShBLXgJ0jy5rOilc/xotP99+7p7oCKc+FyZnI/GotjJh6ddbH6GbektzElv BgkkGTlEVLkoom5rF+pg1md6Oq6qMpvwCl7IkWy05/nbvj3s7N/gzWa7w/3P/wk60XgkUOjv FgoTItpwO3Se+59HSBKpBdnv7tGsCKIaBOGKMcTPEMPahQeU7ssNMziGwIgP8v+JPQBmWG2I jpCbCdfFj8UMiNX+g7Y29sam30laYyu6ZRqRUq0ncaWe3l7JjJzqjWxbg3wCAOLw+8v+MLt/ eT68vjw9QTowMqLNN5tNJ1UvcoZRjptjXhM/JSJkqX2cREV8mYzSyULYjx9PTgnKhQhlxBog Ownizzypi3koGVge621syXMuWWhMPMQ1bdRcUoKCM/gSUsOv9/fb14fZb6+PD1/j9oINPjzQ N5Jcfjr7hVbbz2cnv9CvO02qhndG4jVcRyKPmPGNSccGTPzY3b8dtr897dxX1DPXrHrYzz7M xLe3p21rEpt5sEkmt9g1FBbzII7EzLgLKbCpaClYErURNEMN17IcNtIxVUXa1tAimDxOg8+l IXN1hWHuuBnOdyNIFdUwypw7TFjB7r69LHaH/768/oGRRO8d2oGMr4AyFB4HASvFqKJpVcig gxp/OcpwAvxaYCXoeBwSB5oXAMfPSbG2kjNNpW84bWnB0WcMEtF0M1AtN7pcbpw6QlCUl4NX 8JDYt+BROmvz8CjwEwwL+c5lbNilpsNeIcjLFpHj9JB6jSGVX5p+oXcEn0/OTq/D0T20Xqw1 1eARUOTrcCuJ4F4Ousk8pNYgkyQDsizI1eHHWdCKEVfH4Gfz3EpnXpZl1DXenn0MOZyxkura LpeqcGLZLiaEwAN+DN5fe1hdZM3/uG8eJL7ghu8aAaVRjby3N8f4cF7kj2kqiU5/rt92bzvQ ng9NU9Sgs6+hr/mciqNb7NLOBxLrwSmp/C16II8tGD8iOzLKfdFxTQ3UZHdYiwXLSu3RpHQW 0eKtuKZqeB16nlKzLvREzNcSJGZoCUYk8O/Ee2M3iaakvGPUtfscb3j1NV+qlaD4d/03rMDH 1WO8SK89SS+APWYMWy5T8vYl/ZbSYLEQ+q2J65+2+/3jl8f79i+EBLQ87IVtANgbKnm8EQRb LiGpvR1uBlHpzcReEFmdB/ajAbgcPEorGvgRu48k2qzL8YYRejkGp/iWQex2/K3dkAHliOXt fKS9RALI9i1fxpvwMN9NHfztggDF83KoFg2mmG/s1A03JFXYchDAsRGERFhxayeW46yYetJv Ts8mvp1s8ZIsdnSSLcOO5IQHoWxS4McGRuGfX+gFZQ5Wj7l21sCpdrC64JFv7RE5Z+TffQhI Bt9GrL07GH7smMli5V5Kp07tCCYipLwcKhZC6kX8/uFg6IXpSGBpdFRf8t+RuhBITwTHAY0P kabuRN9ijrip46/f5tdd9tTEi7PDbn+IapJLlmuWOIPZdCXf/7E7zPT24fEFu94PL/cvT1H2 wMDdk+lHEdgFuBvNInVF0JxTjyuIWdx09o0Vs2T35+P9bpZ06V00yxpJ6GlMNtpGItbDbXCW cYjULX71ORFQIlkmJj5gLr2GTe3hV4bPbfEuGiBmojSia9Qd7FXkpsn9JvfpSejUC/GlYKu/ m6OhmZ5ktWb4Qc/RdbLbIT5keiMh8U00A/3ZJ2duyI6snQ8vJB7PP32KXl47SasMhKr4veOX 7f1uJGnIWUdBnkiYBLFn8YUu3JAY1jBvBHdcH0E/46cEI6hRqfsWhwLW3Aw561uQfVc2VS2Q OmGtysnXhEW63tFkoTZJjS8U4e+E1RCWatYaDzfRKB91dM1fosI/CpIZFrlJh08RQwZ2Dt0a eb/M85dXLIy+//7yepg9eGPR14IcjZF6jOlmtHZTA0XLgOTl+evTbrZ/+44z9oSJKhbhs6Ew soV1XMBuIbMxPbw7lxUrzfIGQcqmVTI/P4M46QhNJq3wvmlE0+U7l2BAxusvpJ7L7Mg4Xp6d np2MToSNhHPx/4xdWZPjtq7+K366lVSdOWPJm/yQB5mSbY21RZSXnheX0+1kutJbdfeczPz7 C5CURFKgOg+zGIC4LyAIfEx3CDDTr6w/HqdEZujwwvf5UE12PAq/fkWXEVeJdnw5W3aJi75Z D3QkTJJmIHc2vGQDOlCcIg4ZtVtyhpyuSsckXxV5pIha1wifTzuVLpsMEXsYc2QTpomd4iHl iUP6kIRmoTLGTcKq1v+PkZ2RcY8LtGqN05O2SsEXeUyrPsDbJpGbR2+BwCF7EOhaAHNHVN4E zQxePXy/vj8/v39zdi18I31hjWqzzPhd1SZ/y5JVbaycGlF8/ZNiVDqwg2Ts0W/msU87b6eU KKTNS6s/GlZYbyfUIUgTMWEXNMbkmJD4IJpIz2PYKJVzPDQivzPKNK5ah2X+eHLqtWYJKotx cFT0Nb1XSm5Upx7xTT2ho0sUO93HaNd2F7GmeuoAf6y8supAHeJXqn+MFI6ID7DTE8BuSOkY w2OShSd9OYGfavcV4Hy/BZohfL1LHLGOkqUCeGl3E1T0l9Z5eVkK91HQ360T7nIQxCdMaHCb fE1CDB3rfZ53VzE99byDA7y/VeRR0b+03EtECBmQQOQD2kqdlWvj6NbQzhlGEZBG2zCPQoz7 1UznlcwJltBMOJMItKWOvz6KixTdubEVhV1PhvRq2/4JdJxWQgOoadOR0e52pAXJBk0nTTFC WbsNStPiKAJItQsCrQWEJlclB4eDTqvqVQ6ICFRPutA3UqRFTSv3A0qjLoVXhRbGHQxe4wZD /j4nOsSVovEy0yICFTHLdMNd87UOU4d3ewL1M0LMq7XR2HHO4hZcp71OlvuLqdkXMJgZPcey Whsm8EPgThrKNRIhf3RYF1G2ZIgHyGhBvyIBjRVWC0VWBd2/wYTJJEqnAGGpXy9Pb/KOa5Re fho7I6awSnfQS9wsqvSst4oq3fIryrS81re9vPfrXGnBsYngdx22js4GgXOEt+1+Zip1oyxF QQfEAMuEF0FKG8EMfZ3B4aALnAal+nNVZJ/XD5e3b6Pbb/cvffVB9MA6Mdv9SxzFTI5Zgw7D +tyQjQJDCsLEVAhXa1fZcZSuQtDQj7DJbc+eWQ+L6w9yp3aTWXxHRCJRCEeIYV9y4rsGMFQ+ sSojaL5dSEF1BF027GAoF1Rh0Jj5aHPCLOJ11KfDoh/2qfs6SU0qDBWzAlVhEcKViH9Rh43s 8vKieWfhnbMcYJdbjN62xleRlVBsbFC8mOR2s6AXtcvgKMY8Ol4xh+qNArCPCxmngHQrcE2p FTtvTid7TEsfB3SoXcNpnEJNFc2SRYv5CVvLaM+EbRXRSDPmK79yBHiKltoF4+lpSIKzlX/u Fchujffrg5OdTqfjjSPcFTvDYTySPKdtTCwPJUJWRhG9c4pmES5NBwRqcWwqwsaFo9FozlQY 60XYjlrb+PXhz0/o4XK5f7rejUCof0Iyy56x2cwRA4ztmkKm7opvLa6+TNSRPXvQmlEXNcYV YOicHmmkuHElgEiQ6/mBxYSphh7nOlJmu0P5WFPbQBfdv/39qXj6xHAGum3CmAgM583EUZMc gZZixsyJ31DxJN/nELIr/T7ISEFyzHbPhtBF26+jGCG4HFbTVgoHoD3lBKMQazkMIaHfDiWR RL3lSdBBmytca4AsYcJ3hQhLN4euxZT7ZwtXQjQUIRshbIs9HGhhjI36d8U8r1b1sUpMB9pO DsaJe6cSIixcO6IPWwlYdicUQkUrgX/xJCOajLLzt8xtwpPZ2F0+UIZyywtZTIS0xNXp/+S/ /gjWs9Hj9fH59adr1ZAfOPYNdK83gyI0soA/mQqnHtCgHQg/IL5f0ettsSaytaNlJPSW+bZD Q3i0CCBsujpJKhy4kpA+ancfintMylWlk+B7AfXcK4coY99LTzE3pAdIww1PQbBYzqkPYdWk kNYbdl6I+naFyUvjhzoLZjCvwk3c+QL2bfsgbAY1KQwjw5KkYI3yfZriD9pQo4TW9H0zlNwV 0tJ8if6vnOOOk5QT/0Tv4QJHqfwdfR/5OaI36ybBKGTLOa01NSJ7V1B7I8DgRD6wgjdiqQVr JPetagXb9v0b+i7ejf643l6+v11HGFSEmDCgUgqXOfnJw/X2/Xqnz8+21VdufGHkF3yYz0/0 aaHhuxQDFoGidi53NYsOjoCdOhTBSue4ptU1aZL6cMhshytQ8YGxIFrokPV9yLP7t1vt1N8c SOOcw3KFr2JM0sPY184UYTTzZ6dzVOo4lRrRtF9E+yy7EUYJbfYmK9BuON2c5TbMXeAnfIOO 04xe8OtknQmDEHUnxvhy4vPp2DCpxjlLC46QUBiEYttwWrFteU5SMvqujPgyGPthqhksEp76 y/F4YvpTI82nTh5NQ9cgAoeTruEaxmrrLQIHfUHQRaGWY81DdZux+WRmnEIj7s0D2jl5lZXj YIad6LzLL7d7ym6NVwP8mKCKvubhchqMjXsmWnVmvtqluvkkKDBw4IOwOvverH8VHcclno56 F5CSDhPO164dFFEGoPbIWXiaB4tZ14GKvpywk7HlsNXCG/cGmHwo4frj8jZKnt7eX78/CmRg FYnyjnYpceuKGG24xt3ev+B/dch+OJ1rliBt4omZpE7a4cP79fUyWpebcPTn/evjP5D+6O75 n6eH58vdSL5Y0yUaooNtiMf+MjXu1KTN9pCZCrS6JcbjYpYwYQeU54b2dpgla4LcfbLFSAMX k6FfPZGgU/75pUV/4++X9+so6+KYf2EFz36lzjUx2zrcgk6pQHx0MtWrLWHpcJgAkTim9GmJ AKpD08gfUot4uF5gH3u7wins+VYMDGGh/Hx/d8U//33/8S4sJt+uDy+f75/+fB49P41w9xda qH4Bj3A2dUioH4LFjddFkLIxUB0kBVMgq9exS+pYpeVkuk206oO4+/7gS20DMciIFrsqENm0 qoqKU+oUyEHRhnUikBHB7a76qZc0WE1daonQbqlttC4T0AtoyAKpZpX5/Mf3v/68/2H3i7Lx 93uGghJstaUsmk+HNS5IG3TO3izFo69WuDdtDewlQeCE9bWisk7mPm0QaRWLrzYSQ08kjNl8 SBUVMmnizU6TYZksWkw/SqdOkpPjAl5v3+FU6ipZp/GwDB5cHeZEXWTyL0RmH4vQJuhW+Svr yXxY5IsAtxpWwDnz/A/6soTmHR6adeAtHNpDJ+J7w10tRD5QWHmwmHrDTVdGzB/D0EM3nH8n mMfH4SY6HHcOL45GIkkyOC9+IAN9+kET8JQtx/EHvVpXGSiPgyKHJAx8dvpg3tQsmLPxeHiu w9oSmSBCar/mSWNZ7WldyES4En2lq8IkkoDxlNILH2iaK35uvBwpKMrHWE9UZtRGvDtSttdy UXZVaIkm+wsoYn//Z/R+ebn+Z8SiT6Dv/UotoJwMYN9WkqkdfxpawQ0/niaZqr/58Qp2vjwy QM6bhHWnxYamG1NFJeH/eJWvv7wg6Gmx2ZjPlyGVo49jyG9yZrRK3einb1ZvovVKwv0/GvQ1 I8mJ+Jvi8JDb7wa09DRZwT/EB/hAnvkaomRVJZlWWhyl60inAwm6uHMWr7X0BtEqP/lSij5o NjInqLEDu3QV++4Emm6eHM8wLU9iOrhz2pacVs0EF9JYuuZ2IwCt5uaHtkuQxQ7ZcPHChC0G C4ACyw8Elq5tXU7qw2ANssPeAUUgF48SrQ608i/zRzs7jIQBiYq5MB8FP4by+Y4bLThYiuUO NhZQuYZlBvDMWpnhpgA94CMB3xLQ2fs13zLjeKCRndd+hsyQ+7kaz3VSOB7CEjNrz2GZcmim 6oxaHuzJpfiwaqw11wnxszBcEZxTFhnnde7IWK4xg9woO028pTcwV2LXKUsun3uBQC5BDdxi m8hhKGwW24FOSsqBsYH4SY7YmYYfeuQNudxYy9Be+HUHd0n5mpTnuCwFerCVOrI4+m6xemCq 8dqhlkvuTTabsADWVFoDFUK/i+GFtwO04qSEwvN6oLOR/8EWkZZDCURsspz9GFhzsCbLBW3O lDoQLycD1TxGC29JXazJ7G24eNmH2QerfZkFLiVRTt71cLNJY7abz7ZxypPCPUuNPX3I2UDW krLLCE7BIzngQ+MtmZa3110cW2okHl8Tdri4w13s2HajWvO9Wcsj6tifOU4oIthjUI+IKNtp HVYbDHWTj8F1hnCWnRPhJUDbyYGNT6uQ4dPILIV6/mh+gR6ZlOcTXuigd6YqjP0Z6R8rtTP1 gY5tuucWZJi0xcRxPPImy+nol/X96/UIf36lzB7rpIpt/+8e85wX3LH9hgw6vkBwSOE7Si+j cCpx+3sehOJqwQ1IomMzk8zKUNgFzfqpHnCGE0MiIu1MU3Z+yIwfdownkkAzQTHd7RmoVVWy foM/vXx/d572krw0MT4EoRf9ZzDXa4SpTI030yQHIzDlNapBlrDKO3SPfTQ5WVhXyWknHWdb b9AHhLc0AtPMjwoYv9ZtrcmBUR/uqaXUEuOsiuP8fPrNG/vTYZmb3xbzwBT5UtwQlY0PJHHV BbHLHum59Bgf7OKbVRHqT4U2FFB/y9ksCLqmtDhL6pt6t6LS+r32xgvjZkdj+d6cUh5aiXRH J6o8diiyGCFxRGZYs3A+NV03CZFg6lFVlwNJHxFdMbNg4tOmG0NmQrlQaRmcFpMZ1bYZ41SB ysrzPbKicLKoSTDFVqIo4xw3Ak5kx+viGB7DGzJpvs93K8rO0TXhMZ2OJ2OixCcxRh7JOeWc SjAHuALrVfSGcg7zMNXfUe8YE2M/7egRtai2bFasKk1jbembtb8j09tUCYWMYPDPul2i4+wR sj7T78RbnoCICVlNlIQnEWxLeaTH/rfMOtMBcLvkBJ4eIZ+ot8b4iqycYvsOjbKVO+J7qqRj ZiuC1s80NeOTuzohNGRRUVfEpszKwDTueBiiSzfIMYngB1m7r9s43+4pXawViVZLajCEWQw0 sipwWlsVmypc0weSbhzy2dijdeZWBtf+vQMyuxU6lY5X4rTeSXcwmmANpvOTk0wg5NKIFUqg 2LOt3KgGpGxcLrEZbS+vd+L6OflcjGzjoen8RTiqWRLi5zkJxlPfJsLfpkubJLM68NnCM50L BAd0hp3DCUgJMDgdU1qsZKfJCth2fhKXwSCp63UUtjhAQuShftmgJc503nvZIm1COB5tQOuG BgdC2K6JRFqBdGqENDfkONt74x09YlqhNZz8vF53s2+X18vtO8IH2l5Ctf5y30G3hxc5L9JY voUg0fO5LtkIaE4qxz4N5DoyQsVFhp6MyGvL4FzWN8Y7y4ey5hL+qUwxPA2t5Mx811FhkeOX ztESpvi8j4yWc1hP8+JrkVEbEKIaGKq2CLNyI45JNreCKKEqltddx9jJF0qUD/zr/eWhH9Oj aiEebGD68wWKEfimv1FL1N6pb0J5aDnDnVJn9LrS+Ioljq+4OZ0ael6d9yL8a0pxK3ztLYtb EbsXhVDzwN9QdwvBNaedYI3mofCejCLVfhAYqqWRBeoD9NlcF8uSvuNy/vz0CZlAEX0urrT6 93EyFWyQ1HLstlhNR7nrY740rxG1PrZT/+Jw7VNszljuuL9vJbx5whcn0rQlRaDLV3EVhWQB 1Pr8pQ43WNOhrJToR2Lo9mbLWAlVrDd6cdGHwSveRNGftlHsqnTY9iQbhuI5LZ0lE3Yqcjkp Swni0yytB6ZMGtrSKj1Omz7UVfgyS2CzzqOUfhLy2Dwn8tgjSRjYpMC1ieBa+AQdA197I8ji ST69aB3r4LAI6hK2FbJf2lIrZn4woneqyXJu7KRhWaYJc/mlFvlN2feoU6bLW/cGiheyAqxE X/rwtg7BuqYWKlJDnRqqT3a0MOibpGXcmnmyLlmwmMx/SGpnHuLMooDO0xsyGKcv6BhI68/m WquXDlUThtFGvsLjAgiu2Ub1g2Y2RKdIypakOOgWiaiQ+mDSWQlQ8rjIaW6+PxS1qe4jO3e8 2ok8kZeT22TnFGAVDQPdlIrXk8nX0nd4Nccps025mqpj6YknOITeWLjY0oDkM8KS59uPkmAD NU83aAYpoIoDv/mwNpLl0zYWDd+kMAxaQMz2p0Zfyb4/vN+/PFx/wIzAcolwNapw+JE1MBtq ycLlbOppxkGD8aP/BVTKWOaArAAGbBBvTYJniBD92LVh+PDX8+v9+7fHN7OkYYrA6lZDILFk aztbSe772GD67dkKnW27NlHLyQjKA3Q3rLeVT+K5HM9a/pw2c7V8h7+e4GfRYkb7Lil24DkO xWKaBq4TLDK544ECycwcR1tgovcaPZXE7JdvETj5wmtr6W4z4M8d7n6KvZzTtgJku7YtxStN 2AHp941erY4O5iwj3Lhxpv98e78+jv5APAcVmf3LIwyah5+j6+Mf17u7693os5L6BColOpP+ ag5ohhj6diAlMqKYJ5tcXtE77qFRLN74Y3cnYcKOKbeLszKNzIldSMumVRKYQ8O+CELoFDrC RWV/ZXVsYGki9YR4Nqdey8Y/YA9/Aq0bZD7LmXi5u7y8u2dglBQIS733yVswFEhz36xqF7Zn VkMFtKVoS3EkVhWrol7vv349FzxZm8nWYcHPsX5NJKhJfqOQ86yxWKJXtnXuFbUr3r/JZVu1 gDbG9Nor7SNklBVQtHy9X5lF5Knx4kxLUgEZdifJyD07HIMQweX2AxFrx+xqQTrEKyyWTsvh hOpX8v6WBkQjhrHkzqfS87oU4j+75G4f7mVoCZHsGbRIBPDZyQerrEwUM40Sh5KjCamBRp4q WiG1LrRF+wuhjC7vz6/9PasuoeDPt38TxYYqerMgUG+xq+TiJ/GCQbm9wUda8FbPCWz//jzC 2AoYkDAP7+4xtAImp8jt7b/6YDRzwrMcUT1hqjyGUDtp8oADGFYTStGNSZKgkIQsohksqj7E ODgRGq6Zq3D8mTqO+L5xW9RpXaiB1KNkyPLj5eUF1nOxpBIzUZYmi8gnp2X5j2Fp3BjomQ2v sELS9ZaQYKY3cNbHR3zdIodTMKN3XMH+2l+JSxhIn1S90Qg9WHdvPD3jcxoujBchtF54QUCZ G2QN62BhmFVFD5HDqGFNPO/Ua9M+/ki7ZYsqXH+8wIgnO1De4LorIC4bHY79nYDvrKLQm3Xs OkVdB7OFTeUnbzbu1Pl19FHpWXXDa2E3Io+qchSGy/FsZuWk9kczNakhuNJJy8lyOrHrEaZZ aE+nis3qWTCxqHXJ57NgTnQeMpYedb+t8307PXV7qj3L81FfDyjOQmBVBw5fWjla5UOuDh8v ORjSc1IMzNpyaEpXEZtYkRvSDo2Pi77+m9GMSKoTPg56SSAgYu9bY61oNRMNQtBrNg/v0z/3 6uyUXUC5NbM+egoYS1z9F3QLdkIR96dLqrdNkcDXS9JxvGNGMfStUxWXP1z+d7VLKlUf8Vgm XQQpwA1bW0vGgo0DJwOfsYoUpl4/U5TxKM8GM5W5I3l/4mBMPBdjos9xiwWLB6U961KL+ZhO eRE4GZ4ryyAeU4gSrcjqd39hGObUY7T7skz1d880qu2OVUah5BtrjNpvw4g1b8NSJjBQf9pv FU1J91pCpwcuuuegG8tuw7FeNbW42DKn06mfoGIoe1Mv1YZNaRp20WCT0D1RwlPpg1633sOC twn3m7ifOex73gJtpkTOikfdyjYiavnGB9xZP/HqNNMaEA1fG+j1hJeYrBE8rliQYwBVIDJs JHreNg0jLYOFv+jTlfJIZJWHG/p5wq4w3nS2WDgLulwMfw0SQb9A0J9Tb3aiyiRY5KKqS/gz oprIWExmJGMGbdpnwAF1Ml1QxRBb9XiwHELE9xb9Phfj7JzWzF9ONbi/5sOqXk5Bj/mpD9Hx Sc1Za9HAV+dre80QRPxbvEVuMUMdxHJ7zHQzrfiJT0XrVZZEdVjeJv2A4vzyDnozdXus8CRW Sb3f7Ku96dNgMWkrYSsWLaYetaoaAoF5k9dwMm/siE82ZahHNkwJbccyGUsHY+KRlY6ypT8d AvMIo3px8sZUqjVU1MnQhpPBmPsOxmJMtxmy6CNVK8PZwhX33cjsgjp2OCy1It74Q5l1mHmz rfOZ9Q64pExjBJfrNw5feWMK6qQ+lWQPRXzuCNvuJDyr9rZAnKawgmT90qiDCtHuyWwHyjtl 8WqbAo6Z49m6XxVx/vTXm35268VssphxKr+MeZNFMEGVYShPOI1mUT/LTTrzAk5UEBj+mGfE F6BehFRBgEF7VSm2OGGHeT/FbbKdexNiPiTwRbO8Ee08c5x0Gwk0C9rj0k5Enukt6hc29ftU GLqV5/vEEETI4nATEwyxP8yo0Qks2B+HBh9K+N6MTHXq+74rVX86tAgKifnY+fF8eDVAbWH+ /5RdW3PbuJL+K3ramq09UxGpG7Vb8wCBlIQxbyFISc6LSmNrZlzr2Nk42T3594sGSAoAu+mc h8R2f43Ghbg0gEb3dDk+rWimAI+I6vAsMSs2m2ONfB3w1bOcrdGxANCcsqeweBZjE7fmWK+w JlLQLFihOkPfm+smRKZwtdudTUN0osqSfBsGm4ybzj728bLlDOkP2QqnLoaNp6hIkypqhFEj ZFCCyTuWW4TmFq2w75Rmo02oYGStU9QZIWwRzoj3czbPfHS0aQ6kDsZaAmkIAOYhWr+85mbX LiTuMb1n5LUaBGi1AFq9s34rHrWpG5t1gWPt7kJu5d9GizU+3MsMj6vdpZX7OkDaSpGxvq/I PMDqOHbR2q+0WRKsZqtRnkQtgnN0Q2VxhEqVGxZZActjOMWLl0k+X2Vj3aZjWaPTsUE3s/V4 8dXavFiqve/QdeJAXqbmLlwd5UEYxVEwNqMypfBMAyK9VNv90cSqoaIQbSeRs3A6Pt0DC2HJ 17q+46v5sOfU+4xj/vDqrFQaO0FHpidF99z+2cioDujt/QcCIAwRL5t3FWDFt4yWaEj1jqMO wgBt4EMdhbOxQh4jpQMG8bB7A7AOEMVPAyEFIE2o6ciYN3TQ7HhdpVjpFUe6ihZo0AeXZ+m4 LLlBy3C136JZKyTR0KidRN+HwbxpsAUZstV30wA949eTuh1RuSUM1dQOAK/K8AwDXE2hgRw6 xi7a+q4ArzJJeT4KmWASbcYtE5WJ7YFWCEuio6rolzc/naQ9dUjTgvtRdgbp6FIhjKP1BIYN y3f6v3fz/Mlq/avVSbLGvFzAjQf0JYuWx1NGTACGSRb8HNeyE0xxzubT03s8ibFEHuM6go/8 uMCMCqTcgEN5KTZp76RQvr48PbxN5NPz08Pry2RzefjvL88X20GltOODgQhZOlEhtFQu9Ctl S/oQdSYIiC82nwHUhoanyqttaH3hjpwbCyHDPMAuuJ+QMr7QmLZ+hfJpY3cqb5cNn1pubITZ 04ZnbPBhNl9fL48Pr58nb1+uDxC/esKyDXO8VHHEUZc2rvzz+8uDjoVDRYvLtvHAbYCmDVzV WSCTs5W7SJWZvrAofS95diJWh9Fq6r0jAEQVf7Genk4u1RyXDspVgfkS6iMciqCvBQaJ9EFN 6Huy9RkWbv5As4/betoMER+gm0kNOoZdQIHDGud2xCK2jl5vh7Y1WHxJwTG9FkDFX6bOOW9a ctIQAzDcWuE2QUE7+jU0aFpKs3yisj0+MmyIYvud5Z/OPCtidBAAR2/756SLIu2ShEhj0IVf dk1eTtGoA9v4dvXhJQMridVyjYbG6OBoPnP7h7nUWfnl1mT0WLpH1ytE0joalKpeztC7GA12 ZwiuKOymHOhVUje+fLUfXKgejh/i60TDK38Xr+UglrHHsJiir+X71E4YSk3tzTNcSfmiXqJ7 HUBlwpGpRor5annCgGzh7g16IjVXa4a7+0j1Hm+Aw47PuvbZnBbT6WCSZZtZMB16kbbF3Evu 6pNArcFb0Gy2UBqC5Ph5L7C1xi9eYrg5jKgW801j4OIrmC6cOEjmvgxXizW08mbx7oINoZo7 Nbdyhk6V0LLI8anrAMvCMcKxqe6LhxZRU8vMOrnoNn3Y8thhrIkJD0eKYzmdj37gYxqEqxkq P81mi5FReHvzRrNkRMH0lEAa2umVtxKfipyNrJfHLJpPp36hFXUWnEiH8R3LYvoey3qNXRH2 ET7tfHsiqcPdOLbilKi6F2nNbPuAGwM8FWu0dVoum8y+pb3xwB5DbzFuXGhxGK+jiDgjt7ji xWyNG/NZTLn6ge8rLCajXo02QKu3fUbTa3VrNHmvpyHN0ipeBGKb87nIMiSRGfGd2TpEJyCP JcAEb1mu1Fq8ON7Dt54uZLqeTRcEtAxXAcMwmIFXaCE0EuK107Yd458BWPAapDWfOT58bhCm 5riomlhHswWNZDlf471Hg6jHH5fHKEc4tAiJgkdr+27Dg9a0QK0+YZBW7FCs1cL1lIyJNfes FBSt0d4MShveHf0l10K2zacksK3LLOyY4Z/xIy8y/Shh9DsMrF1vkAyzkrnn4C4oA+wE0uJZ ZNFquUJlD3SlG6YW30WwnFFYp4agWAgXM0h+RvMIZ3iH7bSV0dr0yguRta/CeGi0/Anx64AW 76gvDuapKhbmG43doH7R7hbbJBZMGyaat6O3U4PP18eny+Th9esV865n0nGW6cAiJjm+mmtG 48rpXB8wXoczFjsBxuE3Vr+wFQPz/RvoZSXj6icKVPGfYFJ/1BX4G8Nu7g4iToqz87LVkA7z 1JnXDRUC4dIhxQ2P0U0yAYHiKpbvUK+C8WEz9PFYw8lZGwYHTwLvtVnMylppLnZQdcDi+5zB wYnOefhaONM9AYuFp1tJB/p5ty2hgPS3h1J0Bp+dw8X+TNL0PojnmfEPEnTt9qGddYRlOkVf wR8uvU7YYuXMHqYPqY2gHRpJ18TQ3LdoHe02yPv0hE9/EJVVEWGYAmgsN/iJs5GtvpfQv2Gz h8l9z6o7pFBAxj0Lbc53SZJbixqQKlYlWZE7Lzt16ZX6hc7ztxZdOgGXHeB8qokT6baUjK1W 0yV2CtXJ2S4jRzHUZLNd6rrGMNIS4NE/J9us7a+TX2Q9+ePydn10XPrz+xKiZ/ZR6Yk+f3l5 eHp+vtwiQU5++fb9Rf38h+J8eXuFX57Ch39M/vz6+vLt+vL4Zr2b7abWTVwd9KtymaQJH0xn rK6ZdudvLq6+Pz69Th6vD6+POps+9NGbfv32+emfpttr5iqWPWtHOzw9Xl8JKki4OBm4+PXF pfLLZ4gyZapquavQoIkf7hGNnKfPqtj/awI06cg0Haxr98EwPbwqLlU1OJh2mCAE3fUZbh1e 4XX79fnL9avLIU3bT76rzwpBkt5eH84PpqyPXRApp43rJnd8V9yI8Oa3tG8obKyOWRSuB6ul BTonHS4YKDQg0XUUrQhQDyAqpQaJlFkdugfoFnaCaCcRhS2mQ52gw+YklvH5XCkhM0dtePum ehlE+Prl7fJNfcanb9d/vw2P/hu7rA/6led/TNScrrrEN3BmhCRSi8SvclwusNRqkFJynGWP yWAanrfDwQ9SeFsqJBNWS4Xmak36e8JUr3t6uLx8uFOK0uVFzUh9zh+4rlVcHxAZQsbjJb1x uVX+t59MGj/99fTt8mw3qRoqzz8mZgL7UKZpP5wS3vmP6MazDkSm27tjql9fn9/gka3iuD6/ fpm8XP+PbloT5xFp2t3Xy5e/4YoR0SnZDrPQPOzUHFlZr2xbgtZodmUjfwss5zMAmqiDSVVg a2dcWTau6o/zXSZb1x9D+nbTQT9saLsBv779jbALpgWLz2qUxP3i0o0POPBqp9hJG+fNmtlu 7aeEaI8Z8WG1IB7+dTx8v5pOsS17xyBFGriuizokP5V6IlpH+EE+8NUx4eoSwCoI8UMrDbKY coIDMMti9elIOC+aQ8JoXKyJMFAAZjvcrQZglMsNwCQ7MPTiWSfcJZnfiofsuBtpn13GKMNk gJsY149080hclW6rt6NidgHORVU18vxRdU+S5+OJzntT8D3u/8A0oPYU5n08O7m+vu+6fPz0 9uX58kNHc9S+uCebr0+Pf1laxFe1bk/++P7nn2ooxL5DrK3ji6IbT3p0Idmr0cqzGGywbbVU UeMYP2lWkA4KdEgkuom15Kp/W5GmVWL77W0BXpT3qlhsAOiwYJtUOP47WqzSodVOSQomcefN fY31PMUn7yWeMwBozgBQOav9VSJ2ees5fiTHopSO0DjZJlWldqfu6zJgV1MuOGcgGjhjcMdL xNKFD8D4HeXGBJKrtO0cbO3qFFCLVFewNk5Ah53p7853ErJxhW+ghwlVqjLDbcch4f0mqcIp egOtYOP4z06g2ofwNaDAfE7M8LD0EBOZgnr/3sQnDOLOvMDJ7CBiYgKEHikOJCZWRHxKhaVJ NF2s8KVAdwDydTVkSq8T0Jj1PbXIGJSCJL4rB2QwyzuoILsEtXRAuyaFGm+CnGTu7iv8Gk5h M2qJhSyLIi4Ksn8carVBJitaQ+T4HF9HdD/FfY7qzk8K5azK1PxKwdpHI9El/bt1TZO8oavv rZBWb9yo1fVUzxfTqT/WzOUdMZckqifmRZZ4ibJN5EcrteanSulzcp8k7uzLmuJ8F6ztkyKL OkWpw+qvAvyxcTslnlMeD09BgchTJuFV2kHwxEX6OLOfh+K8VH1ZbhxIjNgBj7l5GJD7yz9E rH7MNy40i9bz4Hw0sR0GsGR7pn33Y8LjMorQayePx41TYYHkran1JbLZcjZlWNk0tMaaJC2j hfvy2aoweB2t8CnFqnd7X/IOW3cPMd4Gg4tf67uXhKGAVZnDIpyuUiLSbs+2iZfBFH/loBZC WTNU0dnHmehWcf768vb6rMOja92xPcgbeIyCrR4feh1WZPXbWRZbCLMDx/dQOExb1XtU7vum dsjqZ9pkufwtmuJ4VRzB/Wk/2iuWJZtmq5Qly5suDarBWCvFDrz3ZqxyQhlg3FVRD2yQu7Ff 7BzNDP6G94jN6Qynu+gHsXhoJcVi4mlThyH6crtocvvRAfwJ4U8HNxUucgbH4ikT2KWFdATm 8blzZmaRSp65hP0xTkqXVLFjptQelyiTj43SoFznaC1gvjBeIig57Psti+YcbmxO6iMpyJem ygdkQhagplI/BoUwpfMTO1z7inKjC6h7p+OVl520A2H52yz0cjaLDURPPuPx33XB+4i6FrGL 3A6gF2LVQUVeY6NRl3kQoq4jdulHWvJUNbnvWkWnz9TebqcGkktuuwW0tf/VijKdQfQHwMjW V0zzd5nkhh2TUY42EoPPY7fcCbLxy4gEJXPaQ/jNyOIgioinWQCnkvIX1sJzKui7wcViviBe DAMuxZ4wGdYwHcP9Buu9IOFxG5iaKApGSqhgygtACxOeXDV8JN4SA/apns2IXQrgmzpa4Tou oJxNgyk+82o4E2QQVuiFp3sqxqxOLedhRH8VBS+pMLx5PBbMvocXtPGj5qlPW7r0MatSNvJR dvo9IQmn7H40uRFPvAfuxNOwEU/jalklAlHqJYHGEr4vZrgdAMAQ2ITwUXqDR9rcMMS/vyuB /vKdCJpjLIKMhY8IyGUwW9Efz+AjGchgPaMHHcBLGh7EtnFX1ljSkxGA9CykdlaBt6sb4iOd Sj+eiE50u3QMdBHuimoXhCNlSIuU7pzpaTlfzonzOqM8JFLtowkXP0YbIsNnKDjPQsJNuFm5 TnviTSYs2qKsRUw8rAI8S6gAuQZd0zlrlNhpGaWMcOWtQSFXU8rRBeBFLvhBbEbadey8RusB gkXhyGzd4u+skvqApJD07HE4hSHdCPfZ1luO9JZtH/+qr/adp7Z6rDDTYQk9BXC1BdAvMFUb fkp+W85t3MSucwnG3N5V54HcsMB72N0B8hRS6jzgnAk20AF7wIQ+G0neyCAMUyz9citQs/8O 34stsyP3aWWBx6HjT7BjhmuNJZZLWWB3IRa6j4fS6iJPWsfiHnJglWCnwTaLC+ap+6ey4Hdu iF/NG+vvw7cjYwH1T6Q3hrAPdo7n+v3isM+JeHgmsPfcnYn45q2wrpJ8V2MWR4oNwrpZCZs9 esME8jy3yxLeY16edXGQmwZIweZqj0/ke2a8aiybjZ503m49alm6Bx09UaCReQBtYGwNGiRJ 74hw8waui1LlTjLA1TrqjdKAQv3lxBbVZLUHZWQ51ZYxFnfJvXXlpBOVYRCEHs0YbrlNo77e rsgreD5uXeF3tEFLJnCzv/WbBYyziIA+BkbDBwHySRXdzWGXZBtRxX4r7LaE42wA90VKxQHT aetlNKMaUBVAR3B02+ruPnEJDU+LneB+sY4s9Z6W2PneV52Rg5NIcIYe9QNWH0W+tx18mTLm UqgBaEfiAXrKB85PNTnJiwO+TGlYVWRkVOnrGB141s0sE/B2uNjWHrmA2FnJoNvquIG6ZYl8 crVq71xZalZN7lxSyXJ4o58WbpewyN54s9MmuapH7hW4TGoGvtTdJi7V2Et57LEaItylY8y3 o1scBnk4kMTeMCxTBvbSStOR/tfU55u47gJwVXDOsEUWQDVxmBZ1kkiWySbH3npp1MxB/SKS 3yMjXpZJAuYC2HGUxuskSSH6WjKoj8q6TFHHSLo6mfAT7CC0KpNUoDIQmSmt+ffifkRuLQ6F +3HVuJWqEn5u9V6NNOxQ1YBVI2vjgNoxIrfoY/P/kXHUylxjQmRFnbj94iRUF3ZJn5KqgIre atNRBrP1p/tYLYiFN5sYDy3nfbMZfBqDcFUXeAij/xooD+BZHFUgjF42GEJOC7c8cXIYyNVB mzC5OsCUsIYSCCn2XJzBhiFNWosMzxbdv/rTWqn2DeXacOtoFXsmz3vuZuGx5bmaSngCAcTb a8Deyt61uoX2ef0CTiPe3LbpvMGAPYZwnXhomHxJYFe73p2PezWoUyPBgpq0FOeN3S+0Kbob oA1IR90UG7bFPy2ExOK3kFix7/FCp16uTtOpbjJP+Am+i6KjQwAYEoTBruCpCYPpvsRkgzfo YHkaFQ88s2U4ksNWtaHKos3A71IkVR8MewXqMSkxX6Vu8tvlsiOjGW8PmUZBMCxVT1YVLvye ZECO75f1u4WILZeL9Wok37ZWbq5A1H7TYaPc9X7oNMb6bMKfL29vQx8pevzY1016d2WCbrrE Yzxonzobxs3O1Tz5nxNd1bqowCbl8foFbGnBol1yKSZ/fP822aR3MFLPMp58vvzorOYuz2+v kz+uk5fr9fH6+F8TCHxjS9pfn79oE97P8Hzr6eXPV7ciLZ9fzJY88k7J5oK9BaWxOtJYzbaM 6lsd11YtkLzw2rcDhYRdMY6p31mNQzKOq+maxuwHtDb2e5OVcl8QUlmqdriMaju1saYURpvt jlUZKaN7naIaDo2YZfOq3ei52SwheLNT2ob1kQqhc4vPl7+eXv6y3ms4OWcxx32qaBCUZkep VVRReg9kDe1wm3/sDBRCeGQyoHdvp3PVgzV2zeluQEGYyPYcOxbvEiJHwxHDM/+quDlZKp8v 39So+TzZPX+/TtLLD/2SxCyPeobImBpRj1fnDZyeB0ShPnyKbYp1Rkc+82sBNL3ckbXQHKP1 1Byj9dQc79TTLI7dMztvuYf0ZtFwqSFSn3BQWmPyf3n86/rtQ/z98vzrVzDsgCacfL3+z/en r1ejahiWTmuCVwZqcrvq2F+PgwKFoHyIUu2c3LONHu7rO9ZwIemg5iaHcCbRM0Cg9js1PKRM YMvh3nprFWMvlO6YYL4eu3XeidFhEQeq3A0A/2WVZ/ViM5geMd4CHS/aVv2sob8GuhQ2Uq5C f0bWgaoHk5oJX01blllM7WMsROqZiYqD+0QcrO5mSqtCMXPchUJ8P7ODxlqIVk/3yWBhMSi8 UVbKAE/SxI/OaEsvlYaGWRXaPO1Mn0WEkCQrE6oPdoHBazBjsI/ELfCgVKuKkC1Khhkd2BwV KjRR/Wuk4h2s9qvvlDwKQvu9v91dtKEpConySFWowV4lWAxwxliyHA6pUdEtPoads7Ia6Kou RyMZYQ5AMeMnIyQ3fohAspMqxIA5WHsb0CFH5L+XRoQc32+gYP3x+C/UQrG/05cs5rkuwHss KceZUinw719shJqhOD4lZLw+N1Rn1obHOFLIlZlHsQbQaLDAonBS7BEaicRmOjXuxY+F5eyQ scFWuwXLNJyhnrUtnqIWy2gRocI/ctaccEStQXCqgYKy5GV08hX1FmNbfD0AQLVaHCcx+pGl SKqKHUWlZnApcRH32aZIUajGO4h+GPI743coelLr12B70y42R2LKMaFHcSjLRZ7gnxGScSLd CQ7Ezhk1dR+F3G+KHDvYt9tGNsFgO9Z+y3qgGXaLpq9N9VqGe/SEXKKBlCQTaFSRFgu9xZ/F Td2chkU5SHI5rUSx8GuVJruibmM1OZJS8tDBsWDXSmC7xvP7FV/OfEw77/U0wLi7uXCy1Ot8 kqLPtnSV4S4wVopdyu69byOk+nHY+YteRwb7Yn/KRkNqAqCU3pwnB7GptBs2t+TFkVWqHQfN BQcu1MfbQ4ArfSKzFae68faUSlOFy4ntYM2/V5yUdpV80g128qbjvRQcfpktpoP9GNwDgMmX fmRPFpbvWfH/lD1Zc+M4zn/FtU8zVds7PhP7YR5kibbV1hVRcpx+UWXSnrSrO3YqR+1kf/0H 8JBICnT39zCTNgBSPEEQxMG3zBnfoHI3Nr4sOC9yovge33adizoL1gmTVdhayBo1Gym5ZYpv H6/Hh/sf8pJKi+jFxniVzPJCVhqyeGc3QGbSRKVr77oxGTrisbxaUDDqCq8wynbXXWFmOXRx JJ3Z+oSc/gYmqBev+GMCq5UlWZ020qSeA103moeX4/O3wwuMZ6c0tgdTK1xrM++f+ELZh2nV pDujxT4Yk9GmEJnu+hUhbNJTIWPqhwVto4PoZRRiTV58kEaz2eSqJsOGIgGcK+PxtbN3FBDN qd2JFKi5P9LNOt/Srshiq67HwwsaDuFs4dfwJvESTtUi53Bds9u76mtr6+6GakEZ8mC3NEW6 avIl27uwzP3OqmEEiPVAvF5ydzOtmjIDHu4CU3Tc6rS2Fm7Vo+7U0rY2Qvxz5Vep4xucf9m4 hr7mJFYOowOA6siHC8aBsHdwfwTltu11rM5CFIhW3O1Zh8FPeLtgkInG+R5XOrLO2MfegcZ0 XGZaFaaBdBmmR1Mi/BEuqvPW9ItDGGGY424b2LsPeP829qmg1rhNmpS7LRSWDySQmleNgob0 uHxDm1JJXLRcF/0GI9TvKWLQqHHsffKWLcOANrWpb2lX9TT1RKNlKQdxmmoIvqLaVhD4S3pZ UrBmBf9vIz0BnJJ1BbmI/Etd5DT2ajp2vgBMeDq3Pb4F/LYMKNMegZNZ3t2aFLSXSVYgPXGw Zbsw3PS0WzQKOJu1GY2cLyHOTBPVAScE0AwGpoDz2bBfHJ1BHcowYTsMqRYnVF9newJ6ZQay FVA3uq0kNZ1hBcSMUGzClxGckePe/KiY+3w6Jt9fZJ+qyWwxcb7cC0MqoFUYYEjL3rxVSThb jEgnZ1mbG0a1XYezf5yO5NXYjAglixtR553VLd4h//pxPH3/bfS7ELfK9VLgoSnvmNWdsqEc /NbZsRgx1eRIoqye6tctrKl6OT4+UhsJzbPWtJ8U6m8xx0acxJXleBmMRncN3G7iJGHa85Io z6IgBMk/R3MHDlK9YV4lUESASoQTNZVVKFy8zFh6ABI8g+RJIIMp6w0SHdR7dQ2kuGdsR/oD 2T+MaWMfxBUYvA5OwbgkVcVAEQGHVBRuxQHzSKEYAJCVYe4J0yA+HMbUm4FFAzyPlKaxeFnb 7pgITFdXpPcqNL1Z3hXIolWa7G4foMG8EZHSgIqBVFHvXt4wqF6fm6twOb6nJoVeogE86dWr CIQNutkbBU+d+O7Kkufh5fx6/vttsPl4Prx82g0e3w+vb32bJF4Faxk9pa2XJ3FIO6wCA2em F6387d76Wqi0EoK7ljDrb7bLP8fD6fwCGTAgk3LYtUkRpzEPKbdHly7mwa+Q4fr6JTI0fb/g bqnoxGVFrZKnXi1RtZh7HHIURSaqwLQKF78RmSpUC7wKeOVB8Xht2xoo7C7dzocejxJFMh/P Zg2n3xsUyVb+dUIAKZoiyJhxiRI/tTr7z6EDLnPUAvw56z4hEeEGY+HCGcO5L3RLWfHZeDjv 7YUY9sfrmzJ9aLenjOL38HCAa/756fBmQk/3P86PIs6dCqD3cD5BMTvabBBdXw0NjaP83Qh/ Duwe7GfRbbNKXd9fx09fjy8HmZGJrry6noyurCNJgtyUCSpI5/P9A9R8ejj8QrNHZhJF8Xts deN6eqXbHYlWtoEE+cfp7dvh9diLMPj4Aezm4fx8GKiYqJogO7z99/zyXQzAx/8OL/8exE/P h6+ipSHZPJBy2miSyfHx25tRpeZQOo8kT8aL4YjIVisxY+tFtQLYbE69XiDmn+t/dKfT+8fT 4U0uCv/HMRmllffHQajMImqdvcG6GxxOh5fHj4GoGFdjHJodZ9fz2dScGAEwqykPr+cfKE75 5lhGaVSC0ODTQEbX/HE+WeYqsQhbjK/s/TAHLZGMTTPzRnzar/suYCDD3X9/f8Y2iegdr8+H w8M3yyGsYMG2pi4jQlMKMhRjWcPvsgDfL7gwmS3T2IoxrI4sGYa014bg9PXlfDQsRm5jESQy Sq/HZsYKC9zc5Jbdho3cCjNBos02mfhlXKRsbG26sSUVk4ipdVmL1hklG0broDBbt4ZbZLEO MGwdfV2lraz28ysjvnZr+dsxmJCVm4gWAwNew00AJoX0EYlYkiis1R0E8zSf+0Jgr+rPccVr ouIeiUibSXd3U0g7DB9SP/GReDhdAuGYeakReAXZFkHUy3nm7vwoKKwhlRc7uBglOf3SLXKQ 3qa0ZIiG/VVQXmxZzjfxMmiWVVOutnBbuUi1CQrPbQGbEaYFLWbLXsSeshiIq6xI707pnkGs DI258QRrEM9YzTqtacFENqj0WMWp2zS6UgAkYyFNVuxg7j0XC5VXMw2h1zQFr0t50Jf5BGTX qvIEsAs3JYhG7b4jXBh1iEL+fDz9OD98d2WUUAD5+f2FStsYJlteQjNBSDOOIoCyXeVCxc8m ycOtRblMopaym4EqxRGIPb7IGxm0BtbMTwjSqvZ4EmuKyhOjlKnAOMDvqT2HipxlbgXWKkJ6 rtBEuQyadJlTN8U4T9PazX6xxqP6+DAQyEFx/3h4E4GcOwNJdSA/nd8OGJ+9PzMYUb/CbAzG ew4Hvi98s5tSIWQ1z0+vPdkUXWR/4x+vb4enQX4ahN+Oz793mT8jIq9xne3jhpcepSemdCRH shCnwapkN7r36udgfYZvnCzpR6Fkwl4pQ+QZdCjILPcYk6xgJS59fDEmzw6DEl/ZebBj9inS EbQ5t3rbSDeZGJeufw3b+bzr2R6ZhU/7m5O+rrGZJgqFKvmsSMGacGmDt6t4JZA2WOmqWETW pbA2MFTZxzgOdEsy9lxwVMFlGozmxi1gmYZwK5DnaLdcTahKSKqP1c7JSmLtF0ohYVYaFezJ Z5btPvy8HQ1HdnLCNJg7se46zGI2G/Wz5Em4t4SZmVNEsrdT3lfb+YSMwIiYZSBM6v9/t8Lx YmRdrxYLQ7eM2dXm88ZJH49ptyZXnjR/YTGZjqkWZkF9LdXJCtCmfWpi5wMdZkenisRsklE4 nI8MizhfsnuR0RDg64Kuq8szqvopx+/p+QewLvNk+3Z4EgYM3L1j7b7MxaDJE/L4VVGIq3x4 fno6nyx3f70U5XawXeMctN4fT+YWSnl3cexuW5wX+rvuN9X6tgvROLVc7Xj8mIZDrCZ6Cc2G dlYVTA7oeVsH1HRKBX9Pr8YT0xQRlthsZOj2YVlNr8czPTsrtMU/nB4+2jv+//AKGUXczBIg ZRFxON6/nV/+iI6YVeCvd9RjaJri2/3r4VMChIevg+R8fh78BjVgdgL9hVfjC7+iSGi50Xpk vurI316+tL4rc5otSYyHK8XVeiL9jOTie386fj2+fRjN6U6mTeUJ3LyJwtFw6PF1BomI4lY8 vpa8yfg9HuqlE8OQv+H7yNPh/vX9ReZVeT8d34wB2qZ7M2dInO2atKivhphnUxxVpE7FvgQE ieeGEH2GkZ2QWd+CBFba0NIuBkXEFxPyNUugFmaqtuVmdG26LoXpZDyaGz1BwMTKpwWQiSfs MaCursjzYF2MgwLGNBgOzWMXVUWjsXUqmMebZ0AMkqIkBcvPPMAQTN2XQOYb4ltn975WlU5A 5LyoYNSoxhdQ2XiISFOc3E4mI8tyuAr5ZDqiHjgE5tpojuZQQi9mJ9kE0HQ2oZpR89loPjas G3dhlqj0ML+gPAu288W1sciD7XCxMFMiKpEjDdZ2wuNgDWuPZoHGTGAxVsGFC93XJ7RzbZqG k9mYNJFW/AFrkfyhx9UFCpVSGq0258OP46nXZ6qFcRYmcXa5hQa5VNOQMV8v6vqMVgtbkbIu Klp85Hd8xY1j0eLGz3C7OQFzb8VHUypxV4jk/pjO6f2FZN8g+Qw9u0xYx1KnWJGMRsZ6gd8T BHQiHZ9dmetH/rYPBYRNrntz6QSRMaH2iV7NpkNbJX1C7a2j6S5ezv8cn+yjQu/zOEI1Tgy3 v525AfeLWbdxqsPTMx6r9uB1A57sF8MremenxXB4Ze/fO05yEYEYW1e1rKKNYXYpQy98ihWZ 9g7wQ64h6yYOwLCkb+OIk0nJ6aoByXuVIcz7gtoRKC2Lp2Jh0jG3GD2CQZz1Vgu4RpkpuLyi QL8/aTHbFpFCLyZi9GVnaf2X8rAiY9TD8mOVkXWyG+lVameVSMNmFWyZE4LCwMKm3ln2iwi8 LXEZyjSRNibcQOPZTvOAYnM34O9/vQoNRLeW1Xu8sizuBmtzh6atzXiepcLGmpoDk6bmS8sQ ZhmmzRaTjSMCST0VaNWhWx5xbH+X5XwqbHQBTc9qR7cfjX+Fbjae/aS+CrCjsed4EkqNMKD1 t6nt2y0H/vCCjrniiQxuHUcQtPvv9mVgLbxqU2cRXrqSvu9t9yLSiWdZVOYxffok8TLbRXFK a5EyYAtkpO7K4AnwQ74s2SCe12XYmtMYVzChbRXmo44Gttq4m95Fr02r0xbKPZXB0rmg78Wv VTSLaQloDxZ8jsHnzL+Pj+/AwvFVt+dKjTTGhoNfTboumy932Y3GybqOL09CIdwLWMIiQ+6C H01uxslpMyPBBKXiwUhfU/AZplzWBiSMloFloxClMWmhCXDF3Z8sUIhOkEG4iTPWZHnWsFUM zEiG2bceQDCIRRMvV+jYkVFfWOf5OmG9PGnr8/nxx+HCYKhy0Nmwjd6vxg9fHgXXMvWWIbSW NbcYZ0tafRmmWvtqbFlQK0CzD6qq7NE1aGO7h1qSPoqzsC4dWzLATZoVdZACZmo5MyiA5wtT 5wsmhmVheee4iukiBs5u1tTr3v55GRmCCv5yrXyg3nQpBtW44LAYzjfAmIPZAoHUfHlo4SKS fJytLENToyo5DVQj9ZfaUp/NsfOUoKYI4d6hwDIofqMJsDFXe/l1oxaE3NQgqlOPr/SsIti2 D0QIbARaFb2/0M71iturGEPwj50B0rAmH5NRRVo89tjqnMRI/5s04Nskp6PDmHTkol9W7grR EGuAOsFAY8X6EZx4jfNHfr0lxkQAPMiATrhy0Xd4Se2PdCPxAYeVSL8aZHEix4uSwMa9BSJA OLIXS7Rsxy7nW9o9Kr3AfV+Q42iyHV2SZhQCK1SnAfmCg0MY7C3OQPIvfF+zvhoD/8bXSGl/ 2JbO8ipeGfwtcgGxBIjXIKuhgUQQbRTb0qQVADQeFT5/4o68orsnfBAUPW5Mx1pSInybUmKr kllPWjertGp21P1MYkyVPlYQVtaOwDzSKz71LKIao50awxw6bqT5jpVJcOeUVmZkD9+s1Itc Mnjz1iFBkj+Qi0HiN8Au83UZpNYEKaRvtDQ+X37GhDh2aDiBEi6ClhljC/XWapCYrZI9jj7B PeiPaBcJuaEnNsQ8X1xdDZ19/DlPYtI/50vciwATraiRjnL+xyqo/sgq+rsrwSNMNQeUsJjm ziXB39qYNswjVqCb2HRyTeHjHC976Hj2r+PreT6fLT6N/mWqBAQ/6N9MXg/vX8+Dv6kW9/LB CMDWfa0T0F2KYHKvYQzwOy4XvAnE3mAMwFg6Vdj1gRyaRCWjjKe3rMysLDVKVaHvTWlhz5cA XJQhJIUjGm7qNbCSpVm1AomWG6tY/NEng55dNGsWK/SOVyw1MEHUO0QUqCk9FkWr3txpDiyY uy3oapAyrLU48cZpJfyWwTntSKgt9OKoLZl76DN99BoPAT2Bjvk683mlhJsPF6IqHfbgQvPR Pqsbsp/Gowk6MEf6BJFkHG61MkeXW9pZEC2cOApbHC2KIhI9+VARCUexigDhH4Qvjt+IhCZf KB8tiRNJo/pFynrpiQkOwp7HZTQEZkpOEL+pA74xZ1xD5MGvrw6dZsxCR3HpM9xqCSMMFlg0 GFGa9M5xCaU/I/VJkwDNKEJPXuq2gO9K0hK4k9Iiki+exB8dATVx3Ze/GMxMA6ciQCTGiRQJ FPoELF0yOxJKN9xlsE4ZiDjqYMcKJu0ps3e2bRpnsJydS0Xa26Udeyh8O/gm2097+x2AV/7K yktfKtAX15MW5I7vfMVqL7vc5312JGA+QUOGQuxOUVbd5uXWZOyUEUdi8DD4oc9o62Q20Ppo b+Bot75m4q4ndPZFm+iazp9uEc1n1EOZQ2I9zTq42c+L+/tBp/R0SIzXHwcz9mImXszUi5nZ 82Rgri50YPGzDiwm/uKLn4/+YuIf/cX0p1+fXzsdBgkWV10z9/R2NDa9R1zUyEYFPIxjuv6R 22qNIC2dDPzEV5DmqyaFby1q/JXdeg2+pnuw8HRsQtcy8gy0+byJ8G0ez5vSXRICSh9KiE6D ELkjGRdI40MGp1voViwxcP+tS+rcaUnKPKhiOzBYi7sr4yQhn3w0yTpgSRzaHRVwuBlv7fFC cBxijKOoTx9ndVy5C6DtvJPOvkdU1eU25lRYXqSoq9Vc3wu3h5fT4cfg2/3D9+PpsbvlCB0I +oqukmDNXavh55fj6e374P70dfD16fD62A+fLgMaNEr+0meqdGpDKSthO5a0R8DUfFnMK106 Yo5bb6fGV6HX6VeK8Pz0DFe3T2/Hp8MAbvoP319FWx8k/MVoblejUOShdpY6IjP0yhBaESDE PK5BZSciUBRpzSupd6Ju/ZjkVlTy52g4nrYCSlXGBXAQfF5OrStHyYJIVAtIykQlq0UsVhG8 zbyQitDMt5n5ptqPJ7SBytGY1tFWS0IQ2FEWxwtbGlThxpLwHJwcFk9EYBGs/zYAqUv2vsiF Csq8zpvwXoNzfEi7ZcFWGP2CuOoYy8TiGk36a8sa8F7MktZo5/B0fvkYRIe/3h8frQUvBo3t K0zcYqsFZT2IF6m7vF+SyhzeL6oQMHDJyhNuyybE8OD+aoSzACVh2WQokvkrKcNarAB6d1mk 8poAp1+NM/TT76qdoLf2qF3mSb3UpNb4CoRPzbYJdkzPZMrSBFZCv1Ma420b1B5um5pLDYVT ekc/ACskppAj7RfUC4GwmQe+YXNridzE6w39imx0S7QNdZWrJL/t7UMaKYqLrYVdp3fwBmMU aHtcXO8DNFV9f5ascHN/ejQNd/JwWxcqQbcZzQNNJfpIi1ujC3BqEhauu9FPiZtdkNTszyFV MSYw+uWKXeJ+xSoH+QZNK6qA0xH9ZVlUSeQFaRhk4ttvWEjcpXldmZ/mMFGR91Ilsep50ITp FyqLTq5olkUt/3bWHn5/y1gRZ/STj3Y5cradNP1C4+eWRw5+e1UeW6//Hjy9vx3+OcA/Dm8P //nPf37vH6FlBadgxfZkhDu1NqFVtneW2kyyXL8zt7cSB7wivy0CMq+dpMRqG8GnLb3qjniA QQAmULfyyENpHJOL9ctCFliHRkmYXWFXCB15gyJuDwFqdMTnYYdhXEjxmmfoby15zFgNuA4E kmCLkiV7+wL/ddEK7QY72W7UERgLxAVuyem1JpHizSr2haCQNGHJMC9o7BgjS+exsCYPbjG7 gHS05RIIMlTBUFgjpRNe4OOAoOsJIM48dEIZEJs4WoGDRJ5nRMSxm04n7AzBUqwCkERQ00VN nh7HhpWlMLT9LAUyw+ZGvn20COvpME54QiYoQZQUg/T+sUqBtLXFPCg3ta/TggrfTyXr89Os cC+TLbDaTQitLkW3/FH75OT4SmDes/COduVus0jpg7aMMao1RykzL+4kWyVYkYeQEs+5ZLC6 gb0EVximVKBK52TXMfl+gl2XQbH5JZpV4bATKX+oS9RK8w8/srmNMcghM1mP/JBEp0I6BIIQ 0/7ZJPjMBStKNlRsM7eSUBWUtThMD43d943TRPnV0D5ESuSxrjuh8IQU9Bbzhz8V7jEOHQv7 g9ij19agHsL+5LqD6p1T33R2pmXlDYg1K4XxSg69etWqk6PPe6PKs0Dm/TEfBW2UvnPBKJFx 9ZYYHXmjY9mgfZz7BCrgmImtQh6sCngiebXksFYoQvPE6/VWp47S5hvW0zTUvGQXfGINPCm0 ePbJhS3S8Ts9u6p31Px59lBXh5rJKoBTp/AfOhggy3fodJuvWQJT3KRBSe8cA21owEwCXzus rckw+nAaFOJRwF1iWJkc7V6IRXnSv5+E6qY6vL7Js77j6duoou9rIqg2yidwCfBYMS07XgxS mK8D5RLNWCTDdGxryx1GxNdYei3J27KnbiknXk1bMdDYltj+DdtHdVp0R7mA4pmTocYmKWTI LRO5BWxlxwcQcKFDo4OcCHwJh99GRBvx7YldHDGRDm80WUwxDp+45lJ8oI4TuNjkIS8tsQGL oMzru/LIKdumTo/as9WBL4uVAzGMaa0KhOLQ8jtmqW+5ygEOKtjIKitzd2EL0O/fq5aQF/B1 ZL1/4m/6bU4KDsCkaJUARq1R4ri40NbGJYMFZXKntKKW1tGAiyCx5JdFPJwK11XjmqMQNBek 8zLH9Hb+ax0TeQbrBP24zHS9Sv7f6z2lwXkNq0+qiRxaNDFJahE/Vm8/Gb2ichJi4zy0fK9/ CmPIRlwLTXVXsGa4nw//r7Cr7Ykch8F/Bd0vYAaGm/uwH9I2ncnSN/oCM/OlmmW5BemA1QC6 5d9fnKSpU7t7EhLCj9s0wXVtx7FHP32KyWQMVoWYk6clj5rP3gVSFAMKw7GriTj4IzcD7gb+ ZC4t+MYVY+ITekT95FMnw0TZIWQyk+1QMal+4z54pU18qF+o3XhVzEUY7Ehay9d8DB9Ezke5 4PUDvamV0v8wpxXX96jqtGoy6tX5Ve6QQfNw/3GCs2pkh8K87GOMwLZ4BVtRA6BuERiN7Og7 1jVg0gCd3TEwGUeOYRI+N7kwUEu0McegjGLgLB2SAT9QgkSv4X5uG55BIGLCPoJernLPV0H3 PKLScpKXrOE58OyFKZSPvpSbabqPJ/aN2hQC3Gf+3cg5BaOVZqD3bvMeElO0b9h1/HkO9x6M qy1irGRC9MsfPvVgp11q44Kj76xo9kU86QBpaTvcFMSSqht0wBtkwB9Ij0+fP99fz+6hTenr 6ezx4Z+fuPOCZdbm70Z/M5GngslLSpciYYmUVdvIsWklOI/Qi7YCa2FEpKx14GB5Gsvo9wem WAUJSsw00QOinCx7u4YTGQcONW6nN3R0OpJLtma5+0Q1ZnfDRPAI1yZdLNd5lxGg6DKeSIeH V/imk50kiPmVMPPPLTK/BqJrt1ohMZfOGEXDdeBVWauTzjXrpMNAOfviJx/vj3CS+v74/vD9 TL7cg8zDMbF/n94fz8Tb2+v9k4GS4/uRyH4c53QghhZvhf5Znldltl9cnK8IQyNv1C2hSn2R /l75k5+RqfIBHTrf6KNEMR22pesQM3IgcREkR8vqO2b9Kz3M/PrvmHvrzwlUufdlO45vj3Mz yEU8VUX9Nhd0XjuY7DN5uNtc0AqsydMP7ZXRwer4YsmsmCHbU4XM9A08P38D6zXKuHdKg+3i PFEplRhWZ83KSp5ckmXKkxWlKS0+UOVQcW9SnSeLsBEixU0tEObC5YorbDPiF7j56CDhW7Gg Yq9fitUVR14tOM3ZburFTHOdQR1Vq7BolP2SmQ5CVOSEpAKraf1qTR8K6IXyojEBiy5SzL3q mP6v9Nf6LlXNdhYYSyUR+RO5zDK2X4jngISO+eublsu1QvAVc1nC+pQOTM1v+t5vxUEkVAxE 1ghOPCydXflBezJaU1JbQn8OK1nQB3L0vmnk0gwzfYRWCmbu2kZN+dSkkIE00ZjAdmI+3Qeq cdjaTHSxU9gI+J2QHyb1vKc4n6nswPUl92ZNsp8JuB1rHR9fvr8+nxUfz98eTkPJqSdchMu/ E43SPhJnZSV1BDGOouMRVulbxGpKsmaAxWziN+Igt/yq2lbW4PFB8IYzrHrOtB0A+zRUZDze OCNw/rE8K6zSVCA9yNrMMDg5nTtgd8yQ2tzPodmpDRxZR/uTAasuyhxP00UhG8K0gW9M2RHc rc7/6mNZw3YfJJz1ZrcSmSDVddz86bPqeNTGCGUQDQMPTHumlbRHEMxREBiBK7wTQzmuv409 92a6m7w9/XixxVtMZl2wF2uzv7FvXAe7KBRvkOtlgjnXt8jcGyiwBwLtxXkkne44OXpfl10b nvQZUBN9xtcBUYtGbIJ6ClJEOhxWBBjCTJMLrMOXMgPkjWKoEOGoZSZ2NpocS6zogeE2nY4x bDwlqm73WWmTAiE6C884M2/b3mMEXRKTOgi3HTzef3xJzMXmY4wkxcwy57PE7Jp0plcaHwvP NtO9YgdEqhC1C1ymX3zBoW+n4+nz7PT68f70gg3aWqjkKnCsI9XWEvpPBHsLY1R2xLkAv1kG gayOYZGbti7iat+ndZlPHEHMksliBi1k22uZw9ttAwRlISBgDYFs1VIcenSoMihoMUCzZKRv hu1saI1hz/dUmQr9tlj7V1pDB6TFVchBTWo9Ttv1uN0GWOUBhzbS6X6Go2v1JqP9OlSqCOGT 2R2LqO/mP93AEbH7ERpDVbCgbSLxTuKgY4IRYbuGEH4QLdeNx0tPkZQ5O2VtCvgDbSHVnrsK 6WB1wAcHrJMJ1dksaA6HkrkzULk7G/uC5b9k+XcHIE//Nm79lGaK/FSUV4mrS0IUdc7R2m2X RwSAXBx63yj+ikXHUWezaIe59ZuDCpK0PBBpYMki2QEHMRGAT6IF/OUMHa3E8K6alJqwUVot IY+vzMqgjCumwl3xCxrFyK2F9IVGgsxytP46r1h6lLPktEF0/fEoY6W1pFGntUC2HKgYrY5k PiXBvksfqCmTXIOX1NZj8OFftHg3SBcXWXjA1Gs2v3dq5C01B07hEdEA2QH6JyBCWSfhSZAk 4azHvFL2GKP7u1QJ7OQrrezR3NMS3B23xYRLBZR83oDhX/9aT+6w/hX2u2kgDy1jFZmfewOr J1TBLAuYJf2wWfYfLpsmYGCQAQA= --ew6BAiZeqk4r7MaW-- From eflorac@intellique.com Mon Jan 4 05:26:58 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 334C37F37 for ; Mon, 4 Jan 2016 05:26:58 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 1FA138F8033 for ; Mon, 4 Jan 2016 03:26:57 -0800 (PST) X-ASG-Debug-ID: 1451906815-04cb6c3032420500001-NocioJ Received: from mail1.g1.pair.com (mail1.g1.pair.com [66.39.3.162]) by cuda.sgi.com with ESMTP id GrB2B7GLSvga7p6J (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 03:26:55 -0800 (PST) X-Barracuda-Envelope-From: eflorac@intellique.com X-Barracuda-Apparent-Source-IP: 66.39.3.162 Received: from localhost (localhost [127.0.0.1]) by mail1.g1.pair.com (Postfix) with SMTP id C7BA02CA96; Mon, 4 Jan 2016 06:26:54 -0500 (EST) Received: from harpe.intellique.com (labo.djinux.com [82.225.196.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail1.g1.pair.com (Postfix) with ESMTPSA id DBE1C2CAB6; Mon, 4 Jan 2016 06:26:53 -0500 (EST) Date: Mon, 4 Jan 2016 12:26:56 +0100 From: Emmanuel Florac To: "zhao.mingyue@h3c.com" Cc: "'xfs@oss.sgi.com'" , Chenyehua Subject: Re: Facing problem when attempting to recover the file in the tgt shared directory without the remount operation Message-ID: <20160104122656.4091b5a2@harpe.intellique.com> X-ASG-Orig-Subj: Re: Facing problem when attempting to recover the file in the tgt shared directory without the remount operation In-Reply-To: <1CB94550540AE44E9CAAE8CD5AF40BF846D24DC6@H3CMLB14-EX.srv.huawei-3com.com> References: <1CB94550540AE44E9CAAE8CD5AF40BF846D24DC6@H3CMLB14-EX.srv.huawei-3com.com> Organization: Intellique X-Mailer: Claws Mail 3.13.1 (GTK+ 2.24.20; i486-slackware-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Barracuda-Connect: mail1.g1.pair.com[66.39.3.162] X-Barracuda-Start-Time: 1451906815 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25816 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header Le Sun, 6 Dec 2015 06:08:44 +0000 "zhao.mingyue@h3c.com" =C3=A9crivait: > My question: > Are there any methods to show the updated file information in the > shared directory without using the remount operation, thanks! Using XFS, probably not. Maybe using some form of cluster filesystem (like GFS2, OCFS2, etc) and some SCSI magic but I doubt it. I'm pretty sure you must use CEPH as a filesystem for these sort of things to work as you intend.=20 --=20 ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | | +33 1 78 94 84 02 ------------------------------------------------------------------------ From andrea.gelmini@gmail.com Mon Jan 4 08:12:55 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 1B9D37F37 for ; Mon, 4 Jan 2016 08:12:55 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id B46AF304053 for ; Mon, 4 Jan 2016 06:12:51 -0800 (PST) X-ASG-Debug-ID: 1451916764-04bdf0264e44b2b0001-NocioJ Received: from mail-wm0-f44.google.com (mail-wm0-f44.google.com [74.125.82.44]) by cuda.sgi.com with ESMTP id tFzlMwXhZE5kiiZT (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 04 Jan 2016 06:12:44 -0800 (PST) X-Barracuda-Envelope-From: andrea.gelmini@gmail.com X-Barracuda-Apparent-Source-IP: 74.125.82.44 Received: by mail-wm0-f44.google.com with SMTP id f206so141748268wmf.0 for ; Mon, 04 Jan 2016 06:12:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=bxKk3BpTPTHsZV9oogycKlpVPNS3r7D1mcwgB2C7pbw=; b=AU02oHNKray8FZg2ZTHa06hDR9tuA9GwnuIMR1n3C7oxIAQfH9cFFXwHuW8i7LEPSd J+FuFESDG7rEKEXX7dKuY/Ge7U2xSQnJ0DugRa8306K9+l+PBy53cf7O3jIVVSehuhzQ hRNwbQxcVVTHd8yXFJjm3LEl+SJcQhVxUcOyIKCS7NOtr/E8SgynbDcaRj3tE6hxTnS7 jTu5SsI9glsPweek7EWQYIZRpeGRdCtp96wl8CLTRdT6I5txoHF8S3cJkoaoe3gFnO1I l4H5QATN2ALFJbs6vTWdy08vJz7Wjs1rKXVP725hiybT4/jdC/QU+Ktj75LtrqnJ6UsM N0sg== X-Received: by 10.28.187.67 with SMTP id l64mr77822037wmf.39.1451916763677; Mon, 04 Jan 2016 06:12:43 -0800 (PST) Received: from glen ([195.206.31.238]) by smtp.gmail.com with ESMTPSA id o132sm73180728wmb.7.2016.01.04.06.12.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Jan 2016 06:12:41 -0800 (PST) Sender: Andrea Gelmini Date: Mon, 4 Jan 2016 15:12:39 +0100 From: Andrea Gelmini To: Dave Chinner Cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 Message-ID: <20160104141239.GA7054@glen> X-ASG-Orig-Subj: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 References: <20151214180048.GA15690@glen> <20151214195422.GM26718@dastard> <20151214201526.GA25152@glen> <20151214212220.GO26718@dastard> <20151215091145.GA19282@glen> <20160103204758.GW19802@dastard> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="jho1yZJdad60DJr+" Content-Disposition: inline In-Reply-To: <20160103204758.GW19802@dastard> User-Agent: Mutt/1.5.23 (2014-03-12) X-Barracuda-Connect: mail-wm0-f44.google.com[74.125.82.44] X-Barracuda-Start-Time: 1451916764 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Barracuda-BRTS-Status: 1 X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-Spam-Score: 2.60 X-Barracuda-Spam-Status: No, SCORE=2.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0249, DKIM_SIGNED, DKIM_VERIFIED, MARKETING_SUBJECT X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25819 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 2.00 BSF_SC0_MV0249 Custom rule MV0249 --jho1yZJdad60DJr+ Content-Type: multipart/mixed; boundary="OgqxwSJOaUobr8KG" Content-Disposition: inline --OgqxwSJOaUobr8KG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jan 04, 2016 at 07:47:58AM +1100, Dave Chinner wrote: > > Maybe, in the meanwhile, you can do something with my files. You can fi= nd 'em here: > > http://mail.gelma.net/xfs_kasan >=20 > Any update on this problem, Andrea? Hi Dave, and thanks a lot for your interest. So, to make long story short. Recompiled kernel with debug info and all the rest. Run it. Then started a flood of this kind: Dec 15 12:12:24 glen kernel: [ 5326.351571] BUG: KASAN: use-after-free in _= _check_element+0x1e0/0x200 at addr ffff88004a201ff5 Dec 15 12:12:24 glen kernel: [ 5326.351574] Read of size 1 by task kworker/= u8:2/10221 Dec 15 12:12:24 glen kernel: [ 5326.351578] page:ffffea0001288040 count:1 m= apcount:0 mapping: (null) index:0x0 Dec 15 12:12:24 glen kernel: [ 5326.351580] flags: 0x4000000000000000() Dec 15 12:12:24 glen kernel: [ 5326.351583] page dumped because: kasan: bad= access detected Dec 15 12:12:24 glen kernel: [ 5326.351587] CPU: 1 PID: 10221 Comm: kworker= /u8:2 Tainted: G B 4.4.0-rc5KASan #1 Dec 15 12:12:24 glen kernel: [ 5326.351590] Hardware name: LENOVO 2356LRG/2= 356LRG, BIOS G7ETA4WW (2.64 ) 10/08/2015 Dec 15 12:12:24 glen kernel: [ 5326.351594] Workqueue: kcryptd kcryptd_crypt Dec 15 12:12:24 glen kernel: [ 5326.351596] ffff88004a201ff5 ffff8801086bf= a10 ffffffff819d2e3a 00000000ffffff6b Dec 15 12:12:24 glen kernel: [ 5326.351601] ffff8801086bfa98 ffffffff813f4= b61 0000000000000010 dffffc0000000000 Dec 15 12:12:24 glen kernel: [ 5326.351606] 0000000000000046 ffffed0009440= 3fe 00000000813f42cd 0000000000000000 Dec 15 12:12:24 glen kernel: [ 5326.351610] Call Trace: Dec 15 12:12:24 glen kernel: [ 5326.351614] [] dump_stac= k+0x4e/0x84 Dec 15 12:12:24 glen kernel: [ 5326.351619] [] kasan_rep= ort_error+0x511/0x540 Dec 15 12:12:24 glen kernel: [ 5326.351623] [] __asan_re= port_load1_noabort+0x3e/0x40 Dec 15 12:12:24 glen kernel: [ 5326.351628] [] ? __check= _element+0x1e0/0x200 Dec 15 12:12:24 glen kernel: [ 5326.351632] [] __check_e= lement+0x1e0/0x200 Dec 15 12:12:24 glen kernel: [ 5326.351636] [] remove_el= ement+0x206/0x430 Dec 15 12:12:24 glen kernel: [ 5326.351640] [] mempool_a= lloc+0x155/0x2a0 Dec 15 12:12:24 glen kernel: [ 5326.351644] [] ? memset+= 0x28/0x30 Dec 15 12:12:24 glen kernel: [ 5326.351648] [] ? remove_= element+0x430/0x430 Dec 15 12:12:24 glen kernel: [ 5326.351652] [] ? bvec_al= loc+0x250/0x250 Dec 15 12:12:24 glen kernel: [ 5326.351656] [] ? set_tsc= _mode+0x60/0x60 Dec 15 12:12:24 glen kernel: [ 5326.351661] [] kcryptd_c= rypt+0x5dd/0xea0 Dec 15 12:12:24 glen kernel: [ 5326.351667] [] process_o= ne_work+0x48a/0x1160 Dec 15 12:12:24 glen kernel: [ 5326.351671] [] worker_th= read+0xd4/0x1170 Dec 15 12:12:24 glen kernel: [ 5326.351676] [] ? process= _one_work+0x1160/0x1160 Dec 15 12:12:24 glen kernel: [ 5326.351681] [] kthread+0= x1c0/0x260 Dec 15 12:12:24 glen kernel: [ 5326.351686] [] ? kthread= _worker_fn+0x560/0x560 Dec 15 12:12:24 glen kernel: [ 5326.351691] [] ? kthread= _worker_fn+0x560/0x560 Dec 15 12:12:24 glen kernel: [ 5326.351696] [] ret_from_= fork+0x3f/0x70 Dec 15 12:12:24 glen kernel: [ 5326.351700] [] ? kthread= _worker_fn+0x560/0x560 Dec 15 12:12:24 glen kernel: [ 5326.351703] Memory state around the buggy a= ddress: Dec 15 12:12:24 glen kernel: [ 5326.351707] ffff88004a201e80: ff ff ff ff = ff ff ff ff ff ff ff ff ff ff ff ff Dec 15 12:12:24 glen kernel: [ 5326.351711] ffff88004a201f00: ff ff ff ff = ff ff ff ff ff ff ff ff ff ff ff ff Dec 15 12:12:24 glen kernel: [ 5326.351715] >ffff88004a201f80: ff ff ff ff = ff ff ff ff ff ff ff ff ff ff ff ff Dec 15 12:12:24 glen kernel: [ 5326.351717] = ^ Dec 15 12:12:24 glen kernel: [ 5326.351721] ffff88004a202000: 00 00 00 00 = 00 00 00 00 00 00 00 00 00 00 00 00 Dec 15 12:12:24 glen kernel: [ 5326.351725] ffff88004a202080: 00 00 00 00 = 00 00 00 00 00 00 00 00 00 00 00 00 Dec 15 12:12:24 glen kernel: [ 5326.351727] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D Dec 15 12:12:24 glen kernel: [ 5326.351730] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D Everytime it happened (usually when writing) I had a little stall of the= system. After a few hours it was impossible to work this way, so I got back to an Ubuntu vanilla kernel. = (I guess it's related to my luks partition). Anyway, now I compile rc8 and try it again. In attachment you can find my .config. If you please can give it a look and tell me if it's good for you, about= info you could need after. Thanks again, Andrea --OgqxwSJOaUobr8KG Content-Type: application/gzip Content-Disposition: attachment; filename="config.gz" Content-Transfer-Encoding: base64 H4sICGZ9ilYCA2NvbmZpZwCMPEtz5DbO9/yKLmcPu4fJjD0er7/a8oGiqG6mJVERqXa3L6qO 3ZO44kfWbWcz//4DQD1IinKyh800AFIkCIB40d9/9/2Cvb0+P+5f72/3Dw/fFr8cng4v+9fD 3eLr/cPhP4tULUplFiKV5gcgzu+f3v78+OflxeL8h/MfPn14ub1crA8vT4eHBX9++nr/yxsM vn9++u7777gqM7lsL84Taa6+9T+3lxcA8n6PP2SpTd1wI1XZpoKrVNQjshJ11oqNKI0GQiPy tim5qsVIoRpTNabNVF0wc3VyePh6cf4B1vrh4vykp2E1X8Hcmf15dbJ/uf0V9/PxlpZ/xH/D Atu7w1cLGUbmiq9TUbW6qSpVO1vShvG1qRkXU9yKbUSbMyNKvjMqMrgomvFHKUTapgVrC1bh tEYEOL0kdC7KpVmNuKUoRS15KzVD/BSRNMsosK0FLE7CGiuFPK31lGx1LeRy5SyZWFiwnd1c xdss5SO2vtaiaLd8tWRp2rJ8qWppVsV0Xs5ymdSwRziOnO2C+VdMt7xqaIHbGI7xFXBWlsB0 eSMCjmthmgolhuZgtWABI3uUKBL4lclam5avmnI9Q1expYiT2RXJRNQlI8GtlNYyyUVAohtd iTKdQ1+z0rSrBr5SFXDOK1ZHKYh5LCdKkycjyY0CTsDZfz5zhjWgtTR4shaSQt2qysgC2JeC RgEvZbmco0wFiguygeWgCSPZmmlW4oJTdd2qLAPWX3368+4r/O/20/A/73Ss9prtxCy0uqhC zlrpa3mWs6W+OvnwFe3Uh+P+j8Pdh5e7+4UPOIaAuz8DwG0IuAx+/1/w+/RTCDg9ifOoqWqV CEeFMrltBavzHfxuC+EoQbU0DIQANHkjcn113sMHywSircGGfXy4//nj4/Pd28Ph+PEfTckK gSohmBYffwgMFPzHmk/lqrGsf2qvVe1IbNLIPIVzF63Y2lVoa5LAaH+/WNIF8LA4Hl7ffh/N OAiHaUW5gS3j2gqw6Z/Phi/XINTw/aKSINgnJzDNsCaCtUZos7g/Lp6eX3Fmx6ayfANmBxQH zhZkaf90EkOCLBsVKPkaVA7kaHkjqzgmAcxZHJXfuGbSxWxv5kaoOYRznflrGtjgLshlQ0iA y3oPv715f7R6H30eOQKQONbkYHuUNiheVyf/fHp+OvxrOAZ9zRz+6p3eyIpPAPhfbnJHwpUG 6S9+akQj4tDJECtGoCeq3rXMwLXqGK5sxcrUNZuNFnCBBNYuOCLST0Lgt8ByBeRxKJha49lM AppaiF5JQKkWx7efj9+Or4fHUUmGexh0jmxB5IoGlF6p6ykGbwcwwEgRH8ZXrqAjJFUFk2UM BjcS3BOw+910rkLL+Ec6xDjtIEHOxHQRROQIScDz4nCXmBVcuKl3meiK1Vr4n+XoUWnVwBjL 8VSF149LkjLD4oM34Emk6EjkDO/nHc8jjCeDt5kc+OCN4HzWu3wX2Sa1YimHD71PVgCrWPpj E6UrFF4WqfXvSKDM/ePh5RiTKSP5uoXLHYTGmapU7eoGrWuhvIMCILgsUqWSR87IjpJWjYYx Fpo1eT43xNEquIzhAtLETrpmaPngIn00++Nvi1fYx2L/dLc4vu5fj4v97e3z29Pr/dMvwYbI LeNcNaWxcjKsZiNrE6CRcVHDhjJH5zrSRraQ6BS1kQswLkDocDHEtJvPI9IwvUYfXPsg660G ExFiG4FJ5W+TuFXzZqGnJ12BgSkq0wLaZQj8hIsaTjV2geqAmBaNQyK0OBFsKM8jcoPGjQgo jokyu18H2ELRJkrFlkOuRZvI8sy5IeS6C7YeQwgxfyTMFc6QgX2Umbk6vXThKBEF27r4M8/U N+D8WGcGAoPU6uKcM1s2EEQlLGcln7q85GcnaI9gmqbEUAw87TbLGz3rR4NHe3p26VinZa2a SrsctqBZ89mhMziHGzfq7eCVTPUEmIqN5I6pAx6B9+3QIXtxaIeZTABgX2YpzCFxddcANzJf Bj97t2DY3wgFNxEPIo3s0hKtPVe4W8skoul3DmH/BEhn7PgGTNZtFMMzsNjgOFzLlALmUZqN OyAmyvm6+5o7zIZBI27+JK8h7BUJIyEchWAl+JpCbTSi4KiLmMkFLwzuS+5GEg1KrevVg8fl /objrT0Anrr7uxTG+221BL3qyR7hjsww4AN7xOGKijGn9iN25AcIIwUJdermc+A3K2A2e1Nj aNDPkAaOOwACfx0gvpsOANc7J7wKfp977OZDhIu+CWUKYkY08ENZCQGHLFXqnoA1MjI9vfB8 XxgIVpGLimJ/sp7BmIrral23Vc4MZqYcrlWZu9hZGx98tABvXeJ5O+sA1Snwupm4N/YsR7B7 yLj0DhP56hrAeld4NqyHtfEhVQ2C7QWajtUQedb66bp5xkB0S/6Io+GNEU4aSFTK26VclizP HMkj98QFkFvmAoD7EXatPNPEpCNeLN1ILfoxHl/wNCiKymLKUnHZ/tTIeq09G5qwupZ0iqMI YDYqjWqcFaUxBdr7El3ytTq8fH1+edw/3R4W4o/DE/heDLwwjt4XOJaOk+FNEdg1QsJm2k1B WZ7IOjaFHd3fEq5JyZvETuTYX4j/GVyntWcGdc6SmB7CBMF0NvdWG8l8oTaioICg3YC3m0lO qTfv4stk7t3tysIEeCHDOnpYtyXSrCoX2zn+D3NMZm3LQlqhc3IBYTrox6aoIEZJRO4lj8Eh hhB9B2oKOoKJEk+07CRRj4zWRPl1UEhQATToHJ3ZufWLDFglcatN6Y8IHBuUAfTlwGsFJ9lL AKxrYcKN0e0D8KYuIWAwcCAuH2zOD1QfXSkYGgbpEz5ZaOQ73SHE4d00ECu1WcwO08YIsVJq HSAxq86MqXUEDr+NXDaqiUSGGk4U46ku5o24knCT7uBGxwiULDRl6IKv1GIJNrVMbSmhO5KW VTKg43l03ZUM1Y5wq2vQO8GsvxHgCrmFsx/RmtYQ3nZ/faxuUQZUIIaNTNzbmbrbcNoUVey0 YgrTVQA2VuU0y4AtRYX1hnCGTsAtx8m1Ddlpx9lE4wwuVc1Msr4zbeiJ2eRGn8eM0CqIikb6 2Fa14EjQgjHxfO85uF0ktwxEpRKY93UMbwTl+kU+Eo63jHlGU0I4zSZnfzEbsFxFQ3G7atAn sTWki2vPRhN6JgEQ2pNp6D+j7yXmnkRXXYlIgRUorLzArRcVQ60y06awLMflLVTa5GBt0FKi a4MecWSJYgvGGb1JTPsZNglJsZRFw0H7VTEtZHFV7TrbAlGoo0coUeD2dFWlz5Mbt8Mz3l2M NrfP1ebDz/vj4W7xm/Ubfn95/nr/YPMzw4EiWZc0jpzhsG4i6++/wL2kxfdG0hrRlcBTiF77 LJFl5nryeBeDjLg2npxCjX7J1afgELwwlEA2EwmKzaJRqKVpSsSHR9oNjSA7/daTEbrmQ5nC 9SZ7tFxGVqjRq8VPzS8QSQK+Ohi9YqdRt8CnOTs7/ztUXy7+BtXny/P3Vws0X07PYhxAiVld nRx/3QPByeQDKLV13HPpNY1SVDlc3U3lRp5emqcPRBO9jAK9gsEYtRqxhGg9EtCC5iljfEeS gv8ipdIzGf66V69q//J6j90PC/Pt94PrdKMHSwEiBBGs5G6GgkFQVI4Us4iWNwUr2TxeCK22 82jJ9TySpdk72EpdQ2Qq+DxFLTWX7sflNrYlpbPoTgswVFGEYbWMIQrGo2CdKh1DYKI3lXod +BKFLGGhukkiQ7SCa05qqkxH0A2MvIZ7JTZtnhaxIQgO/DS9jG4P4pA6zkHdRGVlzeoiykGR RT+ARbuLyxjGkewJE1HXuvtlKISphb799YD1aTfKlMrml0ql3JJVB03hNsOPOAnhDsMztz6U /dTl6Dr0iOqLlpbeiWItGD/7Tqmzn+7k9ut/T8Lse+/ourk05teemC5PPUkobR9KBdES3hvz iWdmFEYCdXHt5nPgprODQZPUdem6ebabZgZJ5zGDG8I4KrWmREYls5FkHhMOrq/jQyfwMX1t DeLL8+3heHx+WbyCQaS60NfD/vXtxTWOfeeKI51uNID6lwkGQYiwmVH3wBG5PYMAiEdvL0QX FRnpWTx4ouBAYatPl4eKCA7S2aaVvNKTBbBiHNzl5GP1UTR+RSI9oSdIaBRwzkFiump4xmTe uMkzqy8gTQaYjl0hXQOX4z/uIMzaSA3++LIR2svy1wy9bHcjPWy2QDEQRORl67rn8ANrxZ98 SLVZbQofBI7AMvFB2kbDlBn32Cy6wnObxVyE7gOOPdwU4aYR1E8R3fhfFxIH0qC4VCqqh9nU 4ejZrC/jLlWl49JaYHo03h5SoOGIrGioKFeNLz8kC5jB7hrlbMnswiXJT+dxRnN/vi4cDjo3 sZK98SF4pRZNQQFcBhd7vru6OHcJ6Ay4yQtde5UrFHmrZFMwaNgUyCGaYI0r8JUwYQ6OYKJo sAUTwgpnVyllK8aYB+470MiiaKIHwFkOFLspxdAkI5VXvSLCdiXyyk/xFmwL5i1W9aFGQ6cV zGq4Lkyo9AX35He4t8q4kesJNioHyYZNRDP3ROMmUO0g0gb/hClbgkmnQESw1D0B1gIucGOr L0mt1qCoqCoYD+tAcNyEXQcI5aEHe/LQAzFK1iuIe2PT/Ci4GS0viTjEoxBNtptJdmRTXF7M XAKnF5NmZqGrTG5DxegbTTrZk361XV6uY4GO5LXCpmfXQe1AISdGhMeLEYzxPxmBjE0YC5rn sQLkTqYjiG7/arUDdyVN69aELdy2hRoTj1E0Kbmsga3tMsF8i2dusfQ+l1SwTUJgRrsKcuhF DejOkQvxIsePdjcmhI2TtCHVw9doZlvMQDnamudiCcLQXaLYUdQIbGM97O8+fZq2sb77qXGd ELA1LIYJs7F2HoyAhatuDkO2EP4WIobawP8VQ9U9RkG1nNYuqGqNWgoU/nfmmi4viLM9cEv3 jDfMiggEhaxOI8O7/UqMsULdoKm767TFtBBNH78V7TQrZao86rPoKgf/qDI2cPGsK63Qsqwn QzfO+AulYIcHQaZc1kG5yZ2szzLG6N5Rqt4LbrHP+ep0iLnA/rr6az0V8DfcYgTeNNP8/FoX 04CJJMX2pqX11cWXL58v4pHKxO8Mud5hIlx/P+caw4KEXrOd9huSImSFrcHGEtW5YCW5M24w okrjF684hRnjrQ78nvi7U2zU50QsPizQV/8eztD/2E2llKP8N0njWOobXQSN9H1LOJxQFfTE 9cRULXzHDaRe877UNt/Jn4m69gsj1Knh3VBY0yJMn6WPfNYGHpug8kAJE1ReNKl8N+lSCvBh NEUtMm0CwRG2C9QNNVbMXMdoL2t8YKOuHS+zMLXbvAS/IK6A/UmvscqH99rb3yynM2QkklgE QX9tQmxzRWYaSGs4XXyuxPw0BqFt6j8I/wq/uWoMQMEPjzBDZNLP+WjjltIRQhW/SV0p6B09 /fQpXmm+ac++zKI++6O86ZwwcHVzdereoeTMrmrsB/V75beCz/SxAwYrYfHrgNdMr6iWGPNw wfRKdFxBhWp8oXLq3+i1QL/W+FfoUJWh6sMcnPoSh5cv54GrgEdHT4noC9q/DGhFVKicrqi7 1rrYbtglXo0YiBYuQfxobLj7l2RdxWiT6vjzgU6PB5e2pBdjsdccAWEXd7mrn8w1F7X0eUjY bh4vCshs1+apmTbx2Dd3sMQqaON3bmnfdRjyVc//O7wsHvdP+18Oj4enV8pYMV7JxfPvmNd3 slZd4c3NzNr3XZMOzv5tGIbieY4dgd5F5zwdi4lt0epcCMeY95AuSzaGeQV18xEuPtE1W4sg ceNCu1c9p6MEetile1UV3hRB/gpX0pU5BpS7TGxx6dn1zpanY1NakH0oEB8Y9H30ED/yB6jX SHH9k61wOO0x0wCDu80Y+KsXZTIZelKYso8k6emaLSTjkCrlwSRdK5P9Pr341M4b0WHnREv8 WIo6Fj7RXJ2g+KMw5sm0/cLcyFpsWrUBr0CmIvZAEWnA8nbOkBMpIoKFe0qYgRhqF0IbY/zy FgA38EEVzJexkCr1M+oIovRSLeDkvN6lfsNCY9pTppNdDMgALqtCBuvw7W/8C2y5BF+Aof/j D+6yC+GqG20U6IsGs5WFT+VCiqhZtIulNlsrecO5zZPPZXTtXjgKjjKhfxYm0uzqwKFmspzA e1ZJ5WeArFgmocRYHyi68QJCU5XOLTZZRtQCXNkGTc0KYk2qyKky383NAP8ybgfgAJzl0qi9 rBJhg9MA95ufIuQj5XIldAwOJyHYhLWEErL8cbJqi8EXye8esNhCTLt0R1dYglIVCG78ztHk TPbvehbZy+G/b4en22+L4+3+wXvK02uhn5wlvVyqzfjMJESioRtlwgPDZkDE3E6XAd3HsDg1 dpDhM38bHI9eQ5QWTR9WFuKORmwI9qZRC/7fH6LKFKIu/0Z7fwS+14XwZc6b8ljp7zdK0e8y wlhvSzP4fv0zaHexQDJIx9dQOhZ3L/d/eMXgMXipgr9pQKaGU/mCxMjLSvZ2/n0M/DcJJkRG lOq6XV8Gw4q0kzNRaojnNtjy4VGASyJSuJBtcaGWpfLx1bkt/YD32fPg+Ov+5XA39Q/96bDv xGGavHs4+FoUvsnrYcT7nKVp9PL2qApReoECeSAYeeiRjqumir/MsQfULYMWWhwen1++LX4n n/i4/wPO9uh2icl/Q9xHI9zpiCJ5O/YcWfwT7Nri8Hr7w7+cfhgu3WcAsssX+7Ci6J6NeVCv 1EhD6Vmq9oAC/ScvRTYa19EMAgi8mjoea3YDutJBvKtfMO15wh1k4g+P8N6r9D+DuPeN1Eg2 2oGZNWFuxecQMGIm6Uhs1nICmHkFTOyeKw9zvIltlqqLsfwn++RoYFbi0avacol9VpSqE67x WRn/sTAO995iIkC61UcEVHWwmYppmfqgvlnHRnwgnL8+H18Xt89Pry/PDw8g6xMD1v1ZFb8h G61JmfjswXR3PEnSOUj4uejbSpg6lWq2SQGf8SQTNRN/Hm7fXvc/Pxzoj/Us6AXI63HxcSEe 3x72gUXCHsvCYLfp9HlGFIXNTdiA3d9Z2Ke6Eiz1emO6oZrXsjKTPD5TTfSZqB1USM2d60ah J+e2fkj2+cwroLlwnNq3zlv3b58MbUr+TrEO2Vyc25xL4Rdbuj9kEI6kpxkh0FayNyQWqvIi ak5dPt7Lu8kyAJbLcg02W+uuQua805PlsrZvMuiYy8Pr/55ffsO7dXLNwIW+Fl5rB/4Ge8qc 57bYq+Z3rgUE28x9foW/6A/3+ATkQbh+JAF1k4DtyiXfxYUXaWwlJGaw7BT4BEYbr0GREHAa mKJ8dHkDp+GuoQO98wnp8V9W9pWP/4AfoEPGgurVtYfLZNKCNyDa4Dl4P1mF7z8oDeDhbOXb UjD3zzUNOAjdEuUmFgcMz5n2LBdgqrIKf7fpinshfwemJGH8ybwlqFkdbUJA6atkwHNZLdEW gCZuQ0RrmrIEczqlj00R+dsJyEPacrCLwuXewN/Y+eLj6EIX7ebU544FnrkKXoL+qbWc6Fu1 MdJfb5PGt5apZgIY2eCePyLZyun2QoDQVQAJZZyAJP3h5wkTBVolwyKsrZph8miW4v0JEiHC sXmtgm35xsOui1cxMLLRByMh/HMZ6YQeUInkkQG8SdzcwgC/FtpcK5VGhqzgXzGwnoHvkpxF 4BuxZDoCx7wzFQymqLyKzuMGFAN4J1w5GcAyhytCydiHUx7fAE8dTg89sh3fJl2yNSznnS7Z ftark9u3n+9vT9yvFekX7XrmoEEX/q/OjmJPQxbDtN2LD0fnAWVfa+Nd0KYsnbHmFxO9upgq 1v9T9mU9juPIun/FmIeLGeA02paXtC/QD9Rim2VtKcq2sl6E7Oqc6cLUhqrsM9P//jJISorg ouzbQFdaX4RIimswGIzYuSMLMih4vbMgjttcvxocf7sA+uYI3L0xBHezYxBTVe2ZW+1aLqKf Q6YyhQjeuki/Ixf1AS3BYENZW7RPdWYRnUIDSKZ5Xb/h9RDyvcZwi8OGxwXABd9I0J3kQbKi 1vgSAf9ncOJZsOZCp/66rc1aeyQn0cNL9flJaULlyl/U/lNwyTpezsPvazC4YZo43Jkwbnh6 ylDKg+7g6/cXEAeltP8qNyoBj5tTypMg6ZCMBEoWM0rSLm5m6Nq11wyD1jsOZLj+X5bKnICg yruL1hR6mXur1TDJbVNMBZsaanyGqfpgxNcymMt2CUWI0DNgGxKmqn4ToKt+aiXdqgvSlZzh 8QSPKVS0QgSRtIFX5Cqc8zYLVC8DrR8LEI92miPlvI7WARJvkgBlEgD9dNlzlOVHKQIMoixC BarrYFkFK0NfL3jopfbo1qc7ajA89ocA2VgDz4yYU36Vsj/tUCWjCZZq/5ileMIwcKDvTCRf T5ioTg8Ckqd7AHz0sNrtDphdv4C1vpfl5l8rAT3VIyV3WcLuibxkFgQX0ls+Dy5huWfHlBaO R85pQ7EiaxlFSLHkc6PWMYqpC470LeNIiYDWpNiaY3daACYerQyhdihk9YvWmY/Va9TwecKc ShoutPvqvxvrWi1DndI5/Vh8+Pr5149fXn5bGAelviWoa/X87U1VjbYZslBFJHm+Pn//18tr KKuWNSfYdylXkJ/nWJSZmbgWb3ANosE81/xXIK5hvZpnfKPoqUjqeY5z/gb97UKAXl8fAc+y 5Vn6BgPp7h6GmaKU7I2aKMHt0Rt1UR7fLEJ5HIaOR35DbJVaGGZluYkb1FGZeOMD5ia9iavN sjcY7NnRx9OQsz8fy1/qnXL7VwjxJo/coYi2UZM/Gb+fn18//D4zVbTgsDVNG7UF+TzDBN6z 5ujGd9ssS34VbbCHGx4pmYKaeJ6nLOOnNhNvcumtyZtc1org55ppqonJFrc9XPV1lm5JEx6G 7PZ2Vc/MWZohS8p5uph/H1bft+stLIFNLPPt49FJuywNK0/ZGzy3+d6SR+18LiYGwCzLm/VR sOQN+ht9TG/PibrDw1UeQ3vJkaUS88NZX2+e49CnEPMs5ydBJUAPz6V9c+55vFZEQnQ55md/ w5OxvHiDI3lr7rFkdg9DpU6IZlmUVcZbHEot9wZXAxcX5lhmVw/DIqWOWYbrOsJaJyMlkmcw yf4l2u4sNOYgL/S8dvhHChkRlGhp9TQN5h1fgganA4jS5tIDWjhVoJZZ4GXfFyiCfGP2xTnC HC38HZLIj0TsMFQIbuC0G54R1eOgVsbnQjcR9kCuqHKrAg0mwD+wdlEhp9rF6/fnLz++ff3+ Ch6PXr9++Ppp8enr82+LX58/PX/5AEerP/74BnR06q+S0zvjNqFHbyNBbqj9BKaXLC8tSGBn P64G+Z/oc34MPjfs4jaNXYd3F8oTh8mFjpWNVLejk1LsvgiYk2V6thHhIngvoaHycZAf1WeL c/jLZXcbm36P3nn+9u3Txw9KRbr4/eXTN/dNoo0w+R6T1mmKzCgzTNr/9y9oYo9wmtIwpa3e hLRlNmnQYlg47FnBAb85PnGow77eIaRwsT6QCWuoJsbLCxpamxEwhzFQBK0cCnyOj6ZAUIJc s4alvo8ForcO5H7KnxxoDuG6Dk8C2i9bsaootUdTRTWfsnNInNe2OkrjZkNz9uNE6MWEph4V /x5q2+Y2wc8+7jKpFogQXd2aJpPNN3ljapgAg70ttwpjb3mHTytPeShFs1PjoUQ9FTlsRd26 atjdhuTO99oQc1CNy17vb1cWaiFJmD7FzBT/u/v/nSt24bliF5grdrNzxS40V+y8c8XONyns AmN955sYdr4JYDczAezemAB2/glgF5gAyCnvLjREd6ExigjZle82ARq0V4AECpAA6ZwHCFBu bWEXYChChfR1R0xuHYJHVWgogZSCk8nujdlk5x/eO89Y3IUG484zJe3enJMwR1mPuuQ0S768 vP6FMSkZld/8o1wcWAx2iFXjG376vJb2RHOG654lGIKrmtfxL6ykhqPgY5/Fdv81NEmAg7Vr m3lJrdOghEgqFVH2y6hfeymsqPAuD1OwkIBwHoJ3XtzSWyAK3U4hgrNrRzTR+rO/5awMfUaT 1fmTl5iGKgzK1vtJ7pqHixdKkOitEW5ptOW6Q3V02kYrmSyydKeXwCJJePrD6e14f6XeA7Zo xjMC5ls7ZskqJ+OX/vz84d+Wd9nh5XDyQ/mVQ2//Rs9ReyhEx9xCanwA+zQ+9VX8Lin9V+o0 z2AspewD4YAkASMn3+2oEDv4YyVO7kOMAV/Lit/KH7t+sqgmuzH2BZoX5EOvDcQmA/M0vG9u wSX1Z/wkZy3ZdejmlrVIVyUfpEzGSRYDBu7QeOK9hA8sOTmLB6SoK0aRuIl2+42duEZl6+v5 zHcBnShN4cm9s6pQHOdJAdx+L8O6VTKDnMgsV7hTnjNo+UluPQQ4paSOZDUVpiEzRbtOrNVQ EMg3ouKWc/MKHRBPWH+6NbWPuS8IQS9suILNUqet9nxVizf68oGo4Dqcknw07ue8PYDlF2yj eOtZXecZhXmdprX12Gdlgp2adNEWFYjV6GJJfa6IvmqXV/caz/UG6Mtz4gVluUTrp4DERw+E MPVc1X4ClUgxpahinhNpB1OhPYhOFROvpJKyLIOm3m58WF/m5ocKM8PhE/D1ZMRpK4wRyalI ORHZeeq5WF/xVdP+4x8vf7zINeBn4yiW3CA13H0SPzpJ9Oc29oBHkbgomZ0GsG545aLqyMKT W2MdZStQHGMf6Hm9zR5zDxofXfDkzSoVzmmLwuXfzPNxadN4vu3R/83JubpkLvzo+5BEeTNz 4ONjmOJppbPnu2vuKcNgo+hyg2OOwfDx0/OPHx//afR4tPskuWXOLwFH0WPgNuFlmnUuQc17 Gxc/3l1Mnz9MVzU1pK7meT1TabJre6ryFbfaUxqJ7jyFAT9HDuo56NZVYB2Qj0lY52h9VlA/ KRNmQhZMgX4RKbFv5hhcnYR7KeTcBuHWFmkiKB99PkLCSp56KbwW1mGX+mxGbOrAQgiMI+HA 0Coq4BA0AS/32o4ydhMoeOOMZKZ0Iq0L2sYsugiZbaikYMHtylXoJfazJ7Ydk0Lp9mZAnV6h EpjMCYgQDlXHvU5axrHPcdyGNEG1lJYQVEZUEMIZh3FoC6b84CMxacT6GEfRQHiKL98hvEy8 cGEuGU1ek1BSATG4qrPyJu4cxsJnD0gVxJhw60hVk3eyMsNX8m96dUXT3a1QTituRcI91PGi IHHoNqD0ak1R2zMhIP1JVJTHFSoUKnutZeN/Fo3dI/RnSbEt0CfyNahitN07upvb4CuSzVFF WyWOXWs3uqUyz29wBDZEcO7HKeEVwoOKp54Gh4sfcxr17R23JgOYFI3SgV65XLy+/Hh15JX6 0kI8GjKK2sTdfJ5Z0TDrVu/oThi9Lx+oHgyAOCkocLqPiyErF+nL/3788LJI7avKwHlzUhe5 AxEzCAASlidwegYXS/A2BGh5RqKPSuQdK9/LXR4r11ZO75hy+uYD4R61nzCGACTUrBDKUwAB 64xdwiiOUQH45cYgypTLn3cumLjNkhhuX/EMzU6l8KSSPDwsPZBbIRpG+Y0NLmq++Aix/P75 /OHFavAiqaPtqsPsVxEH2aFeJd2qbJECGFmdzsNp6tTBVRs46B62eA4KEZX0DIQ8bA8u3rXD SO9dz0ZZoutjle8pGw/z8WX9hkzGvKGmCw0Y9OHnlKlwG2w8X4d0nXvWik+7S1dBxnPBSCx2 ST0C3jQ0bHrqcTCqs/nyz+/gOeQnZZTwmx7Ok+cBxSN441LGpNv2SQoK492g9OuXf316cc0Y 0kop3idfkIIPGPLK1HLxJBy8zS7gaNCBK16sIynT2gS4W6AXJotQsJ0c5jZ64o3c+7rMsjuv IpcdXFrHWX6BIPTuB0TLpZsUOC6FkCkOLlL2/j34+3QIh+1hQlXNHmeaQfbsoVdOsUilKJrl UjTCcrZIKHDnZVyBu0AMiiKBHmqxspxT4JYLG+GMAkUiKBBjHTOcF2Qplspkzz3SkTFCfUsi I8l3y6ymiUlA5ujEqBtI+nzdQz3zlKZ0FoSO+618nNQPk1zXjlEzfX4m2lH/Z0Z3/OmPl9ev X19/D7YonEsoX+vkCxOrZlpKf0wY/Y6Exy2Z9RCoUvvTR2hwFOKBIFK8o9folTWtD+vPGzsB BccJNjlCBNae1xcvJc+98PrOm8xLsTzUk9wLLw61Zn9uUkTLdefUWy0FBRc9eqr4dsYrMpzL NLfcAXqnVnVNYOTO6V0vdpQSZqN05ZNe2mDBXUXTXRjaIoIrieZKLrJChebkluaA9MRH/T1T F2Zw7SsI7jFakKifHCZ+w5GVTqC2WxHpXikIV8o7ZxHyuD68CGt0lsuNTtPL3UsJs+w8P3hM DlURSlQflpDNwEQcBrKbulY3sxyySeO5LBJYkScHmzb5TmqcwKA4JS/lPB4q0UJkLk+17D94 mrRoCVGfWMT2Qk9WRnKoBo0aduUoZlcqUGOTkMhppkUS8AMv2oaElfBQ+3NLA5h4WG7ndqZY wDo6oJ/Nc/CQ9bfPH7/8eP3+8qn//fVvDmORibPnfbpLGeGp43jSEYOLc8vjOn1b+Vab+0DR ssH4tlNR5qcYlXcOtsifyaNJNYdB9Mt+2hdfOJ519bP1WQbkZY3vHRtUB2clRgqGcqpttfCh tp9VcAHFRrb8hzo8chk/UhmeH2eZnSuYCqTbg6w+G395U7oGA28SUhoJ5jCwQegyv/qpPBLz P9nj+InDSQgBS7yEGADiXrkgXYgBPdvvinOaJ5Ni4fn74vjx5RPEZv/8+Y8vg2nq3yXrP4xQ gm9HyQTqcrte0zTtpQmwtjk+HB6WzMqdFxSASWyFd+gAHtPaAXoeWXUlC7LZeCCXU8WhUQGx /bDnDbJKDwjtGhPq1LqCvYm67SbaaCX/Mj/qpiK3dU6H0FiI19NXutrTqzToSWV9vDfl1gtS 7vxulHb02HhSc+m9oa0t0tGAX768fP/4wcCLyt7zXpXvgymGlQ/ulWerv/0NHei2RY1n2wHp CxNKajKQaOFmee63hpAjXWUjNzFaQQrh6JG+73hX0X1xwUZWuUHU0YvRrreTk/zIgQo8pqPj kbsBu7wMcrOvXbl7Sq4CfoPWw3VoB+4R7wGahVpO/FMpv9FzeZshuzWZXwiDnf0UDNDLguLI zahfMBc4d9Qaz2kRyk5ES62faX81mMCeFEcMu+A2YFFgBfCQInYMCR74xJlBQJP4ejxajZeV iV7f/fKsikxNA1KNDkyduRhOSaxYJyqglO0ipGhTvHjJRyVUe2MbtxAXIlUh0SDiKtL8Y5K2 HoOABTqQ0k8rmjxJor+Wyg0nazO/M0r3DZiSA16zgRnHhLVKyJqHEVY1d/0hp5JC+wNYsC+/ LVq4eaOdUS7y5z/JXhtSiPOL7HFWsvozXahvkBRzbMmKYT/1zR37LSP05pjS14U4pglWxFAy FKACP48EUeF+rJYe4+ZCoD51EuJ0roYVPzdV8fPx0/OP3xcffv/4zdVCqGo/cprfuyzNkuGY AeEncKRqnz6YFNSRV6WC6IQ6IAypmJUXKVWk7blfkZ5sU6NZ6obI0C59H+yQdiF2f5VzHYV6 rfx4bn2MwiK7kArdBDNU5P1cLqD5IGfVY+UXqWhTF5dLH3PRa8utPie7igVUFsBioa2RtM/m 52/fkA9u8AirO9jzBwjEa/WvqqhlsYeAU3QGU3dlC0//1rBxFhKoFBEn/anrrJIW6cOucz6A J2cXzEQcOWBy2S83Lq9I4giipuELZYDLrcbryyeK5ZvN8mSVi5zgqOFUQyjLFHuFUQVVHrJv TV9WFgXOa5yWykdXBkPjiJdP//wJ/Aw/K6cpkimohFSpFsl2u3LqX6E9bGl5F+yyhiuoL5As ENJ6qDXy7kjo7w3XTkr58emtVPqqra2pIdrW+6XVVlLK3lp9XORO3dVnB5L/2xgcTbRVCzGd YMe9WR52FjVrIDyToq6ivbOaRHql1iLyxx///qn68lMCoyV0uqq+uEpO2C5eO1qQcn/xy2rj ou0vG9IppcjbZ0liV/qAwxFAoKpL4px/fClOzh4UkoqxUY+q6cJxtDa+kGZSauBBgjtKMDFt wzSRNOYu+0mPhuV/j8fVcr9c7Z1XjJbCrZlKTfrg9QN2ATMV1PNUeMoipdbKV00pF5eqTM68 9mY7kfX6OrrqmyuB56VUWbgt53OI41YNuNm0ZefbeL4jYcfMA8M/RAEwUtwj7ZF05oJvl1Yu Up5x+58BzQyhpwva4wYOs3OwK3kgW86KPRxRB5V5ginA7GnzWlbr4v/ov9GiTorFZx2xwDub KjZa/EcVXdQjTwmICWNP8gZUuqWN8jwoBXJrzbzG1iCRQH/PUWhia5JSDHEWG+uTaGnT4FCZ bKoGAniK8+U2RFod67k6hsLtouhUdQKCqx11ykC+3XlJY4+V9ago1Q7NHaG3dk/q5Vs0rJbc Chizoel8T0N9ec1zePCueAPT0TcuByJYhwgBCwmv1xEWTt6ThQWe+vz9Rkk4xnsmySmpH/uE CyHXvtnSpCw57JYzJboWyq7XeTGp7maennk5r7CpOUZVeFcdt3tv09X5QuV/N21ism+F514r 7XUQDD5boDJO3TTFxQNWwsfZ7V2QNAwCzeetdj6aWu5JZFtMTRlyhZ+kUpoEe64kveHANxg2 WgVwKjDpownDXdmF+C87qPBc9DIJhL3Qm15P2Autgge6r2ec5zq41XwjLLpurtluxWjoUHz8 8cFVe0C0HDnTgc+OdX5bRjhSeLqNtl2f1lXrBanuBxOIgjS9FsWT0upMdjlx0TNR4DsdrGyx 6C9OEP8j2eCbS8dCH5BS6KHr0MkYT8RhHYnNEmFZmeSVuDZSVgQtWYIvWZ7rnuc4Rl2disN+ GbEcX+8VeXRYLtc2EiH5d6jHVlK2Ww8hPq8e9gH8wYOrkhzwsfi5SHbrLbLhSsVqt49wfcC0 9LBdRY4BagxKIywaxkW93KMbaPqZNqnBSGuCIYOxfj0KdtjgT5JSRyurV4qn9bo34X2m79Kr +zDGIupXRj/LviK5WNNHK1WFOg5KJhfxwjV90rgchhHqJRO4dUATkdiGC9bt9g8u+2GddDsP 2nUbBCfxw2o59MtpClFo8CRrosphIK6FVuAMsk/78t/nHwsOR6R/QJzSH0MYqsnNzSe5x1z8 Jgf0x2/wc6qUFrQPbmeC0W3aVmXB4F708+JYn9jinx+/f/6PTH/x29f/fFFuc7TDzylRBqY9 DNQade41bR6t9r7IPfmi4IlSseoNFjJq0yGOeaLimw/b5oQfvdxAwIxT2meI6BNO/FyJ1n0p ef7+28xLENuHFMlXHE+qX799/wq6l6/fF+L1+fVlUUzhZf+eVKL4h2efCflVePqTAvT9MbOf RzFe7nQbKZs2WQLrzdN0+pwlZ7KjSrpcxSz2W2tIoj44gVBHQZYsO4fiJPKUxvFI3dhgIhF8 0Hs4IxaI4OYeh1rmqRysbYNmW+DCN2PBorFgFjK4aiY3ciH1MQif9w6u0OE9+8lkTBXYlHTx +ue3l8Xf5bj69/8sXp+/vfzPIkl/kqMeBTcbJRosUZwbjbUuVgmMjm83vsVcNBBEIq28weCG PE6efLGSQH3kuORZuNJaMHKeofC8Op2IXbhCBVjtMvFUJqS22mEa+mE1LWyiPI3ZHxMvzNW/ PopgIojnPBbM/4LdSQCF2cAEUyakpvbmkFd3bbcxnS0onETk0JA6YYHQXXYaSXeK15rJQ9l4 KXHZRUFCJ2uwwhePs8hiHXrQ+t538j81rqyEzjU2R1eQ5D50eK80oG4FMzNBEowlnnwYTx5I ogaAcyZws9WYI8hp+zIwyO0Y3IfOcvbUF+KXLURLn0zxDJNeNXUMOp9QTtgKJi7TbDnlczKm JmA4R4KEDR9wsD/g4PmA6eLcwPHmFxz+2hccZr7g8Je+4LDpyLVtAwVFEj153nTDW1MqoDP+ HBBTK0ubZ204/Wth9++0hn1EZXci0MTJsWXDTVKIxp7rZM4RVudIMU+tK2V2P+F4kiMBG0JP ION5XHUeii03jgR3mEhJbu1FI6geZT52Itpq/NYcPXJTvR7FOUm9INXnEoKjGTSTg5RAa3vq uQq5LuCLtno2hwOBuiLdzsiC9Y1OS7AZ1u84W2E5jWNjLfWI5zj3qT+WTllESdWOI2iGjzcC iBEpuvXqsLLrL2Mt80CyZ59OWWqc8P/p0kEYydShGwRUED4WaF2ZjECKDV031xb2piYgJ837 lLZnd9W025bXztJa8pZXLsiIbZiurTazVwDxVGzXyV5OF1GQoiJmatW3FBxU8KJfViHeIeaU 5+snrrF+dpsQBzEXMZ/eOJXRGG/oDmdPbXIU/Ki6eC+HnF0xjznrjyRUWQFY5K6YwOlbu/P6 6PZOAN/uncn6sP2vPftBPRweNhZ8Tx9WB7tIOkgtxerCt17XxX6JtSV66B/ptytwNCAlIs05 ywWvrPGqy3C2xfhz36QscdhACyPuLpwVHl6WX+0RWolU93hqHDvSrnnqQVO1dKotMnRd2kqK IRR4XQmayHkNA6c/2t6tSQOaQhMMUm/lsN8d1lM1uQDofV2lqYXVxeglNhnD2f5Y/Ofj6+8y wy8/ieNx8eX5VW42p7uBSEBXORHz1RHyrQpAk3WarHZRZ78CsqAvLcFzrItR0PE47h1k+T7Y Bf/wx4/Xr58XKVhpuYWuU7lzgM0fzedR0HZWGXVWznGhd406b4n4C6DYkKoDKprjgKoq9eJm ZVfaACiHuMjcGnEQYSO3u4Vcc7tmb5w5SCun3lG5W//VD6xVC+IMNFKkNtK0WCLQWCurxgXr /e6hs1ApLO82DvhUU28gCpWrA9kOK1BKJOud30xopD908/QuKt9gWPtOv1TXb/fRam0VVIH2 N71TZs+lhRaskbvI3ELLrE08KC/fMew9Q6Ni/7BZbS20ylPVPS1Uim9kmChUjp5oGTktA4Oq yu3mhrvsRODWKDabUwjRNWgkk5/aQDw9+23Z+3d4Wa2HAfAnQdpKnHlsl75t+DHP7MKTgaAQ c9VyHAi8+unrl09/2oPBGgGqey6pnKwbzlO9uimWnkq3q+u9uR5PLLD/+fzp06/PH/69+Hnx 6eVfzx88Z9j1uA6QOWw4WKf15exHUueQtcfDuUhBPM1YQyAQBZYOsnIRl2mz3RFsCoKMUSXz kWK6IVlidbhGdGH6GnZ4u2kYjH4r7PduPPgslFlJy0u3ltKC2GIXAfUh5ghlp7I5YvEIEA4W CFzg6UHCddbIUdCCAXyq9xtODsYgVnnoGW4w+XNVh77YYBpcTtbijA/tJNieuTIjvXEprZXE PwgkYprBQmR9eEDhRZM8YySIR6ostMgzuNSpKAt4RQXTe1GTSAKSQmVVCbzPGlq9nn6H0R67 7CIEQetGKWwIom9BkFqV214dnXyCwDKmpZDtYcZ8oTKewbF+h6Bh5DhTbjC4ZRsN2JHnGa8o VtPtBkBQi2gRATOAWPUelZeVZGJrM20uEdcTNlncX4VlJ6APRLIsW6zWh83i78eP31/u8v9/ uOr/I28ydb/4s430FREmR1gWIvLAxOfDhFaCBI8vYNDA2mKuVwi/UxtzIQKpibgVLZzeyoXF hg4fON9G54SPVylevbfdcB3RLM5tt3NtxgoXMfGCPUE1CUNTXcu0qWJeBjnkzqQKZgDuIW4Z dBXbz9jEA3dkYpaDdRy5hUq9UAHQUtfwlMFypWS7TwLxSG6kqtyL9elTyQq8GqroJHluxfyV CJxztI38QS4HtrFzQbG9lngFkI/9TTV5UwnRe/WzN2LuYSw6aLTFnEa7lendGnKTkjXgjtO7 xIArVE9/Na5f+BEdAv/mugfiR+XPgV7AhGMZkbNb5lzmVJSnMgnd5zwLbqVl6wBu6nyFfL6G 6PykMRo/6GYOasciaSZd9ZkcZMo1EYg6rg/ijz9ev3/89Y/Xl98WQu5yPvy+YN8//P7x9eXD 6x/fscU19TosYCe332e7ruvCvokxl4rF2te1736w7AiZ8j8zTR2pfQ9Xn+f166RCxg+3qiFK t/apPldOl9FvspTVLXZ8ZAAQDZojmU3xW6cMz1lZu1qvOj9n3mYVEo3kKmzpVDXSy+2tnBH4 qSp77tNxaBuBVgQKhIVU+bBfrVbUcKqGIWH7EBzebuioH3FogUpgdUoeYRE6X9GnjD5i86U8 UD9XKbGgg0X93Jfxfk+u2cKkwVK4kEauosfeRPWcjrtEjK/eygfYmahbeCLLM+ze0NBgTZqj IyApYJ7Fh9Blhx1BEa26auA15e2sx140vEL+JjSo53YM0ixb+zVaH1B5+G3mrbiE3fgVVVt7 lmsfRFiWHRS7PsT4LYDH+KpIzh+vPDR6jY4TWS8ZpWe7wofxA9avTh7WtYd148Oon0WEKxWr h3A7+kstBTtU5oy0c9L1WYJ9VqaWx2iUUJolAX+AA4MxxRiSyiMkwQpZ40ZumFSsBlMa/fm0 Mym1Z0gdF2cR7czquT/fCzyLGdTqkzjZ9+pagJfUMSQ1iIgsd92JfAg8mw2bOnOAO8Dz33NG mZ7rlT2HGC7L205G+DLqg0894sAYp5g82HUDUIp980iAxP7qSALyKbMe7XrVoJ2NAmlGBoot iOS+Id8in5zcAKOJAEJHB0A42WOxWl6sx5nuwffRtkO97l3hX9YGNd805d+oDCAutMPAc/gQ HogwWRMdmbg8RTSJpyiYBC6bLBgrK/QVRd5teuzb0gC0NgfQqh0F0w2rgvRJE2bbumwa6jOa nri7nBIjSjn8QTwhrmUuYr/fRNi7236/XdnPMtEcY+/lS91sHhWdGuSaHu3f7Za4DQZMb+yD d+gkWxdtJJ9/lBdPDVqo4Wm1xAEGBoSOomPG8tIvrJRMyl8FSnMAkKe6/Xof+YuzXx+WeDRE l2A1lTe5WOIyVU2SpZkVsXbgri4cy6LnyvLdXTPwiytr68TLDPu6kALUGRXgKQMPC0ful5T1 Se70+mPO1uSs9jGnso5+tsUSg5IhYTBrRBjUmvcecys4MBhB0HyxV2n54J9aQI2gfPb55icI LttmaM3dr9YHbKIGz21VOUBfKzlm7MgDLHe+Wd/eufDHhxjY9qvoYL+uPDc2xvjJ5yNtv9od vJ/YwFTHhP8LU9SUzW658ffZBlwqN94UBCuknIFtGtXyE+qkIsse/elw0uoiOUTL9cqfBpm3 uTgQawsuVgf8TA7YwZ9Ni498AUhSsJctKWr115HRseHEBSsEqs6s5gk1BJHkw2pFLMYGTN8m k/vTi9daAbg2gQlFtGo6RWVtCwhuY0VfUZh7BJ3eAZ/uFlr8ambyiauy0ecXRvFUVjWcdflq qs3O1xZNV+bZ93mEjfdJLdcuRjypOj7azZs3PHeCd8/mzHHcnBGyTF4AB898CVE7o4Tv/D1R SOjn/r4lzT2i6yVZ1QweX4XxSeLVkSAuXrp8Lhcrkb/CY5riu0HZkdopKiC0LRCXI7nJK5fp mofc18dUSq7PT+B2zNgDFJwvJBK8oc/k7Fu2srYlE1YZ7JfrzsKKlAJGJKRgKvetCVwqw+Aj LM0UysGZIwbkTp+lVjHM0T8FQfsCAZkTQXEYoBSJk0KvurpGUDgjrZRwuJXRrA3uH2yQJ3V+ tXI36xQFpVDCK/DnSAvayo0MNi4AN7hZu1quVtYHaDnQquBayjWbvQfcPbhvV/r+NoaPvMt0 SxJtHvhMjxnxpg8odbinoCpRijQKQl1fS06CdpviS+n0cNiSQ/u6Jg99LKA9LFCOkJwEdAfQ 9r4LWFHXFpc6AKN7dQlXJLoOAOS1luZf0ShckKy+IUAgQKgmWuQ4CJfIcWwmoI0XcjO8CAIB Aue0FqYOJeDXDnkuE7HW32rtOb2ylrA2ociF3ck6BFidnZi4Wq82bb5f4Rt9ExhRUE73D3ss cQIo/ycKi6GYrNvvVw9diHDoVw975lKTNFF6Zy+lz3AsIUwoEw/hfJV1wMN0IBQx91DS4rDD RoMDLprDw3LpxfdeXM5OD9uu81MOXsop30VLT82UMFnsPZnAtBS7cJGIh/3aw9/IVU0M18g8 VSKusbBbFHxhFNvdOqL9kZXRQ7SkmHZybvE1hRyX146iWS2qMtrv9xS+JNHqsHQ7/nt2bezO q8rc7aP1atk73R2IF5YX3FObj3Jput+x/AKUM442MrDyst2uOqs3QEXZkdqUO/X67JRD8Kxp WO/w3vKdr9MkZymBY3GGyOdyTcuaFi4jSAEKXAJavnYpMahIcTlJJ2Ic2yLIGRS7FFe6ewqx 9mGXbJedSWUyRWu4KLYbTwHga9+nq8g3zHQPvRPxkqXFnjgJVlMUHg8aeHCAkGvd8516F7nn e+wvuz07/mYUxnS0SnSieZ4xpT1vLxbz9jLPTo83JHS4yKKSQkjEUzSJxm1SZZ3rIlpRbWZP Puwc25DjgFjDolVSjv4rQC6jNsPq1afS72RSk+/V3U7VdiJsvvXMwEuoOvUmLq6H76iywqkx vB6OkPs1sjnzw4oGONHI0G3QkbIhmES8p8qG5V4nbh6yGRu8gZXF2l3IIIJny7m7Aa1xZdBg R2q222iNd0RyUVktHaDnogEdJSkT0STrZ3s7atCZfjyScWXfk3K9oxsiA83UabzB53WbNQid 2PO+RISIKSC3ZRDYOoFLHGVq6JOJHeHwW+GNLELE/kLNHGGu/UeYJHXhj600fKBRZOEkHeD8 1J9cqHShvHaxs1ViOkkCMvRWUuygOeJmbdttjtBcTU0cM0e+A5dTRoP7SmpIb5aXXr5GJbKq e+JWXQq87KlLQf5cJVcwDAHJw2Ebd6tJQT0qAiLITguQoxcxQUviBGsZLWIhTvH16CGr+ceF r2SQjWklPKOwzyE94GnsawcQQ7FJlX6efB3/GSD05Y043DDkGps5DBhW8vP6HhFlkQFAD8m1 d1wUC1iTQn0I6JGdVkTSsggqolpLgi8Zir5fkVyrq3CJj5WwyxXNlEvKdpLFkgslEv6Qe37n 2F+rAawBJ9H0VhCuwnpWb1W12r/Kf645ozVqOGIw+JvCRvvDtBveK6tF6ppIvfz6x7/+Bd5C J8/fzqtvfK+7QEmCNYlJZHPYbQmwPmy2gxH8x/98gsfFz/ALON8s2fy8RBlw2QaJnVwq1TI8 Ed1HlMoRE1xz75bhzZ0CDVwh9+hFRgRhjWiR3zfSNVlbUx/vPRhalTiYIYS1G1Md1H5F6mAl mJflDlzJnlQlFa2MertxRGXAHCZ6UwICSdNqMtB419N47fFcr5GMVtCLNuqWZJ8SbZZLkqGE tg60W9k8e/c1Dclf6zU+0CeUbYiyDb8T4R24Lh6ps6Z9WFsAvO2HAsUzlG2Y8rD2U7bB1LaB 1K7lpazupU3qiYp4wvTBxGfahPMEu2UG3K6SzpPrwOsue4io3d95SfZWBZE8grXNFhr591yO +ZQzchJbyJ3+cnUlu+xxU4w33ocIG1MbSLhQakEP0Zq5UGy/uN9nblo2tI9WzOVKaOnp0mcA uznNWkR3YQMcDJ9l8nOmIPNRPlxv4TlWzgF313VXF5HdGpzzY50yOOzk+HEVYaMR/Ux9ng0Y WS8AxOpJ+bynz9S2RT/bCWuMJqxU9JPzp5Q4tgGV0GrVEOllwMIeoaEVsMWxAaycB1TJrA5K 4/jcV0Sy089KJWUlSij44xvB+wM9c24En9noApWmDYgulpIh7h8L1i3g9smnlx8/FvH3r8+/ /fr85TfX66GO08RhnSns6E0apZ9LKN7wTneig2SlmhiQqA1RgsgTDQo8IFRJq1Btx0mxY2MB 5OxIIR32gCdbVDaCeEJDSRa4IzrL9XJJLEaOrKEHO6lIsENG9QgpGwdRNixbB9+gyvF8AU/g 8GGqv5zVsXWwIb8AzpbwnaSSyBzwPB5q+TaJcC8JdJlS5HBOhRDtyC5ZHntJrN3vmmOEjwl8 VI+kPHEVkmXzbuNPIkmibRRKnfRBZUek7q/4PIsWHZjPo9a7vuOtuPZUBOUi9RpH3LCZ0a2w jfoAOmVliTczgDU6XqJxivftj9eg6zcropl61KvyZ4odj3JNLXJyS1xT4GIPubyjYVHLfppd CnLvSFEK1ja8M5QxfMknmBB8wZPNS3KDmZHo5hTva8HwOY1FFUmTyXbqflkto808z9MvD7s9 ZXlXPXm+MLt5ypPdtIMPVPchv/b6hUv2FFfgxGuynzRIz9J6u90jD70W5eCjtJfYl9Zju1pi bTEiRKudj5BfLti58IjTM3YCq76Q+V5qE7bbrHZ+yn6z8n2k7ie+khX7NdYSE8J67U2qe1hv ffVVJMKH1o1cuj2EMru3WPAZCVWdlSA0+FIbLN48lLa6szu+5DqRpOTvrf+u9eNwQavPEme0 QfdGYjA8ysGCjshHqGc5DgI64WDBKf/WtY8olxlWm4MUhzh4lvAlyo9ZXFUXHw1UcRfLl9VE zeQ63mYkzsJUmgxM9LG9KUq1uibnC/emeawSMInxJ3orvLUosobjIIYaZXWdZyormxInxZb4 CdJw8sRqZoPwjdTrL8UV7c8ATRTx1anzm5CSN3Myskxm9IcNTeorwUQky+A4U8KhGmrUAelZ yWQ/ml6YCOvUh2J7tBFNqhjfXR/x0xHfbZngBlu3ELgvvJQrlzNYgS/pjzQV75YlPpLgaXbn cIXJQ2wL7K1jSk4ZZwcJVG9tEyNs0TAS5Ra84ZWvDAU7qWsHvrLDDf+qiUMkcFHto7W8PPm/ 985T+eChvD9n5fnqaz8mtkt8SD4SYOm9epuqq1kagPvj0VM7imJphlWnbcHlI5oV9LM2AEmy BGeDSbwGBYOPdGrxzgARzqy8E6s1RLvE8sGh6DlGtqzccm/s0aZmGS22oNJPIGjn66xpOb5H j+ksFQ977DabEh/2Dw8ztMMcjU4dHjrZdBI6mGj0Bb6nSMhXsJbvEt746fE1Wi2xjyBMTJ72 SVucVvhMmdLbVtS2AwiXIfhxhh78OE3fvJnD5q0sNuE8UnZYYhs4QoPZG9vIYOKZFbU481DJ sqwN5Hhsd9E60I3MjsdPPFVVimU8TJO7x4jEnycvXsv3oWKSeY5SAh+uhld/px73XIZgk0j5 crXah16WMuaWGD0TYiFWq02AZokRpHbKrOOBzykuD6tA+0sxVYeE9VdQKnd67bZbBtpS/W4g gs0M/c4D1d+Cr8T1etv1rQhU5DWJV5tQVc2N+nvaKtPnYBPd5d5g1YVoh4duhrbc+qdPoK2i GdraT1OGeFVRV4K3WajPrNYP+8AspowQTyzHUVVses3Kd1jUtenrIkzj7Qwxa69NXIXpemQG yWmRQPOvljPZN7rfhxlS+56VUwi4niLXzTcSOlXgTC9IfgehRJOZqshn6iGLeJj4/qltqpLP pd3K9T3ZbIkYaDPpOSCcBhNPMzWgfvM2Ci2WspmU/FEFydFy2c0sZZpjM0fczhEDwkedYKUS poh2Fa0DM5+4NpvAxCK6/W4bKmYtdtvlQ2B2eG8J8ZjWVOdCSyNYWWF25Rxf/NKYlItW2DUj Rum8RihEDNCUuGDEMt6optbdUpanJQoMo8NLRH1pnELqOaiv743/taJg+42bUVFf10sXlrvc EtveavRUR8zF4I5GltVZ4yW1PG8dXZPJpM2Z6OO2FC6F9w1s7bLIJl2yJ7nVKQ3ZoXbtuwPR 1k6wKUXIGmrQkN6zpmBuyk9yGoNrHxacFKvlwQab7HTNwZGulNJg++3S22u4pVQvjlb7MMfV qwquk+N2uVvLPlBc3U5z3G8fnE1JfS/m2q6pWtY8wRXsKnVZtOjaVyXx6aZpegXvfb3XVTKz tMvXvtGkYP9w0iTPeOKFkJlc3XZiayLXEdiXB5wVXOLUf1Zg8pJrG2xURS5/xcypIVElZhD3 crfPnFpKm1u0k7Py2SjlfOTddp78gMhKkX1+/v6bikXEf64WdogRtUL8ORPk0OJQjz3fLzeR Dcp/afwpDSftPkoeVksbrxNONJkazXkMqJU0OdbTkPGN5ElCQnB+5bzQJJT7ai2PJ1Zk9AMG pC/Fdrv34PnGA2bFdbW8rDyUY6G3Gdo46/fn788fXl++uyepcMlrup+KxvXgta1tWClyNkS4 GjkHBh8mO6Uc2MiW6u7lnuA+5tor33QppuTdQU5DLb7nK8dD3QrjCFK+xZXXbuL9b7A8Je9N oIlNuMTtJcU/5P8b2Z/A3fT/x9iVNMeNI+u/UsfuiO4x92Id5sAiWVW0uJlglShdKmRJ3a0Y LQ5Znmf9+4cEuCCBhDwHy9L3YSPWBJDI7DU/Z1dpmWT4Ui69uoYzP+rdfdUMiTQGU+LTUw6L d3HI1shVneJpfkLUc60JO+9Vk3XNdaOaiShU4221dnvNN8Wq8ypp+KZrjsiyq0QZKg6v/irH Jlfz0wWHTDdS968PN4/mu9yxuvOkK69S9DBeErEXOsgn1ALzvNoObB7lmTDdy1uMemKsRJBu UAnC6IsoG+SrSk1OfU6iEnV3PvKmZP8OYorueIcrqnwMszidUcPkA0zx6OmkwlZJzTtv0yG3 TwrPDqAwDB4arVXHtyc9hPhFlXUssaWxY+UvImfaxDkn2ntxPNBF38HguKDjVcVsFbp+ef4T MJ616FjCfqLpn0xGrpLBx+YRVHwwcGiYEm2wRwIbDlVAaw/6rA6sEWNpWqvvomfYjQoGRxF4 X6TTH0REVzsGy1qzv44r2ec+2cNXG2mP/K84qErZH4MPAm2TY9aBNOu6Id+kaSGL3RANkdlK YCuQwGCUySxdjexaz4jAsWVYGkOOd+Vz2ZIfWcFezfWV3ebhlI7KJ8p6xjE53BRgUA/XR2CR 3t41i6OGodSirQo498/KHFnTFzjfdBTpWRg1JvX0q2J0aiUvn3bIFrOgWWEAowP0UeMXqeUD z4qdFuUSXMVn6t2cLBxsW5qdEpov7LrN2hk6w2TIhaEqJ1lpOY0gkIeDBUYG5VUYr61K9i1S gkFOjDt/E6luKNq2BLMPqi5dohq8BYWvsW8o9iMGiecn9m8vjBZjcerVEfwF2+KWgKYWUS0p 1fv0kMOdGVSeagt6jz9IAKqLihEQ173am1aVgjeyNbJ/oLL18dT0Olmjs9l0TyVPJzvkGpCq t4pTxqz3/etWdUiiM9ohrs7ie4+8TLFpbhAm0SPSoSjLq636AntCJo1KqazjpYR+FNrD8SoT uhbgwl4ZVlJRtk16DePLN1Yb4mB1HGbHzz8e3x6+Pd7/5PI7ZJ7+8/CNLAFE0u7pJ5RvfTdh 4NqInwRR1GnflSbR5XsMHvISvOrB0zb8YVK9AIVNyn2zLXoT5OVQK3jeSIJr2eVbpYuIdMVT 5vg/4Eh28RFhipsy8cIN1Ql9BiOfAAcdrLK16i5hwc4siFXvzSMDlnExWKALH4EgLxwSqbRK AT8WAYZqcYjokSAvzSbWvpIVfC+5McEIqUNKbBMNGEMT6wjIC0HpgThtC7rCWVoVaKy8f3+7 f1p95Q02hl/99sRb7vF9df/09f7u7v5u9WkM9SeX82559/4dJ5nCGDQ7Nt+HFftavJbDMpRG mrap9ADoBR3n8io/afVsZn+RV63qhwWwRtPtEi2ZJmoRlpUdOL4R5LlTi7pol6rPtYllAGXZ eW7If/Jd/TOXiDn1SY6Km7ubb2+20ZAVDSj/HPXpKitr7XvbpJPnDvgWDyguNjT97nh9fW6w kMC5PgHtspNW133BdzBYDUj0qBa8aimum5u3f+QkN36L0mm0HiGV2MDuHT4rhlrrj1u9noX9 dPoZnje5c4Om+0UQmKl+EYR3VWqfJITxZWPV2h9VAFclTNoDkrtpPtSqm+/QqIvnMFNPVLjv FfK3ItZyzDCBBOBk7BqBuBcAgns9II1sTQzybuxJp4LKlmPGybDYxSUYsk/dmM9NjlaCQRhR xNA0BBTs+qr+UrXn/Re06st6nsa5hl+aZeAY33Wa3wG4fBc1N0j7+vL2cvvyOLaM1g78H1rS AevLPPIGdcfTqmc2yFj/QTgoXkQIebrKCs0l0gI/PoCn+aUQB+HbTxwLy0WzZabM0LboIIv/ afbKOfaYhZIKipiWBTjEuRASKvVucAlTZgVDBqdmZuxuc55/gwumm7eXV1MI6Fteopfb/xBf 1bdnN4zj8yTwyZny+ebr4/1KWk1bgRJ5nffg9UpYC4BSsz6pWnit+/bCC36/4nMRn0zvHuCR K59hRW7f/6XkM0pIY+u0D88Wh1VFXakq5xCP/7YAo41EhVAO2GBOGXOialUy2D7/BG6Tq75L itJk+Gai665ORa4c2UyctgObE+uaAe0BJoId665gudTCfULH+5cJb1N5WMf31tC40obf9N0U AHJ5p57oChD7WBwjwh58tPWrVRgEIGpLJCVdfuPkDSuVAhXa4M4ih98/vby+r55uvn3jIovI wlibZGGrrO21tLLLpN1q2Nzuhnwi6EIVEgVSXtWDpu0s8NMQh6EW9nqYOyYfHX+OpYZ7mQ9K 7jp8UwWG71QVfMHs1i4c4mkl7FVzRLJ2jUJzxJe2SWeRUJTg/uc3PgqJ2tNfMyht4VCopxdL 7Gl8E4VryKU1d9kvytEV143qx0C2o6YzJ0G0bgroc1Jfn/u+1GBdZpTN2vob1QiMvLuGm9iN q6drKD4JVFdamsDNJpinUy5lfvy9+t5FNjHMBuIoFne5ik8Ujd7UXZb63tLUsEx+mKVsVD3L KvX9OHbUVF5ef91vqrT1fObE82Mhtv04AhI/p1d16lNS9yzvKERy7p//9zDuR40Vn4eUEpt4 i6I6IV+YjHmBaj4YM+pGUmXcy4oi1HVyLBh7vPnvPS6TEHrP8CgPJyJxhk7gZhhK48RWAkyC ZuBsSnmCq4ZQdYRw1MhCeLYYvmsjfGRhCVN8a0C5u1BDrSOHTnkdWwlLWeJcVVuame0Xb43N 18IZqbBbVl6py5WKW3cELViQhYDY9RYsHUmW8mW+531CkchHBQjdJdgIaynB+2odG1M06krF YxvuWnDPxNmWmSDUHbL7rhH42kMn57XUCKHP3BMOOrlrZJ9cY1RnOwdwqdfN6i4mU7AW4pgE TyzeOESMso3X3trE8dZrSaYG51CqHxlJ8AoI3HAwYwhCnXlUwgvXNLFWj8wUIoyppPhu3Q+I lEZ1orVZt/vkuM/PZZ96m8A1I3b9JlBFGjlOhIl6RZpdQPjZIzuzkkxOTFVDQMbYxZ98FciQ kRcBjgcCXLY0tkK1dNFNXG3XrOnYOeHb7eP+2B3VuyWN8gkuW/uqereCB1Y8pvDKddQniZgI bURkIzYWwqfz2HiBQxH9enAthG8jAjvhWojIsxBrW1JrqkpYuo6oSryI+xwpY0y469DELqnc 8KDPr3M+8ASQVSlVgq12hT3i/dAS5cpY5DkU7JKfkYGBc4aOVSZG6tUlGVGmIrzgQuKW+Ei+ N3DCHU3E3m5PMaG/DplJTLqkZAl2fC+hXgNO+L4M3ZhVJOE5JMHXtYSEiQ4kN0DqY5GJORSH yPWJmi+2VZJXJN7mA4HzHOT0RFR7SPUEOK2k+xzelU3o5zQgPo13zM71qL4jXE7vc4IQc3ZI EnwFcmnCcy0xPM+zELY8vMixEUTm4smNayEiJwotjLuxEFFME5s1iUeRv7EQVIMIInQshCUP 311viCiH/kiNfr5R8sklosrrneduq9TWFfmgHIjOW1aRT6HUnMtROmxIomsSjSk0JnOLydxi MreYzG1DprvxSJTMbRN6fmAhAtdGEEVs03jtU70fiMAjil/3qdwAFqxviBWoTnveoX2aWFON wgku+3s0sXECqnC7ONwo39nie+E5HA2DtOHR3cPjQnhkmaHITiKJRUdeVZBCs0JAjjPPWYf0 TOIHQUAP2jiKiZJwwTjgOwuiGo9ptnGoCR8IjyKuy4iUFdihpyZdDlODn8MpBet3yLOkUOXu 2ie6XM6X8MDxScJzLUR0iQylz7lXLA3W1QcMNQ4lt/XFbKkoM45seggjoXYHtslIncYpmSqK SIE5db04i2nxm7mOS0ZifAscW4g1Jc/yWomplirqxHM2NE5Nz326Jrpzf6hSap3pq9alhrfA fRIPqKYDnCo9+EBK2yMtunAyiiNCMDv1rkct4qcefCOY+GXMpUg3o4mNlfBshG/BQ6qHSQYk RsutjRKwXMdhz8jUORXVe5KKvPVhZ2NykppOiD/Q3ph7XtoW1i1Lf+HgV/GwuiSl6tNXAPpu e4LB4yKYLDj3XaFakZn4yQDpvjmBT6H2fFkwZIWaCrhLik6qQZO381QUeJIhDVf8z1HGI4Wy bFJYPIiGnWLhMpkfqX8cQcOVvfhB00vxaV4rqxkIHPOKVx3qUy446xcx0zJRxydfxc7tBRwW Vu3c4k84Hjw4yno+DTVsp+vjoABUfD9wBpMQHXEqcJcbWbbpQYm0nAiPeqrUyIMXzw1jxVZo b8sLhpfnh9vvK/bw+HD78rza3tz+59vjzfO9MjBUzRVIgoFxXEV4ALcwcNmMni8xYTMVjHSr WZqsls5oeX/bFdleiwDWE/X0ltlHCWD5chGXN4MezaqKApzQb50N2NNfggMZyUvWcikrvQFo bSJsP96+PK2+f7u/ffjr4XaVVNtkaRHhI+EJJWE0gEBldaUFWWUoBG3Lfg7Bq80eYvn+X4bZ g1X5tKp/HVCrLi0QqachVEX/+vF8CzoLVu941S7T3jYAIqxOOaoAIcLp1sIUULP6pBBICaet wB+rPH1HOY6HTeihgsDRdSogcCo06EUbQZyZSqDigXeUNmFF6uPAcib5cky6C0LPr2xTfA8P ALrjXtIoW9XoIcalAoONRHpIwInb47RqsHlXTuj3x+Jz+8hXDwYENm3lFT/j14N8AYvqmroE BRyeCWPEvPOYH/2iw7oZxTU5XlzrHqMhq/nWWAV7Npj1IlCkRyjQOuwjVRoHkBXBOhqI7FgV qvLqDMnyosphF1exG6iXRsl2CKePwEGvGHrQBhiyY4LqCFj91h/uSlxHvcGRJgWQsGUYGRD5 GLcsChoT6MbVRuKEmmMRzCSvfaIay8oP8UWsSIj2/Cl6GlZSAWhSsXgnQHNymQhjvF9WIWxZ 3nVMtR0isXjDB4qJxQYGsjKBEfUzKVnMHjOXo4XFLvL8Jt9qSXsOIR1ZnpqyR6evSwB42nIU D9FqdkSvQZYwIB0K4fDDUEnax7G6w1WoLPQ3McnUCdhCIRhzAVG+W7t8RQzy3KQxLsXwnVXo hyFZcjzrLHjByo3vhBaKb5vchOJglK5dK+PRTLz2BhtDF7vsUx9Zal0oOJIK48hCxVGwsVKR Y6M2dGsISj2hVahxVcVTAeaRJSJMxRs6Q7540Y2sL2sLg+04qLi+Qinc7nido2MyhVOVbBb4 C5jywjq6C2msOQtlrhoLxyfa0I18G6ctAJjzfLo55aLg+XZubU0TLxs6F9jjIc20hdMV0RCD JkphYH325Kc+dXq6v3u4Wd2+vBJ2qGWsNKnglanhBlCy0oDouT/ZAmTFvujB6JM1hPBjZCNZ 1lnjpTYGfPt1YAOmU5+CZrnQd1V9QwN0CkoP3JSC/Y9EXXEWWl14JZpkJ3N50cLIxaUqaujB Sb3PqbcKMmh/rNVVQxSpyiuP/9OKDMyuTNgBjGaeU/4b0z5oe9yBxhKBgiMxtieIUyXOLqgo p62Jetq0tOC8vE3LKAbyhmYp9hRrLYBnLbSHy8b/0EoFCPLv3vewNZW+dnAweEaaZEnbg9H7 WGXAMiPsqEQrstkjuhg0xlavS/X5mkdEsye8WxEmdpArJvURdtEJ4AyhMFznc2yEd2lowSMS /3yi02FNfUUTSX3V0Mwh6VqSqbg0dLHNSG6oiDiiauBRterrKFXsQ6Ek8hr/fSiG8JB5CCvQ raosE34VxcP0XGgrcPF0KwlQ9fqTXKjeHN7b+7g++i5Pqmtk+IhPxkW9berMyKjYN11bHvdG ofbHRFWF5lAP3g0KXF1l07RCN1QNKLQnUEZyRhSeg+fJUjoAuf96e/NkGqkRzoTFrKTNLhpB G9wXBvyYfGCqQFWI3hqJ4vQnJ9J9Vu/LONK8GIvUztu8/kLhHMgHkmiLxKWIrE+Zo8otC5X3 TcUoAh5WtwWZz+ccHkh9JqkSbOlt04wiL3iSaU8yYIMwoZgq6cjiVd0GFLvIOPVl7JAFb06h qnaBCPUGXSPOZBy+/fFUy4CIWft62yuUSzYSywOHJuoNz0m929M58mP55FYMWytDNh/8CB3H Trl2KrRTkZ2KrVRkzcsNLZXxZWMpBRCphfEt1QcXUYGFcZEJD5XiAzym6+9Yg6N5iuKCPjk2 +T64a2jiiG1+K9QpDn2y651SBz1lURg+9iqKGIpOmIJKC3LUXqe+Ppm1l6kqOY6Q9Vhi4sl5 dZx4+aSmfc9150eBnjNvlct8a3wI8zx1QyzT5ER/mu8qn28eX/5e9SfxosNYG2SM9tRx1tMT GmH9zRcmQWh71yXlmYSaKXapVU4+ZDwo8QGnghWNLvLKvhk5hmM0zCapekyFOD3Kvlkj+6cq ik9KEVM2CZJ19GiiXZwzepwtG+LT3cPfD283j79okOToIHUHFZUbmHeS6lKjgw0e31IOelIj bI9wTkqW2GKZm4hzX0VI70ZFybRGSiYlnbb+ompAfNdc7I2QdQROfBKrZZtjFVshytBJTuRZ XGFffZD6FDQls3DWVN7Hqj+jw+mJSAfU7ya42qCFcEmfb8ZPJn5q146qA6fiHpHOvo1bdmHi dXPis+4Zzw4TKYRYAs/6nstJR5MANwmqDDc3z26DbBRj3NgvTHSb9qcg9AgmA9fMRMlSLqN1 +6tzT5aay09UU+26Qj0UnQt3zSXgNVEreXqoC5bYau3k0R/qWirAp/D6iuXEdyfHKKI6FZTV Icqa5hHy7jbheeqqirpzL+HCPNF8ZZV7IZVtNZSu67KdyXR96cXIaeXcDKctu7jCuOho5+0x 2+c9xaCdNauYTKjTxsXWS+GoJR/SpqUmEp3/cFZJmOxgys7rD5i5frtB8/zvH83yeeXF5tQs 0emYiqKo6XSkiJl5ZLrZVx17+etNWAe6u//r4fn+bvV6c/fwQhdUumDpWHuFBY8D36d2O21L yAoPiddypwpnNNpOVR633N58e/tBHVPKclf5VW4s8U3ZROgpzLiYXEbGannddElNgucs9R2a AeHLiSzk9nhtS8+1RCmrUt2ZGlRni5icWMQrgC3P55Uq+3QzS3WWyitOvXEuChjZd3ZbMvwh H4ojuD6sirqwkE1HiGnVYHTCrPfdMPzoYz798/719eHug29KBzc2JRI3tooqsaqoPp5uSwuL aUGED5G+KIItWcQxhVkDn7clHzbbQtUWUFhi7Ao8r4XG36n1nTAwxTUeYqSoyFWb62et520f B9rsziFzJmJJsnb9wAKTnzlxplw5McRXThQtjQs2Mr+u2SZlj3uUIlyDMlEibTlpImRyWruu cy6Uq4QFxrUyBm1YhsPKlYg4nqaWqClwQcLJiYRbUML7YPmSuh325Qsm3Y+WL75Z7xtNEMkq /rGasNH2rg6oGkBJDTYDzXqQBMYOTYus1Yvj/D06+BWlyEbdPYSyqsB+mcfLgGMLtq5xnwpK xX306PhWHzXJLj+nqWqzBPQr5c0YhRFWRiCf+UqHzma58RGGRUtkWHTs3YfzKT/iVMWjckuS p0I+uTVBcekh7GZGgU7zYmjNaFaQlA3k1MuFgqpKP4FC5GQRTdV05hIWUFjEkrd+863LO8b7 PAnX6HZWXhIWwdoZ8GnziGk29jC2xNYPl3Rs/lKdmJLVE6i6WD8jzNi20/PmtV2I34xCHZLu ggS1g56LHHV+IRYnsNeptWOxKtmoR5NKharT4pgRny3XTnQwg++iWH2eKWGpoDTNov39z5vv q+L5+9vrjydhrgr4+OdqV42XY6vfWL/6evP9/k4xMrgkFS/WS+aOtHt4vQev5KvfwLvzyvU3 we+WqXlXdHmmb2dHUHeiPF0Mw0nQZNJ86sW3L09PoMEpi/zyDfQ5v5vihOcHrrHo9Sf9QnH0 rgoFqbAhNH3aJZcu1fEfgs+nE961sCKpecdCNbDg6pK6oIvrbWUdvHm+fXh8vHl9X2xGvv14 5v//wdeB5+8v8MuDd/vH6q/Xl+e3++e777/rSgJwf96dhFlSlpdwsaHrCfR9oro8GmXMbtQ8 k2deP2BjcXd/+3InMv/2+sJ3F5C/MCH29PATdYGpAZIjGlYjnCXrwPcIeBMH5oYgB0/IYUri nhG8Yv9P2ZU1uY0D578yT6mkUlsrkjqopPYB4iHBw8sEKVF+Yc3a2l1XjT3OjJ1k/n26AVIC GsDYeZrR94E4G427u4mW9vZVIlbRcuVCiyi05j59ymAWY+XwVMbGU8Abqj9tnTR1E25E2dgz MTzK3XX5qDhZuW0qrlVL6xAkbK2sQMmgx8+fLk/ewCw9btYLa4KFcGwXB2aKwdYBrtYOcG2B 92IRhO45nr0UUrCjizYrw/PSvAgMY7sY3WlrGA7R0LVd/UOknnNr1YaS+mAIsqO2N8FmcK4d liS2y9c34vBUTLxytU2wWbhhZ+hVcPNn8vDl8vwwaQLf3kQNMmF3K0TtZMtue1zc4s8fH17+ 0eLVSv/5C6iA/77g2HKHlmqtZPsmhWSjwO5ckogjQ7X8rmIFdf/tGfQK3tp3xordYbMKD7fV 9OeXj5dHfKbyhJaQL4/fYJxwflquQvU6fPLMIbXn3Q8YAO8gtZenj+NHVZ1K087F1Yi5nq33 YrfJJS+HhfHK8EZB3ZaGv0zCmY/zDa4zrYyYXKDfQDO54yJ0c/UxNGTCoFbmg3ydIk/ydWpj XPU0qK0/re3GQ7XvVsvKXWjs8YG1OUXuaWkgmg9udBcDOgejThxuF2+Qxi1zkwyADbzsNtYf 0huknPEFb5GeL0vBDRkyuC40n60Qbr14g4u8XKirfsIFkScv77tg4WmjcSC3BEzO9Pxqcksv Vw4FfKgbUrHZTedhk+VSxAtfDbAhDNart2Qg8BQmTxaGk26LC9/gojdT9HyZ+WsoT2AM89Ve HLcCD/E8NdT1sG7xiZ3gYbDyiCvvtkHkEck2Dn3pQXtFi0DflTZkqwzSAKpoGRrXZV++w/j+ 8Pzp7l9fHr7DkPD5++XfbtNic3kiut0i3m7JzkS3W1vHRnhTYrv4XwtcwySHoFCLqYjU43BX tj5Kw7v/fgerJRjjvqMDH28G03YgZ3izuknCNCW54ZOAqrPX4+438St1AJOapbVPJkH9CrUs WBcFZIfpQwE1pXuwvoG0VleHwJinz7UaxrFd/wtX/Yd2S8n6d7XUwqq1eBFHdlUuFvHaDhrS Q7FjJoJhS7+fZDsNrOwqSlVt5Ip/oOGZLXPq87UL3Liaa2HLw0DTEaBzSTgQViv/5S5es2Dt qq9NoItYBwvRX5Bj0cAgSPOH2GAVJNw46gFAurfZDkT+i/XSsLZ4y/KSpFINnS1hIN0rh3RH K9J+Kd9hfZU7N5xYMFq+LJ1o48ws6Q7yJJnkIUucCipaW3KRhqBeWwe6DOjWrTzBpWfHCgyd IN78d6ggmn88Vx3zTJeaZNKCXnnB/haHC1f9hM4mprpK6YvNdXHRCUizenr+/s8dg0n8548P X3+/f3q+PHy9627y+3sidXPaHb05A9lB781manW7Mu1xzGBAq26XlJF1sl7s0y6KaKQTunKi ulEQBYfGbairyl0Qncn6eBWGLmy0Nuom/LgsHBHflodcpL+uCLa0/aAvxG79Ey6EkYQ5nP3L /yvdLsEHdyG9maR9Cqu/x9c7tav2e1MU5vcAuDQ83gZabBZeSltoZslsDn9eSd/9BatIOU5b g360Hc7vSAtXu0O4srCG1qfESAPji74llSQJ0q8VSDoTLo1o/2pCKoAi3tPhhnU7mA1Fdj+G BSSZNfEhXC1WR2sq1cKIRaVDXqKJ6GFU24uIUd2T1F14VT3d09PjC7ozgAa4PD59u/t6+R9D YIzjt7QvyzMoLsuqwP754ds/aCLDOtZme02lww80Im54UAOI+BNASOgnawiYzobkw/h9p02S j3s2slbzFDAB8sRo3/Tij2CtU+LEO/RxUGsPulP96At+jCVHzx6CG0HG+1JMHq3M0Ijnu5ky PsnlQy2HQRUk8ablCLP49Lb7bvBdV+qm8afdNbR47t7UwW+kq6nrlhghkgMMtWsbF7wwDqFn vBoauWrfyqOPm0Rg3tJ8cL4mQ7INwthPsjSrKy8NYgKt5jrZBbKq+2PGet27rQTUScEfKyd8 dZoXOaKSprCVIyOj7vk2WNEyIzY2bVbwklfogfxwmi//eItT7pmXA1n3ciD2/u/Yke0z/6d7 0+2uTpWnfT6YDa0wkFRlpMCIal/imwJPZH1amDEx0dEqg+LvQ28MCW9BU43vM90zPBLvh4LG tKuTg/AVSznOBLkxo2kYeoB6nce5l2+PD693zcPXyyPpNVeLOkZIDsrw+a+Hj5e73fPnT3/f 9nvz54cvl7s/f/z1FzpbonvMuaaM5o4tu7kGw8yzTNGOqoGluiEI+L2r6w7nf473rBhDjueH RdEaB1gTkdTNGdJlFsFLkJ1dwTu9rSeuBV3V8CEr8LnUuDt3maPCIZw4i1vKXwhxTZkSt5S/ GCnndZvxfTVmVcpZ9UaKxmNNrK4sz9o2S0dpnEOPUoCmR18+ri6S49CABlScL1yx1m2NgN/A B5OKFwbR8UKWqlNOym0B+Wd2WHjbJTcqXnYCX1abMvRRyXmXtTgj8AVgbeKlQOVDZXc+npei 69zVA1Wrr4YB6VFEDUQCeiVlOSdNVMGs2Zf6waM0c3nLupKO9HwBRJBKYwQ+vjrylHujb/mR uYvNjW15AIosXqx0U6LYJKyFblHjA2p5w8kUOvT/4U3YPypic3Vn35CqWG9tRF7GP4ggy70V TEYmrWazGno4N1XY/Vm+g9IjiHxTB4y8rtO69grGsYvXobdIHWjxzC/TrL3397LIXSZo0JLr F1luGAwcrByzozRweI3MIJNedHXp0WimrSGJiKTXR2fsWvoYi1K4g0F56JbqKrLZLNL2iyMx 6WxazkOvLqf1KMsM5LKqS7OQuOo0TIbeMHn/bp8mpnqcOMMDByrTFia54pBl5kDB+nq8D7aL wYkunCipLXWC9WpU4CYwvHhMenwsktQeQRFUT4PVU27NkBcwxTJfLMJl2OmHIJIoRRhH+1xf Qkq8O0arxfujiYKW3Yb6KeAMRvqODoJdWofL0sSO+324jEK2NGH7up4s4DpbRyWJtUi3hv8S xFgpovU23+srgalkIJL3OS3xYYij1cZVr0b1vdq85TtNa5LZRpQdKdGhVgDDJssNtp24zIz0 peCMqYy3y2A8FVnqogU7MP3G5I2htky0tNImjk3HOAa1cVK2eRqjptbRgnmprZNp4tXKmUHb BM2Nc/lkubalYTxJS+m4ChebonFxu3QdGHee97BEYPprp0Na8nnSlDx9fXl6hGnSNP2ebrjZ F+b38iG9qE2LjQDDf6OocyhkggZVMCOuFYPczLjF4ILhb9GXlfgjXrj5tj6hD/er9gF1D2vN PMejDhqzg4RO08HkGVaTMCtuz2+Hbetu3j24lrao9y5raqLudfUuf461EKNlfNNgcFULnZm7 xiphRFil1NE8Qk1SmsDhlGaNCbXsVMLUywTfGe7CEBHZ+x7dz5IUAFYtYMJQANxcMaMoYQnT ImVYYFbZRNhdSMmqsr2an12z9MbHh9ZRLZjpibhuQhg5tQy26KVgA+rCVPwRhUakaiAb6yI1 LQDJMrR1MuYkpiOacRSZJCVnlO7G8qpzz5BkVn2GYDGKq1Ni/QNpCgdE2YQnMcC6IU3ZFNGI rjsVYyQO3HLmvDkUO3bKaAiNB0EJFveBnXLZ9MtFMPas7dxZMtHjYGP4LJtawZGVQC+6K0ET pG84JJuhxRSSMG/tHlR2jf7MQoldy1kx9sF6ZdxNuhbVkffJ2xw7ZmZeCXkV44WR4M56ta7K SaSTpUGsG7JT5RSG884JM29tKJCvliuad8EPDakR0JV8aFyYXMQTTcX6OA4WNhY6sIhip5AA H7ooMkz5A7jr1NmuIawSHOsj2tOuncMUhkrYItCnaRKTr0WIoA5nmGvZYqlw8r1YhnFgYYZx mxsGq7rTmIrGbMikG3KShZS1BaOVtpcOCUysYGc7oPp66fh66fqagKXhjFUNAQTIkkMd7U2M Vynf1y6MO9H0nTvs4A5M4En9OEEatBJBtFm4QPq9CLZRbGNrJ0afGWiMeu1hMHkZU1Uhofm5 C25PkpH3kFLNhgjpcbBYCIyF2hWk7SqPe+Jh4UZJtPd1uw9CGm9RF0QSimG9XC8zQQSGZQLW wZEbdVUczDKsAaMqw9WaKsHhQKYFLW86nmYELLMotKDt2gGtSDg0CpQc+Y6WadoPoYMIi0Pa 1yfQpR/lvkItajoGms65ADqXudJLyvN8+pu8xaxdCZbSwKh4MHoKOMNqCvlKYZioSsBmlCWa Xeb66sbJMprDFwaQbxJn2yfW53JkR/+2hWG7x6SVcUkfK/i+ZM6CKv7ImY+SiyQPp3aPvSza G2NUBDSemR47bJbKJGXtsUELIa8o+ivEfNc7s9amwbWJfjILUVG3mf0l5NHbtPKgz0IbbGsY mCGXHzLtzaHsbnTSjeYWKDCSdz4z3LOAKlUJiyE823DCOHvvgV06SUUVhGFhf7TGt2Y2fOC5 8XJTTk6SNLTmX9IYBq+ytQ03deoEDw64A5Gc7LoS5shg2mpNlDDXJ+7x3qK0n/uUQ61wcYHt OsWS81rlVEzpK57aOw0H/TUq/Lh5J+7arNp32gsxYGFlo731tL69ybW6BIPeKx4eZcLW8wkM z5ZobMWMgyVtPzigMc+NrNAtryuk21qUYI+CTgqZFfe8olhXN5iKgeI9Ct27tMI4/KIgrG4Z TRuWoym/z86ChG1C4/2GxNQrRROECt/XVYtucrRT3xmzqiTDWxqkAPj4T38NqbCaAB8gk7Qt S9PSgQTzlkR1qM3xQv22crbv1nFEKgeS7Oqetv/9mTRqD6sH48gFwRMMU/raUaZxbtUekoHy hKUkxu7EqwOraG4qwUHg6fdFQnxBSTCr6iOpQ8ylLc4zOuoza4OAH41WkiuuVyGCbV/uiqxh aWhR++1yYYGnQ4an3LQl5PlVWfeCVErJ0X1JnXcExuOUlgpHCSMTdzReBfOxvQmBHjTkA3sF DNfQrYpaFy8NtPLcZBXkuCJZa7KOFeeKKIsG+maRpE7QuLeg447zcJ3G+NxElgo3k+j2KSRR MLSBDdNYQXUELxkpRFsnCSPFBe1i1aRgpegrUuPC0E3y6SmtUNFkGd7NoNF1KDKgwjOSR8s8 pMykPmWTHbDNsooJfT50hewslLC0eFefzXh11Pqk47TPgQ4QWUYauztAPy4pBlPIbtrD07y3 3VArtROz9OaJc9OSGYIDB+E0oQ9ZW5vlmhErlQ/nFIY3qnSUv7nx0O+cuDp0nX7Noy3af3IO 8WoKZPWI1LTaAKCyb3m9jOeMDO/MGZFJu3CHhI94P6TIpisuxCIEPZjsb9tpBsZaVIhMjIfE zB4JVlWgDpJM7dxcrVQ7noZipVj2A5RtMOXZDu+6cEGyZu1ZGyZD685tZH7ixtMBOmrBnX7I 5Oy1aPi4o8ZWja1FBE6GHcEZGZMdy6kR0yvhcYwlRePp5TueP+GV5Ee800UnZDKO9WZYLGTt GykP2MC0TRRq7IDeUOv8Vpqlc0Yj0RbvgYG8j13nYLsOm1rAZMz1rZWFOZ1bNgy6HvowWBwa u5xcNEGwHtxEtA5tIocWh8hsQrrFDQObqJ2VUF+zTAtTv12Y3hldj+tWK2lRxEHwBgzFrM14 2hgvb8M6wvroNKdr2pU7MQeYpMRl3YwK04rfDEsDOriJ4ZRkdRvxLnl8eHmx1xVSPyRE8OTB j675ZQlSEqorr0uXCtT8f9wpy711i7eIPl2+4bVxfDUuEsHv/vzx/W5X3KP6GUV69+Xhdb46 +fD48nT35+Xu6+Xy6fLpPyHzFyOmw+Xxm3wM8AWdfHz++teTmfspHK2YCX7TWvEcxtrGmQBp D6QhEnSNmHUsZztquFiROQzkxniok1ykIbW5M3PwP+vclEjTdrH1c6uVm3vXl4041J5YWcH6 lLk5WJiTWavO3rO29Hw425CBKko8NQRr37HfrcMVqYieCX2I5l8e/v789W+3qcYyTSzjRXJi Tu1p84bsAyns6OqUgEtHm6Y88eZNSSplX0xby2C2IoiXTTvEnqEdtLciT9GNV1vfvF42jw/f oVt8uds//rjcFQ+vN3MMpez3JYMu80nzTCrjQQcLdVWcyRB+SohdKETk2EvHc0m8WSIZ4s0S yRA/KZEacWfLSmQ6gt8bZ14TGlrFCsfJbap6nfLw6e/L99/THw+Pvz3j1RKsobvny3/9+Px8 UfMfFWSeyuGDGFBOl6/49u2TlYtwGoBo+cM3vOtcg3QtrKNAYIXIcBmSC09tJQcO88+M9LUZ tevhyvT6lTyDmerEyNOVfEvQcWTfUBcXE2iPq4oIpowYqV2/QX+pKAXeuppDKomywjpCWpKF 7Spb0zn+qdNxoqbUifm8Rffq4KjFDI1ivE3QiqWbbO8j4+WyxtGdNo1KDtEycDJyBn3IrHFD sWgzG4b4JCvILqsedwOTssFNTaq8jJ10Zlrc1Ji8wzsdvHaSR66WYjbDG/beTbjDZyAU3nLN JKyF3XmMg5A6WJhbXl7b9WTx5Mb73onjjmbDqrGxxlmDf/PbsmmdQjjzvWBh/PMQwy8EYb8Q ZvezMMH2pyF+nplge/p5kPe/Eob/LMzy50lBkMKtCe4L4Zav+3rHQVEkbuksk27sffInL2G7 mVpsNuHCyxn24XRu6L0dpWLH0iOBTRFGi8hJ1R1fxyu3YnifsN6tUd6DdsbdDCcpmqSJh5Wb Y7k9zb9RY8PSNEt9c/1ZaWdty/D4qMioc6g5yLnc1e7hwKNG5Lubd4YvK40dYDCo3aU9nTyV riw8uqmy4lXWeT9LPN8NuP81lu4PT1wcdnXlHrCE6IOFW6bed27pVa8vtXm8uc/kHIazkq9J bACFZKBkad/ZgnUUdBRqeW05Pyqyfd2ZxxMSpitw4/a3nBpNg2By3iTriHK4EU/al6fkzEBu fuCImBW0yeUxXApzl4KdqXFXgc5h9syS+pnA83yPwBekDB1en8+OfNdKj8NmdusTa6HGCIwb C3RLSsAUTG445HzoerKmghkYHgvkZHw8QzjSZNkHWSMDafCD4An+E62oypG+i6CGpPEvmq3k wGphnMnJiu1or8N9fMeKNhnwzJSsQzO2LzIrikEu0EtdtJt/Xl8+f3x4VGswt2w3B23BVU0O F4Yk49T1HC7PjsZuZ8cOx9r0znOF1Ox5d7bvEl934SwvX2yy2W0I1DS3fnvRogfC156eJ3d2 UN/CZgqF5R3lYXnoYOcNg6ovR3UNXkC4W+1fnj9/++fyDPV/2641K3/edOxT6jastbF5C4/s iA3MsHEnl+VH+2vEotTaBMAYQ08l7NJkisdcBDsXvhjYWu+xMl2torWVGRglwnATOkG8jG1K iyQsd2f1PekX2V5ZJDOXs/IlBCz/fLqI72AIbGrBO6oT7d3HHAacsSDbRr1zldaPGSpf63tH 0Hysd1QJ5WNlJ57ZUHOorREXAmZ2xvudsAO2VcoFBUt8KHXbuzS43ArdsyRwYKGFHRMrIeOC uMKsI6ncveer/qXZmdG5nl+dJLarm5EN4aYq70fZW8xc8e4Aqv49H2e+aKdGd5NG67mD5CDD IMleNvdTsrXfIEMvKRvbRx7oIaYe6zHxcrNoGDwe0RI1ZN5vm9S3XRboskSldAdXGyFsNc/e 7rIqIavP9FWCc2g/LjPy6uEc+dFY57aQv0dPVdGxdk+1w96prOR7mOtIbQR399MUb3Y6FSxO N+45da2KvXMshTUNUFdVfMO1rTf2Y7qTZn1INIg63g/ZYVwqZD+6nD+iv3B5X+YW9qQPEid5 nGYCeOpmIjxYxrqvtrI0tuzhp+dQWDmIUD4iEjRVaZ0I47c7fLphxK6g+Ww8tpmdPJvXbm2i d4Hp5ZGRL5z2vpmtn55ZYywiNerkCpnTCoRBwOuDrKBXOzRLGncsRZeXJnHaifT/KLuW5sZt Zf1XVGeVVN3U4VvU4i74ksQRQXIISpazYU08Go8qY8sla+rG59dfNMBHNwkqOQvLxa8BEADx RvfXFKnTtRiyRyAfJxiFS8KOy6RKtIjHMI+ghPchsRgCbM+30RiJt6lXFdkoZHfZN62AVkDW 3jL/Bd+mYTCNwWr88RPGxXZ8R75ji801stPL5frBb+enP3XcIH3sfS4PPMSedM/0B+eMl1Wh WtiMfCqcZGG+QU2zJL8p0+8I+kCf5O1c3tj+8X7AarRe1oQYvto/CwibhelQBCoxVC0NnlrX 4RqsWYvfbbdYF/h0vycDS2t2bGbfg/YUJFziEiyjYEVc/2JUWV7TzFFjbJVwaa8cB7e+FnZd S8+nMcjt+3LPuif3XcPUVHUnJcbwbc0mB3Axk2a6AmOr9R717DGqLPAbMCfH22cpGzMBtGBk Wg43sL8BlT6mE5BIlWyArw2fj0g8jMWOaZyuoqTj3CGX/Kr0te2uxg0gTrIMdl9hUezGGWeR aS/9cYw6CjwXm9ErNIvcFSG17Rsd5hNU6Sb52jLDQYNjaMlS1+KPH+fXP38xlT/HahNKufik P1+Bb06jqb74ZdAl/HXUF0I4u2H4TfX1/Pw87TQwvW+IcSyGx2bmRCbW6lTHgUi3iZhPQ3JD RuSDQqteHpX7mTxp+l0n6nTm5HmsLPr57QbXyO+Lmyr/UKP56fbt/OMGDH6X12/n58UvUE23 L9fn0+1XfS3JAz2eEnMumunOvVffP+EWkPM0TLN0hiAoFb+5mNpy3U4+ET2mEb0AdP14VO3R JZAUDXqLfXqAa1Kq6kiapn5gYDTmArSNxFT7qAc7lpF/XW9Pxr9wACGsi21EY7XgKFafUQii n5OFZHHueN9Qi4UYoquvx/7CexyM6zWwUhslL+7wZp8mkhpS+21kFquDfh0IGqiQ08ls1MUK wtD9PeFoKBkkR584G2vxmJs2HmQwjt13ULx5iGta6lbm4ZOoAbfJBViHs+DoEdafTlBxN7J1 KaU8My3DnxNYs1FcTRGPgE/hMhKDprXUCnzXsmcE3pzAmJX4GgFzzNo35nB9vYefbWs3jcLF OmSFGWQ6wZrZpq2r9qPIk6nFDUtTUwmzDV1FVQeBryJLJ/H9wSuA2Azcb85Q6NVMJa1mGqfu kwLu2DP4TONfGXMNVleu1dIwZ9qx48/0CGeu7dm6T+BoPw2LyuXKnRwr363ViBVc23kt7AgA 4YRUFuOuvk49323WAUuzxznxeGzsJau7g6IIsrR892/DOP8gjH8vjCqDZFURq0L+dwHl5DAJ qcuYdnS0HEPXmEfL2b5b1ztzWQe+rlv4te4TAm67etxdaXDOPEuX1fCz42ubf+lGuvYP44Cm G41ptFDr7+iwZHO+vP4m1mT3G7PYJpS6biHJIDWC2rMHTnxY2ir3V/dfgsxMYHmK7A5ZMJha DLy1PTqz2gCtzQl/LBDnJPmGkMMC1jL8wfVnnicZJ29XRzQflCg7FQLM69yiRVDHbMKqDeu8 o2kYDZHFD/Acjd3gKnQajJyRbPm+TawvbPTjDK43EU04f8yjpj62N1S46sYrn0n1NFWAz7jj COU72B+7S/bhBJGop6VFE6VrCpTwkTdJnlafqSAGT7c6QYAZfwDgSRUVeOEl0wXGxLFdOwjE RvA4ClrtiaYKeGNde5gZ47AWWFowtm/qxzIxRxLRTD6vYwqOgpCv1CFAoIZY1DuUsaDUwKLN oFsd8comfCzhxIQFebCRG7eBmU003I4JSfNBFXvz4EnwegN3hNPDp5blee4IqBWHYLc/w6Ta BpGG8/cCMJYW06Oy89P18n75dltsP95O198Oi+efp/cbsgLrk9mKz1IdJikcT69TUrs+EtB0 3ss8yCWZ/KGOtvNBYHOb0A0dlq/5bEzYZamcg7rmDJGo+IP7+55J9IOmsclrkcfZV2zEJraW pZDsCjrWo4e0qLMQQg/NC6LWDF/AAlKKFhUxcv0OMAyEcsYWm19aDBKMRQlYAs8UcwssE+VB 9DGaC8VjjN+2r4vmmAGpIcXJcNwneShxirwONoo1ugOyNCpw9xdl5MyiZ9JRAbwauNgKmdXk 7sXqhCLcryWtRbML/9cyHP9OMLHOxSGNUVCW8mhKcdYKwwITB7YgVeZpwTKoqKpii6vbIsvA K41WlPJg9sVllC3xQhXBeBjFsKeF8dZogH3T0sOeHvY1MLN1WQlYmYn6TAtRZCjhTIAysmzv vtyztXIxuBELGgxPCxUHkRYVS11m6nDD175VxtChurxA4Bncc3TZqS3CUoVgcwZ29LCrh5da GFP6djBjthVMW/E6czUtJoC1Q1qYVuNrZWlaFY2m2lJ5dWcZu2giirwjaP8XEwErI0/X3OLP pjUZV5pcSOomsIgTYior9AKWzgtML9bJsiAsI22rEZ0kiHVoHGg7INO9XcB7XYXAtfdnezrg uNqRIO2HmrHMt1yXrqb6uhU/D8ApFWNKKCwNIGHTsK17YlfTFbBY00Kw2HPuiQnl30Rs3c+a Zd3Nmk188E3FrqbTIvFRm7UM6toj535Utjzas/F8U1sbUrYyTfOOTPc+2HGmJrmzHsss+47M mZM1sabFkqlB2+DQ1HBX7tl35ak1OzGBUDMlRrASjGZzruYF3Svjmjpb7ODHXN61m4amDWzE gmNbxtPExB7pOM14GpWqs2uy9TksgmrEc9UKP1X6StoBL/SeKuR0tRBCDDlLzcvmJHEwI2Hz kZguFht5u+1hKK9u/PVca6nHNZUPuGfo8aUeV+O7ri5zObLG+ikDyqeRVHXsaoYV7mmGbUZ0 o4akxf5ObRqoROq1zozycb3SLfpyGcvTjWQCj/fHGXgdaNbASiRJAieyA9v5us4gZh9HOyXp 5ynNJLtT/wlvrGbEuTfa6Du8bsEea4rWNZG7a4OZiDVuIVUtltora08QUi713ETVYym2bFHE yjlZvUtnZQ9JOXkpameVvzRJJsT6308QAE9ijlMslB8ommVjv2bqeRqwxcNa1ENyJNwsVS2W L7jmD7Xn4bYgn73eo1xaLN5vrbl9f9iiPIQ+PZ1+nK6Xl9ONnIIGcSpW8hZu7x1kTyFnCq0m EO7NLUTcVaTczgwrxhyIUdBOGyqvr19+XJ6lI8PWj+XT5VUUZpzzpYd5lNVzI6kPYfsZZBlx RIjFRFNNSJY+yfOSbPzEs2gQ5Flp2ePMdjn94/zb1/P19AQMPDPZrpc2TV4C4zwpEFyvtC+K vrx9eRLveH06/YOqISt9+UxLsHS8wVko5Lf3Gco/Xm/fT+9nkt6KOMmVz87Y2ejzx/Xy/nR5 Oy3e5bH7pJUZXl9r+en2f5frn7L2Pv5zuv7PIn15O32VhYu0JXJXw91Bdn7+fpu+RZ3ic7gV tlYGXuFRCSbzrgVC7moB+Gv5V+/c/Mvz6+mmus38G7cscn18CTkStN9xRqi+fNtNb6LbLk6v p+vzx0K+FTpzGuGqSJZLl3ROAJwx4I+BFQX8cRQB0Hx2IMpfdXq//IDDzr9tgRZfkRZocZPM Ogox+xbRKRQtflsoX/I/Lq+I+SKFU+KkSos4jeTVTasjg2484KBeGvT31PuDUsxI1iSHJK/1 x6aTsMpkac7j09Klno2OmxTRgX758+cbVJB0gvL+djo9fUetp0yC3R6Tayug4Y95vW2CSGQx mJWWRYZZLUfSfVzW1Zw0zPmcKE6iOtvdkSbHek6a3YlJiRZHsnJX7Gel9bGsZoUjJwvSMpBH wMEHAQIwpuWSwq1iKeYlVoe0jSIz/sBcb2HErImTcnApbqxGrqZZ7vvO0tKBWN0vraLpobBE w9q3VqOQKaFFU5EDjo/JFTYyOUCg0ioTS2Vi8KECYAYGifyeKnrrdir7er2cEUXKQyod7sZs SdjFCdx8xgoHVLQbrXSoVD5Zc1LFgdf3SiqUyvLTDjlJocHUT1mdKJmDPZS1VMSTGl0/1PWj dNhSFzUY0RdiMYZYowc5OHRpxYNXF1ZLUuk8kO48rBW6lMSiIo/TJIkIt0q2Bypivc1lvMnx De0G349veLMuNwE4GEVLa7XE5buEUGHkKX/kXKyS0E2qxBSvByXrQ4KcXHYigSKu1oq2IXUk Wa/r8XMTbJhpec6uWWcTWRh7nu1gfbVWAE7MHCPM9YJlrMVdewbXhAenbibW8EI4cfZGcFeP OzPhHVOLO/4c7k3wMorFPD2toCrw/eU0O9yLDSswdbhpWhqcx6aFfbwg3DY8PW7r07FdDV4v l7ZbaXF/dZjgdZo/Eov/Ds+4T/YoLb6PTM80dfDS0MBlLIIvNek8SKbwoqbNd50lx0nQdQi/ YyeED2kWmQaoY8hTCTz0KYEmaMdaH4jRK6gl5T3MXH0KaITsEldkniLWY9NHZQmrikftmNlG 4zF1uTsISv3lfB9g+9AURQjqA7pVEivw0Qg8Ub2TIGVNBCq1BMmT+qGodhQceUID6OBkFnE9 18QpGyFkUQsAuXo9+l7PNIm4VXuF66RqHjDxLyDbGI3nQZYmuXQITcPxPW8ysQQpiHWdtBVQ sG58BylJpkPgBmmEclb45F5NolVY4xFx/ymt+V6TESqZvWjugtXAhoU+JKgAFk213qUZ5pov FVEVQaZcLQDiQjKeDjkc+LED6QVlIgE7ySCbwJL4WQeWqYqCDlaAJa0M4klwMH/YgYBaxxEY XFlpvHfTMLI610EEivopblCaYLjXUXFrJQZGAZpPQ8MeRJmKufdsi3qXPDawZ0CaCd0+NA6w F+5WdSzJs+IBN3uRzOSjEQDIqeugmlSqjNpa3OHQygQvrCetqBNtSb46dNQb4esyfNrYasSR IoUMDvZQs1A84pOssiOj6avEimAnVtWpJoHP+LJI8sA0G4aPiVUCFR4DW/MiYP4WSE4crpcH 0e7TSWlKJrYGuJB8X6mDLvBkFO5rQuMfbauCJf2oRvQXlazgmjFoGqYEY2i9eSKQ5zfgEGYr lr8z6kFdmKzUqQV1UlGEGjXcKNuBMlFWFLA3HrjDQdVGyGBCFOvWhKiIiE0TyAavoy8vl9dF 9OPy9Kfy2w6HTcO+ZogxUVdFIp66ZLWCRMq+DOvq9JJjoNfW6QJEcZQsjf7wbds5kudv51eZ 39HBrSoEv/y8Pp2mSqwixeQgFoa+5dpodIPHRhoy42oNs7gPOVgU1QzaV6pXnuNbZbMlutjf BGD13rofomZ7bYCkddEp1nszhqCi54XUzXpXUKkxSTxmKmgYnRVpKByonZ8WUrgovzyfpBnX lKFFxU6LAzoVCFiscA0kjRW5TnCwiC5vpSbQiTYvTReBDT8w/J2waLBL09UJCrjOirJ8FAtX qtZZJUoBtD3Se7ncTm/Xy5NGRzoB+n9pBoXOThLlU6ypWoFK5u3lfXLrAH58fuEf77fTy6IQ HfL7+e1XOAZ7On8TX0NjjCzG1GPa8CpgM/5/wPRcU+hSLt3WVfK5K1b7uNhcxDteyaltK2o2 xaE76RP7cFEnOdY8RoHKpIJxNCBW/iQAUJJRR5tYvOdCWgZRMs7cxL5/KIc6okT2eUeYLLoE kr9uT2KEa5nBJ8mowE0QRw0ltWsF1OaxBavaXy3tYIJz5rpYS6uFO7IrYi5fYL/HKRbCsao6 SSUBms4vckiD7tbpWgop3J76JnGXlk5KXwAel/dieOHwGfsglj5VVekv9+/MQhaY+MIoZJHp GuOFL0bpFoRIyLUPskVQUnxYIQ+m604QHDEjEZHBMeY9uXjlWL478niFH6NPO9MwbUyOEPjE xbgAVq5rjqwIWnQM4ISOkWNgQy5e78QcbFEgDNz/+j5Q+eODXVAd0Os6j17nWStz9EwuS5bO koZfjsIvVza9LvSX5HllUflqhVcYctNsQufEH953sBUPC44mMb8DwMZHUSwqbQsraADgYCNJ sYJvfjd9n74pD/ZLohn6OQLPKqXYYaYk4IAfCF6DVlNk+OYEMy2fE6smgJltu0eahUNagn9q ODYNBtq04OXth5gWRt/Ut71+rRR9P71Imj4+voc7/O7LKlZLqvPXzvAHLsTVYpD6YWv7mBoe qJPjkVh7zcT4cKM4XNByXnbvHb9Tdj9e9rHUS8f9sw9A3OC0XZe+UC8jY8lI1nZTcmMretQX 1bf0Hco1PHJH6NqeQZ/pvbnrWCZ9drzRM7mEFPMKTd+znGp8Ee56Pk1kiRUg4HmUyXFvJdy3 zLNsfJUt+pWL9YFFL3KW+AgXgJXsVrLq1sA0f3p9+ujvyf8Dl5ZxzP9dZhldtMtV55fb5frv +Px+u57/+AlaAeRaXZkcK/vO71/eT79lIuLp6yK7XN4Wv4gUf11869/4jt6IU1mLocP472/j /YmeBjEc7iBvos1BG8Gx4o5LZsON6U2exzOgxObmvs1jVeimPoVrZzYpmp/4pFgz76X1xrYG zZPtz5fz1/PtY1pn8bbGR8jbGEZx7MRMbH+wD9B0Sec48Wz1r0lFu7gB/cbL6cv7z+vp5fR6 W/x8Pd8mH8kxJl/Ewd9tx46eiQ8LDuAO3jPE8mKy+oLo1J04RofF2T0Vi+78C2fqk/huNq6c IBNdDNumB2XMV4TfSSIr0ky2JrlKj5htmfgOBACivSxmFqKNy8RUgVcem9IKSlHrgWHgVSSo gJi4h+NFV8a1uNjooG/9iQfUKbXYCIl5z5xqeEzoguqKqBYWJejLmvjQ07Qsg2JiOWTb+Jip jrjtYJMHCeCb6O79UrvFo9otjmsTIkrX9C1sWhjlmYOUse4rwAQ7sWfAw/HOWK1wW2jXuizY 5HjQ3djERzGqagiZ1AVLwEWdTRnGbJdokrV9G2LMdHspmh8VpBiPCm3vfPpxfp0rMV4f5FGW 5pqsojDqQqGpirpz1flPtV0kSWC1L+sZVRdplj6IyOD/Jnb0r2KyGe9cYDJTn19NOdfTO4w/ 01KGrCRMsqQvkKN8sX02sZ2NeLYpwF1y+6ae6XzQYpRATmD2cvLNRq/HqHZtpCQk5dp1DKpL 9gpKVtPGze2VPfAvXC9/nV+0k0OWxnDsndYJOfbhx5U7dKT69PIGywFtfbPsuDI80qdZaeC7 1Vp8cDwqyGfccfM6JA9jH+wAlWm+KQvsPxPQuihG4aQKCbXfO7BE+vBr5wfxuAiv56/PmtMH CBoFKzM6Yp4BQGsO7I29WTCkcdEyMR5YCuHFTsXFoefOOyAsHLLQt5Vpgd4PZ7Yf6GHM+gOQ JFyzSRyBEePNFqH67wM6cQMIIkl3hknK0hL8BhEtC7UfqqUhGPbS3TksKaIaq6eIVp3U0k6j KrIMn3msMbeieGjWwS4h96sAioHjQPVdgOexgiacwIEfo5L2lL/vC9vHBf/5x7s82Rs+Q2sO T7nSxQOcijeWnzPJDo/qBov2PERfK4xYsyvyQMIyQRKru35pI/UHhCBLjo95wR3JFy7E2oNE FO5oWv8knGu50/RQqFrIWn3GDoVDwQhb2rd3QUGZ4f1FSB7ahqWq+XQFxhepFSv2k2exlZg6 KK0Ccru4z2PYXGf1VKtrOPPP46pIdQRhYiGYH+KUoa7S+fwribE2eJTH2n7iOcqCFMWDEFhV IsSOLvMDSY3X9GGs1wcQL/ZVlMjjvAL7ZVKVihmJO4T20R7daMNyLSpamS7dmphq97ieEASU skCp+dv5+acY/0ExfHLvAGFQjxNPDdtUze+P+edOptI6X1/kddH0yDeOMZlb3BTY8+46rZhU URDVTsgfWpUBVMo4isOAEJGkhJGDpe3I+UKgCFwcBdEWODzyQlrVi5Eny4C7gNxjgDPJJg1B SSad4TTYFMUmS/pMT2pUpL34JflLrHHez3CN09dK2rG8/YqqeLhoWsNZlpavAkQJHzHfyRLs ujqbiVXtc7i9EmNnUJbEdH0P4qgc9lTrMygByzET7ywjUWsiPvgBV/R+/1/ZszW3jfP6VzJ9 OmfmbDd2nDR56AMlUTYb3SJKiZ0XTTf1tplukk4u87X//iNAUeIFdPfMtNMaACleARAEAacR 2245RAI8KNwJ7SaoMCsnPDkClIQclHDCOj0UREuvpdiqFhQhSvK0V6Jh52F4hd6FTpobU8TC ub1ZRT1NPiXZ0iaG33G3FDmUCY6c9VqGCyUuIeC5ExZ7AiviSCzdiQSjQokqp7xLreqHLeu6 lvwyNY42OhzLT6bF1m+ikk9u4XmYchkfUigDJw8IpGsth63+pPP7qldnFBdkt2IOtAHRsOjL YECpDVtFkbF2rnO5dMZgBOAdNrq7FZZkqVOf3ECGemmL0wk8XcApIdW7KdEnGu8mV8N1+pOS yUvwk7ZGwUaTWzDpWm9eDYQe1gmLqxRlyrr1gnz6pIr3KE2oUlQYhEOG9cXTwWg8k2pkqIhM lSj8Qc6XXocQAMPmrKWRzN8iBkysbIOiVjfi9JCQw2zKOvxm5hEZ2zq/SSYHBx2nB6LgZuXZ pau6E7m1cTMfIDRAH0XmgsynM5CR38PFJOQQFXAlMBF5+xF/glskplFC40Cub5RnZRWi9Y+E sA1V88mp1xSxvaixXcstHesqL7vheuED7FskKJV2hddgBQl87yC8Ti5X7lJC2WTHfXFyb9XX vC3YThcZX8PdfbMDrOVSC4MHDzBt63lFjYiNYor1OuZpYKjiG8hQ1MknnnbwrJHaR0iDmZYc h6EJeuADFhHZVj0O2R/qsPZndp2hehFoF0LWF2dnx66AqQth50W5FZC61lJcMi+BjfpdFdPR P6vlnznr/qw6+pO5ZgiWIUeVcCDXPgn8Nk4taZ3xBhJwnJ8eU3hRw2EUcsW8u395Oj8/vfhj YYUBrjpkCMFANS/7ty9PR39TLUap79gCAHCJV2YuTGm+ziJHILR2KGvFh+vWQyl9uMhaO+7V JW8r+1OeFaIrm+AnxbU0wuOxm36t+ENiVzCCsI22y5xJDWNNglprSjt2xgGDQeES3CmpWdqO TlmgZ42gob0h9gHL/a8hv3aVVANSGreU3oupjVde/W6KPgajhizhQZMRFGOESTA+vp7mS0gD Gd16jwM4GlimixZLhTN4CNCleJ2SDSRL0IRSHY5ZuyP1vbEib2FMcFKvNAoQoZkCCvg3GFGV eBqzWkqf5NYJBa5hxW3tg1oMsBl0vO0TUR3obwruZnCo5L8laiAFYiw0uk0IIdCiA6hJcnZd 963TDdXQYA0ZGMR6AQ+xTA8YlTTHUJJ1mkH0KmZ4uhz9GemcDfWaVI1SJS/s1SmveiY3buMN TOs8KHGoAH4OVSZacFn+RdSScei7GtxqHUkH7pPGshWRdKAoOYH8JypvvU9wf0wnRHG7OvRV Z4bmr9ySda3QQJagy/Dtb7rNy4RHctzOo9yydcmVKjdqMKrSjyeT7Nx6bKgUldrSzmmo9Hll 4wGuqu0qBJ0Fa3sExjhkG3xJQzBeZQZZLZPRCXk+aXsEZUebgIKK6m4TbYFiTInr7awd1x3l GCHGfEgbezVJU0qqsyO2tbNaKbF47apKIXvQmxs5P51s88AA820dVKhhsRKOM974iIqW4JU3 dfDb1ujx94n/25UdCFu5NPLGti9qimERQCwLVFMZ1qJ0fOclOGK8jFwIU0o3SQuP8ciaTDsG 9PeF3YX3sYPIhqwumag+vvu+f37c//P+6fnru6BUKZT27Z4uR5yRmhDahhf+8BoL1ayeamsE JDJm6U4dISnObYhAV+QFELn1mldDMyhzepqpiQwmKoPZ9AEU1coDNI6+OtKE/ba7mA2jrl7B 2UiQfupqQOH9CsZ0sDwk1Fz7P/0mQaMnD39nhkdnvplz91XrvpqB38PavtEbYcBqxijcfnlv SSuI6jxUMly2yWlQkzc7KW82zkYbAZ7xcoRSGloqnOLCMlbNh8UJSl1QIfaGM3i5Mmwg/ZVT +dA38FzUA3oyFWHYOg8WtDV81TZDo63LYs2QZbIMQeBw5AKDEa3SxmVwKZ6lQKR04CpcOecL jVWHy64IbT8aKbu2DqGw7pwNitBaqZGVM0NjHaXqoToPU7teV1cEdfFt51zyqRMyc3rGwrPY ARHBqMG6cMcKf1Ik1PrUiPD44HalkFOO63dvr3+fv7Mx5tA/rE4+uGUmzAfbGcTF2B5IDubc 9ijzMMsoJl5brAVOQjQPs4hioi2wPbY8zCqKibbadn73MBcRzMVJrMxFdEQvTmL9uVjFvnP+ weuPkDVYdOwAuk6BxTL6fYXyhprJVAi6/gUNXtLgExocafspDT6jwR9o8EWk3ZGmLCJtWXiN uazF+dASsN6FQd4HpVuzKgSnXJ3EUgpedby3UwpPmLZWuhNZ164VRUHVtmachrecX4ZgoVrF 7NgAE6LqRRfpG9mkrm8vhdy4iL7L7YhAhfP6U/2M3PxfokZ59O3z3ff7x6+zvbFD1QdyOhRs Lf3Hij+e7x9fvx99fvxy9OVh//L16OkHOA049kpIyzuM1pTpuQfYzDAAfsGvQW0cue3KsnfV dWdKZ2rEKBvSGD5iEuLmRe2P+3/2f7zeP+yP7r7t776/YAPvNPw5bKO+GoOrVCcFyAQdWp71 KadPfxaZbIrI21SLKLthbb6ifQmyBFLdiSYS7otX8C4T70tUjU3LIVxGdoi07GUX3iUbS7U6 H+raPi6Ol9bQK+VBNIolgd9ZJMlqy1mGX1BUlPmoUmeCDIondeGoFsj/6puKjFJm7ijtAhv1 JXgRF+uFLiO17g5W4BLCXtuRA1yMHr66Kizpz1qEV904Jk2Nl1b2PZsND1dJXoO/j9ZadQZH KroIWws0y7dX1laYgdO1gZ68j8c/F24n9RnLrHOdO/co2//19vWr3rPuKCtdjFdSRPJ56CqB MMiF4dHoWyN6HciiTwxZFadAbZ8YEnwpP3au5GWhBjAcXIM50ERVPyQqBqZygOq6PITU6Y2i K0y/GFV7WHRhE8dJV7PZ/Kab2FK4I8whZMUvsiMGHasJlyuMCu4Ju5IJeLgok8y+5hl/zrUA YKh7uL+lbKwaL6rCyREF1/pjXQ9BtzZKgoTpp2DpHsEzorcfmkdvPj9+dR5Zw/Gmh4gvnVqC kXQqGjlswMmyY5JeJxiqAqxvdU1OkYMfrlnR8/lSRCNBhoGh5niOBbtRTM47Q2ugL0wQGuwC Fz2uYg7+iQeZHTTkkvNGn/vmeCxjZAJ6q40LVekkZTPJbxj7mYMc/c/LGNnh5f+OHt5e9z/3 6j/717v379//rz0ruq62U/Kl41suD+1M1cpoXvBxZ/22kpsbTaSYSX3TMNe86tFimvs4S2ta tbvMqTlyR6EqgFE88BGTl7ZQs/CbtoBLEGsgm0yRo+dL/KNqgSuVjg+RzGO4PFAfI3ikZtLR eVd/r8HXVhJlfXcAn7eJ31HI9QEkuo8IL/WWR5MqBUup5IIV4WV4m/YRGYczCWhyRzVw8Q5o I7VJstiQz4qOquA38wIkwPzV/BTFxCXOjr1KYOKin+BXh9wrxjnERaKkOlzh0K01gz3wtsVX P4RNc/YKids953WpNMwq3dERwFCYbJicZFcrFB8GjyJ11G12motJSsiRhJQeKTVHG9d9GMkK hVreV1rHQ6I2hl23rNn8K5q80ZmiXCJz2sjNHowjhxvRbTARp/8hjS4xj4MiSOs280jAbwMX ElDi2g0qUXvFDl6h4w2NtemqLbHcal9C586pBa7kh7iwgMhAbzxjMtQEJETm7TxY3Xrvvj3i Kazbv7x6u7e4zDpaHUM9AjjOIGPXyJdq7ydc2g6FtOPgvHAU9z2wyRPwIYvjkX9cY7zaQ2T6 yjfGJrTMOFtNksA6f2AWSkgneealKMPB2PBt1tv5AhAKu6iC81HROFkjtB6msJ39EBSheJzO PWALRnYd8efB5a4iU0e1TSoWJxcrzHsJyi/NoCADaCMOsDA9q5cHpnxiB3GSpMnjSMoF3/tC YEeY4zbxMjqt42izTm3vS76LaHCQk4LLw3q3OuA7erb6Tcp5ZI+KycPDk1928MdRhqNxxI44 xllb7EZriePEbsGHLFnTyopDBQ8WSJ9oiD3ZwTrE+Ay/oaG/k4uhWXeDT+BqlXXGbGdSG2q4 sq8ebg/YBTqIANUX8K612oZlq21sv2Z1r/aEtksFxcB1reglrYiacFvgiRRbEJAvzJdpzjiC M0kG0m/kFmRIMW0Zwzysw/H2/Hg+m/g4nn1c0Di9K6woQw4WXJk+nth9G7HwuUNtwk/+Igoe 2IUTje9ANY298au0mqj65StKaDpkLYsYr9KGHeBU4DdWQtZJPODGTgn6SyiIDym/pSAn0Jvp 8eQPDAYEiGLIVF7QiTRHv6KZVfeKLaN4idh25f7u7RleAweWT2BnlnODkiNK7oI7j0KAdLGQ SUDetfA+IdPQ2fVQe7cbuH0GNtfwWcklPgtFRkdxwOB+zkAcd1RTnwnAG8UM27wtCTQcJf0n j922IxutplFU/ZZm/hNRySJH3YlETVO9qw/TsEat3jJi7piodqxkUXUl9iDDbKF5MlhqM1sX +/HddPW5VecJPIHYl7mot3gxkhEGdqhm50NVHT6oufIhWg0CZfbajw0/mfiff/14fTq6e3re Hz09H33b//Nj/2xrl2Mo+Vw0tLchYlmxdsIxOuBlCOdOQsgZGJImxWUqmo2tXPuYsJDn6zAD Q9LWvtqfYSThdLfiN70BZ0UaSnQ+2mwW62orWQAz2bkj8LD28aW8P7Ej/ZAJibcQgWHFJV/n i+V52RdBb6u+oIFhSxr8NyAG9nLV854HBfCfcMWUETjruw23MyaYbGBuyuKRGDw4teYf4NZF z0ccyIzpbfXb6zcItHH3+XX/5Yg/3sEegge+/7l//XbEXl6e7u4RlX1+/WxFbZ3ylYUfImDp hqk/y+OmLnaLEzuskMnuwK9EsK8HrgopWTs92k8wKNXD0xf73Yz5RJKGY92F45B2kvhOEpQt 2htiFyThTGyJCpWMg9e0c4jcl2+xZivREBTflCzszJb6+LUubkKnqAN1+IU2PVkSY4PgMcQG iQyHDqCQPpraMwrZLY4zkVOVaUys6Bp5XDDYsbViEKj6OIktx62UrUJOkp2GMKGWF8QJF+G4 tiUkjCDBtqPMDF6enlFgJ8eGWesbtggaA8BBSslPKHpVexx5uljGkYuhTCI1Um2ApNA+uFu3 TvY4w+IaN4O0NZcDzjNkQzQRXLQUvv/xzY39bKSlJMSqjM0voKyqPWTVJyLckOqgHVak9Jab XBBLzyCC0Io+PtLClJW8KASLIn5XEPoIOaqvt/+echknhdt+uieAO6Whh78uuzMaeqhYxiUF Oxl4xmNlclrAXm7YLaF4SchsQO05DY/2Z5RPUUSsoOScaARvGx2El4SrzcqX8Qo1zYFRtEii 1XQ8XHzqzEOu9hEeWyIGHfuSgx5ObtguSuN0avLKgRBeTlTJaWXkcM8Q1HbrJoYdxbX93GWE na+WBN2Kgm3mKOCfH788PRxVbw9/7Z9NAEyqeaySEByjtWNRmZa3CQav7WkMKd41hlL2EZN2 LYkIvvAJ06aBBaJuwplAayqzQ4l5iIGUxRNWGrU+SkGNx4QkD0soMcabcQ9zE/YZ4viwzH1Q EeJQphzCK1lHHCCAIk0jKSVmkit4zLs5vzj9mab/H9rrnLoAkLuy5GDzALVR265+EcimT4qR RvaJS2bh1DnVO2RtT48vhpS3cAeImZTwDtR+53uZyg+T9x2N1dcH3I5oJNYVz4aG64cv+OYT 6hdzjPcUgoT+jWeLl6O/IfTT/ddHHT8OPfAcl8LRHiu1QenjuztV+OVPKKHIhu/7X+9/7B/e We+WOki7MBubWsffPcRLy2qBttbL69K7LFMQ8GlMN6KhMXlGw4e27jv3sa/BYjx2uxwA1V5I 0fgvwEeltx9pAxqswF4BbRXJiQ+UUhBQMBi2vGBbfUuV8qZza7zO/W+Y2GeZaLtdUWtvPsxt oh9LWqR+QARnLJJdw6Sl0Y0uVOLWe34E4/9g1+ppdNhvOyk6AMZolOoDavmVOr/AfLEnKtaO VxZ5YO8s7v96/vz86+j56e31/tE+i2nbkm1zSkTXcki77jhGzHc2M566+ceOMutYZYZWdm2V Nrshb+vSi7FhkxS8imAr3g1qpdnPEgwKYlHBbRdciIkuxDepgCsI++rWoKJgzxcSXnSlZbNN N9o1peW5RwHX/pBJXr92bQrhWnVSxV1F52hG6eLMpQhPkaolXT+4pU6cQEv6ZHrgTmQkUEyS J7tzoqjGrCLcHElYe8M6foAiEZFPf7BzlibhgTw9d16+9BkY7mE49T4zE0JfUrMqq8vDvVeK 0PQG3srmvlqOb5NdOOhXII9R+frlQI1KNmckva3nmh2oVbMFXxHtQPWLhpO1bG8B7P9Go5YP wwCGTUgr2NkqADL7MmCGdZu+TAIEOBKF9SbppwDmWurmDg3rW+GEsZsQiUIsSUxxWzISge+6 Kfo6Al+F233MtFg7UcHA/7AuaudgYEPhYuecLgAfPICy932SbuzAmRt85Gwu62xf0m0nOewM CjZcus4QEzwpSXAuSye0FfdyG02uHbY6JutUKPaOcqBljrcNvL1zruY1CG5xB4e/om+UPZE6 OhRINOYqAyjF3WiHV7ZYKWrHcwB+H+IDVeE+iJxY9uSEgpsjxydy0Elr67b94MUnSotbSAll v+prM9uWlmXWt8pGOCE2apGBJ5SQjs9Sn0J0kM7V5fIaDs2Ee1FdkeZ9pD//ee7VcP4Tl9x8 Y7YOn2b4AyNhcuBtdThmEE3UTdw1oUB/G9ARQCH/CxXSPlsVgQIA --OgqxwSJOaUobr8KG-- --jho1yZJdad60DJr+ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQJ8BAEBCgBmBQJWin3XXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFQTQyRTkzNjBCNzVERDZFODMyNTQ3RjdB NDMwOTA3NUYwNTk2MEY2AAoJEKQwkHXwWWD2QYQP/06hzbYsHnUpmBqgoejoplLu WRG9MqfDxM1aIX5fs30Xa5gjQNmyxG+8h+NyXLP7gTB0TGe2xOMFet17jyq2BF4r 1atLKCDY7gCUFR/fLrD380dUVZH0/pNiaNyETjPHNUxHO5eESLHa18WTYcTNGct+ ZazqBob/mldtrN8oJEEC3YiWcbcAc0Pb6730JwXzluaIlrZ72i32lkeXr4oqvYC0 snKbBCleHuUsz8OYjTpnV1Tef/iSxmEZV1W2xQAi93DeChdXB8WsHr/63xCLf3oJ AuWJny3ZLwofY8DfPhpuGZfD4PDLVPlszPBPZzw/mwDoh8lYhEDQNgUOdXxNOgor ePv59R7A4iJSSiQ2YRAFaqDGsDux65x+IZg7TFbaggbPXXTSCW3IV78V6cXty+Uz s3vJAUXsiesORleiqAJTOqTVXtFkx1rkU6S/7cucMje6+nV0KhgyGwuOhK/73aRz 7LTl3pMzavG3Gh9tsTimRpK4SxhaAnmzrw9WaKCEm3+C9gGv+KJn1TuV6eLsF1A1 LT8bNxuUnczNoM4gYlAIhq5VvcsQD78rZHgaxTxEKf40H0XA+3tmfvrlf/aJddqX byBBBhF3NfUYHbAoWwiz6N487fSIa7dw3KHGJkJmsg+FaaXFV6v7s6LC5leec0ko UX/slcHDJZuOynZIrFoA =gygm -----END PGP SIGNATURE----- --jho1yZJdad60DJr+-- From bfoster@redhat.com Mon Jan 4 09:53:01 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 3CBBA7F37 for ; Mon, 4 Jan 2016 09:53:01 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 1DB318F804B for ; Mon, 4 Jan 2016 07:52:58 -0800 (PST) X-ASG-Debug-ID: 1451922776-04bdf0265044d910001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 7DiGfEVrGI5J9RrV (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 07:52:57 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 9B641C1090AE for ; Mon, 4 Jan 2016 15:52:56 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04Fquca019228 for ; Mon, 4 Jan 2016 10:52:56 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id A3B5C120068; Mon, 4 Jan 2016 10:52:55 -0500 (EST) From: Brian Foster To: xfs@oss.sgi.com Subject: [PATCH] xfs: fix kbuild regression due to log recovery crc verification Date: Mon, 4 Jan 2016 10:52:55 -0500 X-ASG-Orig-Subj: [PATCH] xfs: fix kbuild regression due to log recovery crc verification Message-Id: <1451922775-15502-1-git-send-email-bfoster@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451922777 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 The auto kbuild infrastructure reports several failures on non-standard arches (i386, sh, arm) due to a regression in the recent log recovery crc verification (short write detection) patches: fs/built-in.o: In function `xlog_find_tail': >> xfs_log_recover.c:(.text+0x1c5db5): undefined reference to `__moddi3' xfs_log_recover.c:(.text+0x1c5e9d): undefined reference to `__moddi3' This is due to the introduction of 64-bit modulus operations that were 32-bit operations prior to the change. xlog_find_tail() used an int to store the record header start block for the head of the log. This was converted to an xfs_daddr_t as this is how the value is represented in the callee functions that handle record detection/verification, etc. This was included as a minor cleanup as there wasn't an obvious reason for using an int for this value in xlog_find_tail(). The previously implicit cast was likely safe as this value represents a log relative block number and the log can only be so big (2G). That said, since all of the surrounding functionality uses xfs_daddr_t, including the log buffer I/O helpers, update these calculations to use do_mod() and handle the 64-bit modulus correctly regardless of architecture. Signed-off-by: Brian Foster --- Compile tested with ARCH=i386 and spot tested with xfstests on x86-64. Brian fs/xfs/xfs_log_recover.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/xfs/xfs_log_recover.c b/fs/xfs/xfs_log_recover.c index 5bf8076..26e67b4 100644 --- a/fs/xfs/xfs_log_recover.c +++ b/fs/xfs/xfs_log_recover.c @@ -1330,12 +1330,13 @@ xlog_find_tail( } else { hblks = 1; } - after_umount_blk = (rhead_blk + hblks + (int) - BTOBB(be32_to_cpu(rhead->h_len))) % log->l_logBBsize; + after_umount_blk = rhead_blk + hblks + BTOBB(be32_to_cpu(rhead->h_len)); + after_umount_blk = do_mod(after_umount_blk, log->l_logBBsize); tail_lsn = atomic64_read(&log->l_tail_lsn); if (*head_blk == after_umount_blk && be32_to_cpu(rhead->h_num_logops) == 1) { - umount_data_blk = (rhead_blk + hblks) % log->l_logBBsize; + umount_data_blk = rhead_blk + hblks; + umount_data_blk = do_mod(umount_data_blk, log->l_logBBsize); error = xlog_bread(log, umount_data_blk, 1, bp, &offset); if (error) goto done; -- 2.4.3 From bfoster@redhat.com Mon Jan 4 10:03:47 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id C16677F37 for ; Mon, 4 Jan 2016 10:03:47 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 5CFFFAC002 for ; Mon, 4 Jan 2016 08:03:44 -0800 (PST) X-ASG-Debug-ID: 1451923422-04bdf0264f44dcc0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id xtIyNEWjI8aqhPJx (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 08:03:43 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id A453BA3B6E; Mon, 4 Jan 2016 16:03:42 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04G3gd1021329; Mon, 4 Jan 2016 11:03:42 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id AE092120068; Mon, 4 Jan 2016 11:03:41 -0500 (EST) From: Brian Foster To: fstests@vger.kernel.org Cc: xfs@oss.sgi.com Subject: [PATCH v2] xfs: test XFS torn log write detection Date: Mon, 4 Jan 2016 11:03:41 -0500 X-ASG-Orig-Subj: [PATCH v2] xfs: test XFS torn log write detection Message-Id: <1451923421-16647-1-git-send-email-bfoster@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451923423 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 XFS torn log write detection includes a mechanism to inject CRC errors into log records at runtime and shutdown the fs accordingly. This ensures that the CRC verification pass on the subsequent mount discovers an invalid record near the head of the log and considers it a torn write. This test runs a workload with error injection enabled and verifies that the subsequent mount is successful. The test repeats for several iterations using a random frequency factor for the error event each time. Signed-off-by: Brian Foster --- Here's a v2 of the XFS log crc verification test case now that the associated kernel code has been merged. The only change is the bugfix noted in the changelog below. Brian v2: - Update random failure frequency calculation to ensure factor is never 0. v1: http://article.gmane.org/gmane.comp.file-systems.fstests/1406 tests/xfs/350 | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/350.out | 2 ++ tests/xfs/group | 1 + 3 files changed, 88 insertions(+) create mode 100755 tests/xfs/350 create mode 100644 tests/xfs/350.out diff --git a/tests/xfs/350 b/tests/xfs/350 new file mode 100755 index 0000000..1a1de6f --- /dev/null +++ b/tests/xfs/350 @@ -0,0 +1,85 @@ +#! /bin/bash +# FS QA Test No. 350 +# +# Use the XFS log record CRC error injection mechanism to test torn writes to +# the log. The error injection mechanism writes an invalid CRC and shuts down +# the filesystem. The test verifies that a subsequent remount recovers the log +# and that the filesystem is consistent. +# +# Note that this test requires a DEBUG mode kernel. +# +#----------------------------------------------------------------------- +# Copyright (c) 2015 Red Hat, Inc. All Rights Reserved. +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it would be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write the Free Software Foundation, +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +#----------------------------------------------------------------------- +# + +seq=`basename $0` +seqres=$RESULT_DIR/$seq +echo "QA output created by $seq" + +here=`pwd` +tmp=/tmp/$$ +status=1 # failure is the default! +trap "_cleanup; exit \$status" 0 1 2 3 15 + +_cleanup() +{ + cd / + rm -f $tmp.* + killall -9 fsstress > /dev/null 2>&1 + wait > /dev/null 2>&1 +} + +rm -f $seqres.full + +# get standard environment, filters and checks +. ./common/rc + +# real QA test starts here + +# Modify as appropriate. +_supported_fs xfs +_supported_os Linux +_require_xfs_sysfs $(_short_dev $SCRATCH_DEV)/log/log_badcrc_factor +_require_scratch + +echo "Silence is golden." + +_scratch_mkfs >> $seqres.full 2>&1 || _fail "mkfs failed" +_scratch_mount + +sdev=$(_short_dev $SCRATCH_DEV) + +for i in $(seq 1 5); do + # Enable error injection. Use a random bad crc factor up to 100 + # (increase this value to run fsstress longer). + factor=$((RANDOM % 100 + 1)) + echo iteration $i log_badcrc_factor: $factor >> $seqres.full 2>&1 + echo $factor > /sys/fs/xfs/$sdev/log/log_badcrc_factor + + # Run fsstress until the filesystem shuts down. It will shut down + # automatically when error injection triggers. + $FSSTRESS_PROG -d $SCRATCH_MNT -p 4 -n 999999 >> $seqres.full 2>&1 + + # Verify that we can remount the fs. Log recovery should handle the torn + # write. + _scratch_unmount + _scratch_mount || _fail "failed to mount" +done + +# success, all done +status=0 +exit diff --git a/tests/xfs/350.out b/tests/xfs/350.out new file mode 100644 index 0000000..c725135 --- /dev/null +++ b/tests/xfs/350.out @@ -0,0 +1,2 @@ +QA output created by 350 +Silence is golden. diff --git a/tests/xfs/group b/tests/xfs/group index 634f7e7..5fbace9 100644 --- a/tests/xfs/group +++ b/tests/xfs/group @@ -232,3 +232,4 @@ 303 auto quick quota 304 auto quick quota 305 auto quota +350 auto log metadata -- 2.4.3 From dan.j.williams@intel.com Mon Jan 4 12:20:31 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id A51387F37 for ; Mon, 4 Jan 2016 12:20:31 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 87F4F304032 for ; Mon, 4 Jan 2016 10:20:28 -0800 (PST) X-ASG-Debug-ID: 1451931626-04cbb0101c1f6b20001-NocioJ Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by cuda.sgi.com with ESMTP id UrH0UeOiO7FIipHx for ; Mon, 04 Jan 2016 10:20:27 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.20 X-ASG-Whitelist: EmailCat (corporate) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 04 Jan 2016 10:20:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,521,1444719600"; d="scan'208";a="883602987" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.136]) by orsmga002.jf.intel.com with ESMTP; 04 Jan 2016 10:20:26 -0800 Subject: [resend PATCH 0/3] fs, bdev: handle end of life From: Dan Williams X-ASG-Orig-Subj: [resend PATCH 0/3] fs, bdev: handle end of life To: xfs@oss.sgi.com Cc: linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, Dave Chinner , Jens Axboe , linux-fsdevel@vger.kernel.org, Jan Kara , Tejun Heo , Matthew Wilcox , Ross Zwisler Date: Mon, 04 Jan 2016 10:20:00 -0800 Message-ID: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-9-g687f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mga02.intel.com[134.134.136.20] X-Barracuda-Start-Time: 1451931626 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 Per Dave, resend to include the xfs list on the full set. These are against v4.4-rc5. --- As mentioned in [PATCH 1/3] "block, fs: reliably communicate bdev end-of-life", historically we have waited for filesystem specific heuristics to attempt to guess when a block device is gone. Sometimes this works, but in other cases the system can hang waiting for the fs to trigger its shutdown protocol. Now with DAX we need new actions, like unmapping all inodes, to be taken upon a shutdown event. Those actions need to be taken whether the shutdown event comes from the block device being torn down, or some other file system specific event. For now, the approach taken in the following patches only affects xfs and block drivers that are converted to use del_gendisk_queue(). We can add more filesystems and driver support over time. Note that 'bdi_gone' was chosen over 'shutdown' so as not to be confused with generic_shutdown_super() --- Dan Williams (3): block, fs: reliably communicate bdev end-of-life xfs: handle shutdown notifications writeback: fix false positive WARN in __mark_inode_dirty block/genhd.c | 87 +++++++++++++++++++++++++++++++++++------- drivers/block/brd.c | 3 - drivers/nvdimm/pmem.c | 3 - drivers/s390/block/dcssblk.c | 6 +-- fs/block_dev.c | 79 +++++++++++++++++++++++++++++++++----- fs/xfs/xfs_super.c | 9 ++++ include/linux/fs.h | 4 ++ include/linux/genhd.h | 1 mm/backing-dev.c | 7 +++ 9 files changed, 166 insertions(+), 33 deletions(-) From dan.j.williams@intel.com Mon Jan 4 12:20:40 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 18E4F7F50 for ; Mon, 4 Jan 2016 12:20:40 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 099DA304032 for ; Mon, 4 Jan 2016 10:20:39 -0800 (PST) X-ASG-Debug-ID: 1451931637-04cbb0101c1f6b30001-NocioJ Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by cuda.sgi.com with ESMTP id ShtXkN0OTyPUFxc9 for ; Mon, 04 Jan 2016 10:20:38 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.65 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP; 04 Jan 2016 10:20:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,521,1444719600"; d="scan'208";a="874409453" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.136]) by fmsmga001.fm.intel.com with ESMTP; 04 Jan 2016 10:20:37 -0800 Subject: [resend PATCH 2/3] xfs: handle shutdown notifications From: Dan Williams X-ASG-Orig-Subj: [resend PATCH 2/3] xfs: handle shutdown notifications To: xfs@oss.sgi.com Cc: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, Dave Chinner , linux-nvdimm@lists.01.org Date: Mon, 04 Jan 2016 10:20:11 -0800 Message-ID: <20160104182011.24118.30446.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-9-g687f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mga03.intel.com[134.134.136.65] X-Barracuda-Start-Time: 1451931637 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25825 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Force a filesystem shutdown when the backing device is known to be dead. I.e. blk_queue_enter() permanently returns -ENODEV. Cc: xfs@oss.sgi.com Suggested-by: Dave Chinner Signed-off-by: Dan Williams --- fs/block_dev.c | 3 ++- fs/xfs/xfs_super.c | 9 +++++++++ include/linux/fs.h | 2 ++ 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 739e43a37e64..7d6c66148948 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -1806,7 +1806,7 @@ int __invalidate_device(struct block_device *bdev, bool kill_dirty) } EXPORT_SYMBOL(__invalidate_device); -static void generic_bdi_gone(struct super_block *sb) +void generic_bdi_gone(struct super_block *sb) { struct inode *inode, *_inode = NULL; @@ -1832,6 +1832,7 @@ static void generic_bdi_gone(struct super_block *sb) spin_unlock(&sb->s_inode_list_lock); iput(_inode); } +EXPORT_SYMBOL(generic_bdi_gone); void shutdown_partition(struct gendisk *disk, int partno) { diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 36bd8825bfb0..63c36508e9db 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -1618,6 +1618,14 @@ xfs_fs_free_cached_objects( return xfs_reclaim_inodes_nr(XFS_M(sb), sc->nr_to_scan); } +static void +xfs_fs_bdi_gone( + struct super_block *sb) +{ + xfs_force_shutdown(XFS_M(sb), SHUTDOWN_DEVICE_REQ); + generic_bdi_gone(sb); +} + static const struct super_operations xfs_super_operations = { .alloc_inode = xfs_fs_alloc_inode, .destroy_inode = xfs_fs_destroy_inode, @@ -1632,6 +1640,7 @@ static const struct super_operations xfs_super_operations = { .show_options = xfs_fs_show_options, .nr_cached_objects = xfs_fs_nr_cached_objects, .free_cached_objects = xfs_fs_free_cached_objects, + .bdi_gone = xfs_fs_bdi_gone, }; static struct file_system_type xfs_fs_type = { diff --git a/include/linux/fs.h b/include/linux/fs.h index 0e201ed38045..b1e8e049e4b8 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2265,6 +2265,7 @@ extern struct super_block *freeze_bdev(struct block_device *); extern void emergency_thaw_all(void); extern int thaw_bdev(struct block_device *bdev, struct super_block *sb); extern int fsync_bdev(struct block_device *); +extern void generic_bdi_gone(struct super_block *sb); extern struct super_block *blockdev_superblock; @@ -2277,6 +2278,7 @@ static inline void bd_forget(struct inode *inode) {} static inline int sync_blockdev(struct block_device *bdev) { return 0; } static inline void kill_bdev(struct block_device *bdev) {} static inline void invalidate_bdev(struct block_device *bdev) {} +static inline void generic_bdi_gone(struct super_block *sb) {} static inline struct super_block *freeze_bdev(struct block_device *sb) { From dan.j.williams@intel.com Mon Jan 4 12:20:40 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 303487F51 for ; Mon, 4 Jan 2016 12:20:40 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 22D598F804B for ; Mon, 4 Jan 2016 10:20:37 -0800 (PST) X-ASG-Debug-ID: 1451931632-04cb6c3032429b60001-NocioJ Received: from mga04.intel.com ([192.55.52.120]) by cuda.sgi.com with ESMTP id gtgyJ4izzRnOUXDj for ; Mon, 04 Jan 2016 10:20:32 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 192.55.52.120 Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga104.fm.intel.com with ESMTP; 04 Jan 2016 10:20:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,521,1444719600"; d="scan'208";a="627873704" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.136]) by FMSMGA003.fm.intel.com with ESMTP; 04 Jan 2016 10:20:31 -0800 Subject: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life From: Dan Williams X-ASG-Orig-Subj: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life To: xfs@oss.sgi.com Cc: linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, Dave Chinner , Jens Axboe , Jan Kara , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Ross Zwisler Date: Mon, 04 Jan 2016 10:20:05 -0800 Message-ID: <20160104182005.24118.50361.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-9-g687f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: UNKNOWN[192.55.52.120] X-Barracuda-Start-Time: 1451931632 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC5_MJ1963, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25825 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 Historically we have waited for filesystem specific heuristics to attempt to guess when a block device is gone. Sometimes this works, but in other cases the system can hang waiting for the fs to trigger its shutdown protocol. The initial motivation for this investigation was to prevent DAX mappings (direct mmap access to persistent memory) from leaking past the lifetime of the hosting block device. However, Dave points out that these shutdown operations are needed in other scenarios. Quoting Dave: For example, if we detect a free space corruption during allocation, it is not safe to trust *any active mapping* because we can't trust that we having handed out the same block to multiple owners. Hence on such a filesystem shutdown, we have to prevent any new DAX mapping from occurring and invalidate all existing mappings as we cannot allow userspace to modify any data or metadata until we've resolved the corruption situation. The current block device shutdown sequence of del_gendisk + blk_cleanup_queue is problematic. We want to tell the fs after blk_cleanup_queue that there is no possibility of recovery, but by that time we have deleted partitions and lost the ability to find all the super-blocks on a block device. Introduce del_gendisk_queue to trigger ->quiesce() and ->bdi_gone() notifications to all the filesystems hosted on the disk. Where ->quiesce() are 'shutdown' operations while the bdev may still be alive, and ->bdi_gone() is a set of actions to take after the backing device is known to be permanently dead. generic_quiesce_super and generic_bdi_gone, are the default operations when a filesystem does not implement ->quiesce(), ->bdi_gone(). They invalidate inodes and unmap DAX-inodes respectively. For now only ->bdi_gone() has an associated super operation as xfs will implement ->bdi_gone() in a later patch. Cc: Jan Kara Cc: Jens Axboe Cc: Matthew Wilcox Cc: Ross Zwisler Suggested-by: Dave Chinner Signed-off-by: Dan Williams --- block/genhd.c | 87 +++++++++++++++++++++++++++++++++++------- drivers/block/brd.c | 3 - drivers/nvdimm/pmem.c | 3 - drivers/s390/block/dcssblk.c | 6 +-- fs/block_dev.c | 78 ++++++++++++++++++++++++++++++++------ include/linux/fs.h | 2 + include/linux/genhd.h | 1 7 files changed, 147 insertions(+), 33 deletions(-) diff --git a/block/genhd.c b/block/genhd.c index e5cafa51567c..967fcfd63c98 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -634,24 +634,14 @@ void add_disk(struct gendisk *disk) } EXPORT_SYMBOL(add_disk); -void del_gendisk(struct gendisk *disk) +static void del_gendisk_start(struct gendisk *disk) { - struct disk_part_iter piter; - struct hd_struct *part; - blk_integrity_del(disk); disk_del_events(disk); +} - /* invalidate stuff */ - disk_part_iter_init(&piter, disk, - DISK_PITER_INCL_EMPTY | DISK_PITER_REVERSE); - while ((part = disk_part_iter_next(&piter))) { - invalidate_partition(disk, part->partno); - delete_partition(disk, part->partno); - } - disk_part_iter_exit(&piter); - - invalidate_partition(disk, 0); +static void del_gendisk_end(struct gendisk *disk) +{ set_capacity(disk, 0); disk->flags &= ~GENHD_FL_UP; @@ -670,9 +660,78 @@ void del_gendisk(struct gendisk *disk) pm_runtime_set_memalloc_noio(disk_to_dev(disk), false); device_del(disk_to_dev(disk)); } + +#define for_each_part(part, piter) \ + for (part = disk_part_iter_next(piter); part; \ + part = disk_part_iter_next(piter)) +void del_gendisk(struct gendisk *disk) +{ + struct disk_part_iter piter; + struct hd_struct *part; + + del_gendisk_start(disk); + + /* invalidate stuff */ + disk_part_iter_init(&piter, disk, + DISK_PITER_INCL_EMPTY | DISK_PITER_REVERSE); + for_each_part(part, &piter) { + invalidate_partition(disk, part->partno); + delete_partition(disk, part->partno); + } + disk_part_iter_exit(&piter); + invalidate_partition(disk, 0); + + del_gendisk_end(disk); +} EXPORT_SYMBOL(del_gendisk); /** + * del_gendisk_queue - combined del_gendisk + blk_cleanup_queue + * @disk: disk to delete, invalidate, unmap, and end i/o + * + * This alternative for open coded calls to: + * del_gendisk() + * blk_cleanup_queue() + * ...is for notifying the filesystem that the block device has gone + * away. This distinction is important for triggering a filesystem to + * abort its error recovery and for (DAX) capable devices. DAX bypasses + * page cache and mappings go directly to storage media. When such a + * disk is removed the pfn backing a mapping may be invalid or removed + * from the system. Upon return accessing DAX mappings of this disk + * will trigger SIGBUS. + */ +void del_gendisk_queue(struct gendisk *disk) +{ + struct disk_part_iter piter; + struct hd_struct *part; + + del_gendisk_start(disk); + + /* pass1 sync fs + evict idle inodes */ + disk_part_iter_init(&piter, disk, + DISK_PITER_INCL_EMPTY | DISK_PITER_REVERSE); + for_each_part(part, &piter) + invalidate_partition(disk, part->partno); + disk_part_iter_exit(&piter); + invalidate_partition(disk, 0); + + blk_cleanup_queue(disk->queue); + + /* pass2 the queue is dead, trigger fs shutdown via ->bdi_gone() */ + disk_part_iter_init(&piter, disk, + DISK_PITER_INCL_EMPTY | DISK_PITER_REVERSE); + for_each_part(part, &piter) { + shutdown_partition(disk, part->partno); + delete_partition(disk, part->partno); + } + disk_part_iter_exit(&piter); + shutdown_partition(disk, 0); + + del_gendisk_end(disk); +} +EXPORT_SYMBOL(del_gendisk_queue); + +/** * get_gendisk - get partitioning information for a given device * @devt: device to get partitioning information for * @partno: returned partition index diff --git a/drivers/block/brd.c b/drivers/block/brd.c index a5880f4ab40e..f149dd57bba3 100644 --- a/drivers/block/brd.c +++ b/drivers/block/brd.c @@ -532,7 +532,6 @@ out: static void brd_free(struct brd_device *brd) { put_disk(brd->brd_disk); - blk_cleanup_queue(brd->brd_queue); brd_free_pages(brd); kfree(brd); } @@ -560,7 +559,7 @@ out: static void brd_del_one(struct brd_device *brd) { list_del(&brd->brd_list); - del_gendisk(brd->brd_disk); + del_gendisk_queue(brd->brd_disk); brd_free(brd); } diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index 8ee79893d2f5..6dd06e9d34b0 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -158,9 +158,8 @@ static void pmem_detach_disk(struct pmem_device *pmem) if (!pmem->pmem_disk) return; - del_gendisk(pmem->pmem_disk); + del_gendisk_queue(pmem->pmem_disk); put_disk(pmem->pmem_disk); - blk_cleanup_queue(pmem->pmem_queue); } static int pmem_attach_disk(struct device *dev, diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c index 94a8f4ab57bc..0c3c968b57d9 100644 --- a/drivers/s390/block/dcssblk.c +++ b/drivers/s390/block/dcssblk.c @@ -388,8 +388,7 @@ removeseg: } list_del(&dev_info->lh); - del_gendisk(dev_info->gd); - blk_cleanup_queue(dev_info->dcssblk_queue); + del_gendisk_queue(dev_info->gd); dev_info->gd->queue = NULL; put_disk(dev_info->gd); up_write(&dcssblk_devices_sem); @@ -751,8 +750,7 @@ dcssblk_remove_store(struct device *dev, struct device_attribute *attr, const ch } list_del(&dev_info->lh); - del_gendisk(dev_info->gd); - blk_cleanup_queue(dev_info->dcssblk_queue); + del_gendisk_queue(dev_info->gd); dev_info->gd->queue = NULL; put_disk(dev_info->gd); device_unregister(&dev_info->dev); diff --git a/fs/block_dev.c b/fs/block_dev.c index 44d4a1e9244e..739e43a37e64 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -1778,27 +1778,83 @@ fail: } EXPORT_SYMBOL(lookup_bdev); +static int generic_quiesce_super(struct super_block *sb, bool kill_dirty) +{ + /* + * no need to lock the super, get_super holds the read mutex so + * the filesystem cannot go away under us (->put_super runs with + * the write lock held). + */ + shrink_dcache_sb(sb); + return invalidate_inodes(sb, kill_dirty); +} + int __invalidate_device(struct block_device *bdev, bool kill_dirty) { struct super_block *sb = get_super(bdev); int res = 0; - if (sb) { - /* - * no need to lock the super, get_super holds the - * read mutex so the filesystem cannot go away - * under us (->put_super runs with the write lock - * hold). - */ - shrink_dcache_sb(sb); - res = invalidate_inodes(sb, kill_dirty); - drop_super(sb); - } + if (!sb) + goto out; + + res = generic_quiesce_super(sb, kill_dirty); + drop_super(sb); + out: invalidate_bdev(bdev); + return res; } EXPORT_SYMBOL(__invalidate_device); +static void generic_bdi_gone(struct super_block *sb) +{ + struct inode *inode, *_inode = NULL; + + spin_lock(&sb->s_inode_list_lock); + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { + spin_lock(&inode->i_lock); + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) + || !IS_DAX(inode)) { + spin_unlock(&inode->i_lock); + continue; + } + __iget(inode); + spin_unlock(&inode->i_lock); + spin_unlock(&sb->s_inode_list_lock); + + unmap_mapping_range(inode->i_mapping, 0, 0, 1); + iput(_inode); + _inode = inode; + cond_resched(); + + spin_lock(&sb->s_inode_list_lock); + } + spin_unlock(&sb->s_inode_list_lock); + iput(_inode); +} + +void shutdown_partition(struct gendisk *disk, int partno) +{ + struct block_device *bdev = bdget_disk(disk, partno); + struct super_block *sb = get_super(bdev); + + if (!bdev) + return; + + if (!sb) { + bdput(bdev); + return; + } + + if (sb->s_op->bdi_gone) + sb->s_op->bdi_gone(sb); + else + generic_bdi_gone(sb); + + drop_super(sb); + bdput(bdev); +} + void iterate_bdevs(void (*func)(struct block_device *, void *), void *arg) { struct inode *inode, *old_inode = NULL; diff --git a/include/linux/fs.h b/include/linux/fs.h index 3aa514254161..0e201ed38045 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1713,6 +1713,7 @@ struct super_operations { struct shrink_control *); long (*free_cached_objects)(struct super_block *, struct shrink_control *); + void (*bdi_gone)(struct super_block *); }; /* @@ -2390,6 +2391,7 @@ extern int revalidate_disk(struct gendisk *); extern int check_disk_change(struct block_device *); extern int __invalidate_device(struct block_device *, bool); extern int invalidate_partition(struct gendisk *, int); +extern void shutdown_partition(struct gendisk *, int); #endif unsigned long invalidate_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t end); diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 847cc1d91634..028cf15a8a57 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -431,6 +431,7 @@ extern void part_round_stats(int cpu, struct hd_struct *part); /* block/genhd.c */ extern void add_disk(struct gendisk *disk); extern void del_gendisk(struct gendisk *gp); +extern void del_gendisk_queue(struct gendisk *disk); extern struct gendisk *get_gendisk(dev_t dev, int *partno); extern struct block_device *bdget_disk(struct gendisk *disk, int partno); From dan.j.williams@intel.com Mon Jan 4 12:20:44 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id CAC257F51 for ; Mon, 4 Jan 2016 12:20:44 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id BC0CB8F8035 for ; Mon, 4 Jan 2016 10:20:44 -0800 (PST) X-ASG-Debug-ID: 1451931637-04cbb0101c1f6b30002-NocioJ Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by cuda.sgi.com with ESMTP id sGs31rhbZLgpBR15 for ; Mon, 04 Jan 2016 10:20:42 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.65 Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga103.jf.intel.com with ESMTP; 04 Jan 2016 10:20:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,521,1444719600"; d="scan'208";a="720045429" Received: from dwillia2-desk3.jf.intel.com (HELO dwillia2-desk3.amr.corp.intel.com) ([10.54.39.136]) by orsmga003.jf.intel.com with ESMTP; 04 Jan 2016 10:20:42 -0800 Subject: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty From: Dan Williams X-ASG-Orig-Subj: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty To: xfs@oss.sgi.com Cc: linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, Dave Chinner , Jens Axboe , linux-fsdevel@vger.kernel.org, Jan Kara , Tejun Heo Date: Mon, 04 Jan 2016 10:20:16 -0800 Message-ID: <20160104182016.24118.33718.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-9-g687f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Barracuda-Connect: mga03.intel.com[134.134.136.65] X-Barracuda-Start-Time: 1451931639 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25825 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- This warning was added as a debugging aid way back in commit 500b067c5e6c "writeback: check for registered bdi in flusher add and inode dirty" when we were switching over to per-bdi writeback. Once the block device has been torn down it's no longer useful to complain about unregistered bdi's. Clear the writeback capability under the the wb->list_lock(), so that __mark_inode_dirty has no opportunity to race bdi_unregister() to this WARN() condition. Alternatively we could just delete the warning... Found this while testing block device remove from underneath an active fs triggering traces like: WARNING: CPU: 6 PID: 2129 at fs/fs-writeback.c:2065 __mark_inode_dirty+0x261/0x350() bdi-block not registered [..] Call Trace: [] dump_stack+0x44/0x62 [] warn_slowpath_common+0x82/0xc0 [] warn_slowpath_fmt+0x5c/0x80 [] __mark_inode_dirty+0x261/0x350 [] generic_update_time+0x79/0xd0 [] file_update_time+0xbd/0x110 [] ext4_dax_fault+0x68/0x110 [] __do_fault+0x4e/0xf0 [] handle_mm_fault+0x5e7/0x1b50 [] ? handle_mm_fault+0x51/0x1b50 [] __do_page_fault+0x191/0x3f0 [] trace_do_page_fault+0x4f/0x120 [] do_async_page_fault+0x1a/0xa0 [] async_page_fault+0x28/0x30 Cc: Jan Kara Cc: Tejun Heo Cc: Jens Axboe Cc: Dave Chinner Signed-off-by: Dan Williams --- mm/backing-dev.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/mm/backing-dev.c b/mm/backing-dev.c index 7340353f8aea..9ea843bca709 100644 --- a/mm/backing-dev.c +++ b/mm/backing-dev.c @@ -343,10 +343,17 @@ static void wb_shutdown(struct bdi_writeback *wb) { /* Make sure nobody queues further work */ spin_lock_bh(&wb->work_lock); + if (!test_and_clear_bit(WB_registered, &wb->state)) { spin_unlock_bh(&wb->work_lock); return; } + + /* tell __mark_inode_dirty that writeback is no longer possible */ + spin_lock(&wb->list_lock); + wb->bdi->capabilities |= BDI_CAP_NO_WRITEBACK; + spin_unlock(&wb->list_lock); + spin_unlock_bh(&wb->work_lock); /* From bfoster@redhat.com Mon Jan 4 13:11:55 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 0ADA27F37 for ; Mon, 4 Jan 2016 13:11:55 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id F03DB8F8040 for ; Mon, 4 Jan 2016 11:11:51 -0800 (PST) X-ASG-Debug-ID: 1451934710-04cbb0101d1f8550001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id JpreCJ1UNyyFmqzF (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:11:50 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id F15508E900; Mon, 4 Jan 2016 19:11:49 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04JBnus004581; Mon, 4 Jan 2016 14:11:49 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id C77E2120068; Mon, 4 Jan 2016 14:11:48 -0500 (EST) Date: Mon, 4 Jan 2016 14:11:48 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 1/9] metadump: clean up btree block region zeroing Message-ID: <20160104191147.GA19852@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 1/9] metadump: clean up btree block region zeroing References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-2-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-2-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451934710 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:01AM +1100, Dave Chinner wrote: > From: Dave Chinner > > Abstract out all the common operations in the btree block zeroing > so that we only have one copy of offset/len calculations and don't > require lots of casts for the pointer arithmetic. > > Signed-off-by: Dave Chinner > --- Reviewed-by: Brian Foster > db/metadump.c | 61 +++++++++++++++++++++++------------------------------------ > 1 file changed, 24 insertions(+), 37 deletions(-) > > diff --git a/db/metadump.c b/db/metadump.c > index 8455fdd..a185da5 100644 > --- a/db/metadump.c > +++ b/db/metadump.c > @@ -258,8 +258,8 @@ zero_btree_node( > xfs_inobt_key_t *ikp; > xfs_alloc_ptr_t *app; > xfs_alloc_key_t *akp; > - void *zp1, *zp2; > - int zlen1, zlen2; > + char *zp1, *zp2; > + char *key_end; > > nrecs = be16_to_cpu(block->bb_numrecs); > > @@ -268,43 +268,36 @@ zero_btree_node( > case TYP_BMAPBTD: > bkp = XFS_BMBT_KEY_ADDR(mp, block, 1); > bpp = XFS_BMBT_PTR_ADDR(mp, block, 1, mp->m_bmap_dmxr[1]); > - zp1 = &bkp[nrecs]; > - zlen1 = (char *)&bpp[0] - (char *)&bkp[nrecs]; > - zp2 = &bpp[nrecs]; > - zlen2 = (char *)block + mp->m_sb.sb_blocksize - > - (char *)&bpp[nrecs]; > + zp1 = (char *)&bkp[nrecs]; > + zp2 = (char *)&bpp[nrecs]; > + key_end = (char *)bpp; > break; > case TYP_INOBT: > case TYP_FINOBT: > ikp = XFS_INOBT_KEY_ADDR(mp, block, 1); > ipp = XFS_INOBT_PTR_ADDR(mp, block, 1, mp->m_inobt_mxr[1]); > - zp1 = &ikp[nrecs]; > - zlen1 = (char *)&ipp[0] - (char *)&ikp[nrecs]; > - zp2 = &ipp[nrecs]; > - zlen2 = (char *)block + mp->m_sb.sb_blocksize - > - (char *)&ipp[nrecs]; > + zp1 = (char *)&ikp[nrecs]; > + zp2 = (char *)&ipp[nrecs]; > + key_end = (char *)ipp; > break; > case TYP_BNOBT: > case TYP_CNTBT: > akp = XFS_ALLOC_KEY_ADDR(mp, block, 1); > app = XFS_ALLOC_PTR_ADDR(mp, block, 1, mp->m_alloc_mxr[1]); > - zp1 = &akp[nrecs]; > - zlen1 = (char *)&app[0] - (char *)&akp[nrecs]; > - zp2 = &app[nrecs]; > - zlen2 = (char *)block + mp->m_sb.sb_blocksize - > - (char *)&app[nrecs]; > + zp1 = (char *)&akp[nrecs]; > + zp2 = (char *)&app[nrecs]; > + key_end = (char *)app; > break; > default: > - zp1 = NULL; > - break; > + return; > } > > - if (zp1 && zp2) { > - /* Zero from end of keys to beginning of pointers */ > - memset(zp1, 0, zlen1); > - /* Zero from end of pointers to end of block */ > - memset(zp2, 0, zlen2); > - } > + > + /* Zero from end of keys to beginning of pointers */ > + memset(zp1, 0, key_end - zp1); > + > + /* Zero from end of pointers to end of block */ > + memset(zp2, 0, (char *)block + mp->m_sb.sb_blocksize - zp2); > } > > static void > @@ -316,8 +309,7 @@ zero_btree_leaf( > struct xfs_bmbt_rec *brp; > struct xfs_inobt_rec *irp; > struct xfs_alloc_rec *arp; > - void *zp; > - int zlen; > + char *zp; > > nrecs = be16_to_cpu(block->bb_numrecs); > > @@ -325,29 +317,24 @@ zero_btree_leaf( > case TYP_BMAPBTA: > case TYP_BMAPBTD: > brp = XFS_BMBT_REC_ADDR(mp, block, 1); > - zp = &brp[nrecs]; > - zlen = (char *)block + mp->m_sb.sb_blocksize - (char *)&brp[nrecs]; > + zp = (char *)&brp[nrecs]; > break; > case TYP_INOBT: > case TYP_FINOBT: > irp = XFS_INOBT_REC_ADDR(mp, block, 1); > - zp = &irp[nrecs]; > - zlen = (char *)block + mp->m_sb.sb_blocksize - (char *)&irp[nrecs]; > + zp = (char *)&irp[nrecs]; > break; > case TYP_BNOBT: > case TYP_CNTBT: > arp = XFS_ALLOC_REC_ADDR(mp, block, 1); > - zp = &arp[nrecs]; > - zlen = (char *)block + mp->m_sb.sb_blocksize - (char *)&arp[nrecs]; > + zp = (char *)&arp[nrecs]; > break; > default: > - zp = NULL; > - break; > + return; > } > > /* Zero from end of records to end of block */ > - if (zp && zlen < mp->m_sb.sb_blocksize) > - memset(zp, 0, zlen); > + memset(zp, 0, (char *)block + mp->m_sb.sb_blocksize - zp); > } > > static void > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Mon Jan 4 13:12:02 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id C936F7F37 for ; Mon, 4 Jan 2016 13:12:02 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id B7528304062 for ; Mon, 4 Jan 2016 11:11:59 -0800 (PST) X-ASG-Debug-ID: 1451934718-04cb6c303142adb0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 6ZF7xEFmkMmpe24q (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:11:58 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id E3CEF1EB24; Mon, 4 Jan 2016 19:11:57 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04JBve1021358; Mon, 4 Jan 2016 14:11:57 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id BDA83120068; Mon, 4 Jan 2016 14:11:56 -0500 (EST) Date: Mon, 4 Jan 2016 14:11:56 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 2/9] metadump: bounds check btree block regions being zeroed Message-ID: <20160104191156.GB19852@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 2/9] metadump: bounds check btree block regions being zeroed References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-3-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-3-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451934718 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:02AM +1100, Dave Chinner wrote: > From: Dave Chinner > > Arkadiusz Miskiewicz reported that metadump was crashing on one of > his corrupted filesystems, and the trace indicated that it was > zeroing unused regions in inode btree blocks when it failed. The > btree block had a corrupt nrecs field, which was resulting in an out > of bounds memset() occurring. Ensure that the region being > generated for zeroing is within bounds before executing the zeroing. > > Reported-by: Arkadiusz Miskiewicz > Signed-off-by: Dave Chinner > --- > db/metadump.c | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > > diff --git a/db/metadump.c b/db/metadump.c > index a185da5..1769fdf 100644 > --- a/db/metadump.c > +++ b/db/metadump.c ... > @@ -300,6 +316,11 @@ zero_btree_node( > memset(zp2, 0, (char *)block + mp->m_sb.sb_blocksize - zp2); > } > > +/* > + * We could be processing a corrupt block, so we can't trust any of > + * the offsets or lengths to be within the buffer range. Hence check > + * carefully! > + */ > static void > zero_btree_leaf( > struct xfs_btree_block *block, > @@ -312,20 +333,31 @@ zero_btree_leaf( > char *zp; > > nrecs = be16_to_cpu(block->bb_numrecs); > + if (nrecs < 0) > + return; > > switch (btype) { > case TYP_BMAPBTA: > case TYP_BMAPBTD: > + if (nrecs > mp->m_bmap_dmxr[1]) > + return; > + Shouldn't we use the 0 index max recs value (for leaf blocks) throughout this function? (e.g, mp->m_bmap_dmxr[0]) Brian > brp = XFS_BMBT_REC_ADDR(mp, block, 1); > zp = (char *)&brp[nrecs]; > break; > case TYP_INOBT: > case TYP_FINOBT: > + if (nrecs > mp->m_inobt_mxr[1]) > + return; > + > irp = XFS_INOBT_REC_ADDR(mp, block, 1); > zp = (char *)&irp[nrecs]; > break; > case TYP_BNOBT: > case TYP_CNTBT: > + if (nrecs > mp->m_alloc_mxr[1]) > + return; > + > arp = XFS_ALLOC_REC_ADDR(mp, block, 1); > zp = (char *)&arp[nrecs]; > break; > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Mon Jan 4 13:12:10 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 16CA77F37 for ; Mon, 4 Jan 2016 13:12:10 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id B7F13AC002 for ; Mon, 4 Jan 2016 11:12:09 -0800 (PST) X-ASG-Debug-ID: 1451934725-04bdf0264f451fd0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id nFGBa0XyZb03WZax (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:12:05 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id 79A6AA8D; Mon, 4 Jan 2016 19:12:05 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04JC5BW003128; Mon, 4 Jan 2016 14:12:05 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 7305B120068; Mon, 4 Jan 2016 14:12:04 -0500 (EST) Date: Mon, 4 Jan 2016 14:12:04 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 3/9] xfs_mdrestore: correctly account bytes read Message-ID: <20160104191204.GC19852@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 3/9] xfs_mdrestore: correctly account bytes read References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-4-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-4-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451934725 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:03AM +1100, Dave Chinner wrote: > From: Dave Chinner > > Progess indication comes in the form of a "X MB read" output. This > doesn't match up with the actual number of bytes read from the > metadump file because it only accounts header blocks in the file, > not actual metadata blocks that are restored, Hence the number > reported is usually much lower than the size of the metadump file, > hence it's impossible to use to guage progress of the restore. gauge Otherwise looks good: Reviewed-by: Brian Foster > > While there, fix the progress output so that it overwrites the > previous progress output line correctly. > > Signed-off-by: Dave Chinner > --- > mdrestore/xfs_mdrestore.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/mdrestore/xfs_mdrestore.c b/mdrestore/xfs_mdrestore.c > index ebc5e54..70a160c 100644 > --- a/mdrestore/xfs_mdrestore.c > +++ b/mdrestore/xfs_mdrestore.c > @@ -133,7 +133,7 @@ perform_restore( > > for (;;) { > if (show_progress && (bytes_read & ((1 << 20) - 1)) == 0) > - print_progress("%lld MB read\n", bytes_read >> 20); > + print_progress("%lld MB read", bytes_read >> 20); > > for (cur_index = 0; cur_index < mb_count; cur_index++) { > if (pwrite64(dst_fd, &block_buffer[cur_index << > @@ -160,7 +160,7 @@ perform_restore( > 1, src_f) != 1) > fatal("error reading from file: %s\n", strerror(errno)); > > - bytes_read += block_size; > + bytes_read += block_size + (mb_count << tmb.mb_blocklog); > } > > if (progress_since_warning) > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Mon Jan 4 13:12:13 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 564387F58 for ; Mon, 4 Jan 2016 13:12:13 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id DAA29AC002 for ; Mon, 4 Jan 2016 11:12:12 -0800 (PST) X-ASG-Debug-ID: 1451934730-04cbb0101d1f8580001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id TRWRGYW30vWrBOAA (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:12:11 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id C21AFC0B7E03; Mon, 4 Jan 2016 19:12:10 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04JCA8w004730; Mon, 4 Jan 2016 14:12:10 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id AA443120068; Mon, 4 Jan 2016 14:12:09 -0500 (EST) Date: Mon, 4 Jan 2016 14:12:09 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 4/9] repair: parallelise phase 7 Message-ID: <20160104191209.GD19852@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 4/9] repair: parallelise phase 7 References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-5-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-5-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451934731 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:04AM +1100, Dave Chinner wrote: > From: Dave Chinner > > It operates on a single AG at a time, sequentially, doing inode updates. All the > data structures accessed and modified are per AG, as are the modification to > on-disk structures. Hence we can run this phase concurrently across multiple > AGs. > > This is important for large, broken filesystem repairs, where there can be > millions of inodes that need link counts updated. Once such repair image takes > more than 45 minutes to run phase 7 as a single threaded operation. > > Signed-off-by: Dave Chinner > --- Reviewed-by: Brian Foster > repair/phase7.c | 77 ++++++++++++++++++++++++++++++++++------------------- > repair/progress.c | 4 +-- > repair/protos.h | 2 +- > repair/xfs_repair.c | 2 +- > 4 files changed, 54 insertions(+), 31 deletions(-) > > diff --git a/repair/phase7.c b/repair/phase7.c > index b1e3a55..91dad02 100644 > --- a/repair/phase7.c > +++ b/repair/phase7.c > @@ -26,6 +26,7 @@ > #include "dinode.h" > #include "versions.h" > #include "progress.h" > +#include "threads.h" > > /* dinoc is a pointer to the IN-CORE dinode core */ > static void > @@ -108,45 +109,67 @@ update_inode_nlinks( > IRELE(ip); > } > > -void > -phase7(xfs_mount_t *mp) > +/* > + * for each ag, look at each inode 1 at a time. If the number of > + * links is bad, reset it, log the inode core, commit the transaction > + */ > +static void > +do_link_updates( > + struct work_queue *wq, > + xfs_agnumber_t agno, > + void *arg) > { > ino_tree_node_t *irec; > - int i; > int j; > __uint32_t nrefs; > > + irec = findfirst_inode_rec(agno); > + > + while (irec != NULL) { > + for (j = 0; j < XFS_INODES_PER_CHUNK; j++) { > + ASSERT(is_inode_confirmed(irec, j)); > + > + if (is_inode_free(irec, j)) > + continue; > + > + ASSERT(no_modify || is_inode_reached(irec, j)); > + > + nrefs = num_inode_references(irec, j); > + ASSERT(no_modify || nrefs > 0); > + > + if (get_inode_disk_nlinks(irec, j) != nrefs) > + update_inode_nlinks(wq->mp, > + XFS_AGINO_TO_INO(wq->mp, agno, > + irec->ino_startnum + j), > + nrefs); > + } > + irec = next_ino_rec(irec); > + } > + > + PROG_RPT_INC(prog_rpt_done[agno], 1); > +} > + > +void > +phase7( > + struct xfs_mount *mp, > + int scan_threads) > +{ > + struct work_queue wq; > + int agno; > + > if (!no_modify) > do_log(_("Phase 7 - verify and correct link counts...\n")); > else > do_log(_("Phase 7 - verify link counts...\n")); > > - /* > - * for each ag, look at each inode 1 at a time. If the number of > - * links is bad, reset it, log the inode core, commit the transaction > - */ > - for (i = 0; i < glob_agcount; i++) { > - irec = findfirst_inode_rec(i); > - > - while (irec != NULL) { > - for (j = 0; j < XFS_INODES_PER_CHUNK; j++) { > - ASSERT(is_inode_confirmed(irec, j)); > + set_progress_msg(PROGRESS_FMT_CORR_LINK, (__uint64_t) glob_agcount); > > - if (is_inode_free(irec, j)) > - continue; > + create_work_queue(&wq, mp, scan_threads); > > - ASSERT(no_modify || is_inode_reached(irec, j)); > + for (agno = 0; agno < mp->m_sb.sb_agcount; agno++) > + queue_work(&wq, do_link_updates, agno, NULL); > > - nrefs = num_inode_references(irec, j); > - ASSERT(no_modify || nrefs > 0); > + destroy_work_queue(&wq); > > - if (get_inode_disk_nlinks(irec, j) != nrefs) > - update_inode_nlinks(mp, > - XFS_AGINO_TO_INO(mp, i, > - irec->ino_startnum + j), > - nrefs); > - } > - irec = next_ino_rec(irec); > - } > - } > + print_final_rpt(); > } > diff --git a/repair/progress.c b/repair/progress.c > index 418b803..2a09b23 100644 > --- a/repair/progress.c > +++ b/repair/progress.c > @@ -75,9 +75,9 @@ progress_rpt_t progress_rpt_reports[] = { > {FMT2, N_("moving disconnected inodes to lost+found"), /* 12 */ > &rpt_fmts[FMT2], &rpt_types[TYPE_INODE]}, > {FMT1, N_("verify and correct link counts"), /* 13 */ > - &rpt_fmts[FMT1], &rpt_types[TYPE_INODE]}, > + &rpt_fmts[FMT1], &rpt_types[TYPE_AG]}, > {FMT1, N_("verify link counts"), /* 14 */ > - &rpt_fmts[FMT1], &rpt_types[TYPE_INODE]} > + &rpt_fmts[FMT1], &rpt_types[TYPE_AG]} > }; > > pthread_t report_thread; > diff --git a/repair/protos.h b/repair/protos.h > index 9d5a2a6..b113aca 100644 > --- a/repair/protos.h > +++ b/repair/protos.h > @@ -50,7 +50,7 @@ void phase3(struct xfs_mount *); > void phase4(struct xfs_mount *); > void phase5(struct xfs_mount *); > void phase6(struct xfs_mount *); > -void phase7(struct xfs_mount *); > +void phase7(struct xfs_mount *, int); > > int verify_set_agheader(struct xfs_mount *, struct xfs_buf *, > struct xfs_sb *, struct xfs_agf *, struct xfs_agi *, > diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c > index 3eaced4..fcdb212 100644 > --- a/repair/xfs_repair.c > +++ b/repair/xfs_repair.c > @@ -893,7 +893,7 @@ main(int argc, char **argv) > phase6(mp); > timestamp(PHASE_END, 6, NULL); > > - phase7(mp); > + phase7(mp, phase2_threads); > timestamp(PHASE_END, 7, NULL); > } else { > do_warn( > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Mon Jan 4 13:12:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id E7F947F37 for ; Mon, 4 Jan 2016 13:12:17 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 78810AC005 for ; Mon, 4 Jan 2016 11:12:17 -0800 (PST) X-ASG-Debug-ID: 1451934735-04cb6c303042ade0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id wTMLzirxCiYXbBCo (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:12:16 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (Postfix) with ESMTPS id 86FAFA8D; Mon, 4 Jan 2016 19:12:15 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u04JCF6K003173; Mon, 4 Jan 2016 14:12:15 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 6D88B120068; Mon, 4 Jan 2016 14:12:14 -0500 (EST) Date: Mon, 4 Jan 2016 14:12:14 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 5/9] repair: parallelise uncertin inode processing in phase 3 Message-ID: <20160104191213.GE19852@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 5/9] repair: parallelise uncertin inode processing in phase 3 References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-6-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-6-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451934735 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:05AM +1100, Dave Chinner wrote: > From: Dave Chinner > > This can take a long time when there are millions of uncertain inodes in badly > broken filesystems. THe processing is per-ag, the data structures are all > per-ag, and the load is mostly CPU time spent checking CRCs on each > uncertaini inode. Parallelising reduced the runtime of this phase on a badly > broken filesytem from ~30 minutes to under 5 miniutes. > > Signed-off-by: Dave Chinner > --- This one seems a bit more scary simply because the amount of work involved in phase 3, such as inode processing and whatnot. I don't think that everything in there is necessarily AG local as the commit log description above implies. On a skim through, I do see that we have ag_locks[agno].lock for cases that can cross AG boundaries, such as checking block allocation state (get_bmap()/set_bmap()) of bmapbt blocks (iiuc?), for example. So I can't really spot any actual problems, but there's a lot of code down in there. I'm fine with it as long as testing bears out and this gets a reasonable amount of soak time such that we can hopefully catch any serious issues or areas currently lacking sufficient locking: Reviewed-by: Brian Foster > repair/phase3.c | 59 ++++++++++++++++++++++++++++++++++++++++++++--------- > repair/protos.h | 2 +- > repair/xfs_repair.c | 2 +- > 3 files changed, 51 insertions(+), 12 deletions(-) > > diff --git a/repair/phase3.c b/repair/phase3.c > index 76c9440..0890a27 100644 > --- a/repair/phase3.c > +++ b/repair/phase3.c > @@ -28,6 +28,7 @@ > #include "dinode.h" > #include "progress.h" > #include "bmap.h" > +#include "threads.h" > > static void > process_agi_unlinked( > @@ -87,10 +88,33 @@ process_ags( > do_inode_prefetch(mp, ag_stride, process_ag_func, false, false); > } > > +static void > +do_uncertain_aginodes( > + work_queue_t *wq, > + xfs_agnumber_t agno, > + void *arg) > +{ > + int *count = arg; > + > + *count = process_uncertain_aginodes(wq->mp, agno); > + > +#ifdef XR_INODE_TRACE > + fprintf(stderr, > + "\t\t phase 3 - ag %d process_uncertain_inodes returns %d\n", > + *count, j); > +#endif > + > + PROG_RPT_INC(prog_rpt_done[agno], 1); > +} > + > void > -phase3(xfs_mount_t *mp) > +phase3( > + struct xfs_mount *mp, > + int scan_threads) > { > - int i, j; > + int i, j; > + int *counts; > + work_queue_t wq; > > do_log(_("Phase 3 - for each AG...\n")); > if (!no_modify) > @@ -129,20 +153,35 @@ phase3(xfs_mount_t *mp) > */ > do_log(_(" - process newly discovered inodes...\n")); > set_progress_msg(PROG_FMT_NEW_INODES, (__uint64_t) glob_agcount); > + > + counts = calloc(sizeof(*counts), mp->m_sb.sb_agcount); > + if (!counts) { > + do_abort(_("no memory for uncertain inode counts\n")); > + return; > + } > + > do { > /* > * have to loop until no ag has any uncertain > * inodes > */ > j = 0; > - for (i = 0; i < mp->m_sb.sb_agcount; i++) { > - j += process_uncertain_aginodes(mp, i); > -#ifdef XR_INODE_TRACE > - fprintf(stderr, > - "\t\t phase 3 - process_uncertain_inodes returns %d\n", j); > -#endif > - PROG_RPT_INC(prog_rpt_done[i], 1); > - } > + memset(counts, 0, mp->m_sb.sb_agcount * sizeof(*counts)); > + > + create_work_queue(&wq, mp, scan_threads); > + > + for (i = 0; i < mp->m_sb.sb_agcount; i++) > + queue_work(&wq, do_uncertain_aginodes, i, &counts[i]); > + > + destroy_work_queue(&wq); > + > + /* tally up the counts */ > + for (i = 0; i < mp->m_sb.sb_agcount; i++) > + j += counts[i]; > + > } while (j != 0); > + > + free(counts); > + > print_final_rpt(); > } > diff --git a/repair/protos.h b/repair/protos.h > index b113aca..0290420 100644 > --- a/repair/protos.h > +++ b/repair/protos.h > @@ -46,7 +46,7 @@ void thread_init(void); > > void phase1(struct xfs_mount *); > void phase2(struct xfs_mount *, int); > -void phase3(struct xfs_mount *); > +void phase3(struct xfs_mount *, int); > void phase4(struct xfs_mount *); > void phase5(struct xfs_mount *); > void phase6(struct xfs_mount *); > diff --git a/repair/xfs_repair.c b/repair/xfs_repair.c > index fcdb212..5d5f3aa 100644 > --- a/repair/xfs_repair.c > +++ b/repair/xfs_repair.c > @@ -871,7 +871,7 @@ main(int argc, char **argv) > if (do_prefetch) > init_prefetch(mp); > > - phase3(mp); > + phase3(mp, phase2_threads); > timestamp(PHASE_END, 3, NULL); > > phase4(mp); > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From jwollrath@web.de Mon Jan 4 13:26:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 157F67F37 for ; Mon, 4 Jan 2016 13:26:18 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id CBA4F304032 for ; Mon, 4 Jan 2016 11:26:17 -0800 (PST) X-ASG-Debug-ID: 1451935572-04bdf02651452470001-NocioJ Received: from mout.web.de (mout.web.de [212.227.17.12]) by cuda.sgi.com with ESMTP id KQIzDIggIjQ4HT1y (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:26:13 -0800 (PST) X-Barracuda-Envelope-From: jwollrath@web.de X-Barracuda-Apparent-Source-IP: 212.227.17.12 Received: from saldaea ([134.76.62.130]) by smtp.web.de (mrweb102) with ESMTPSA (Nemesis) id 0MDg9K-1aRm9q0Peg-00H4Xp for ; Mon, 04 Jan 2016 20:26:12 +0100 Date: Mon, 4 Jan 2016 20:26:11 +0100 From: Julian Wollrath To: xfs@oss.sgi.com Subject: Suspend does not work with v4.4-rc8 Message-ID: <20160104202611.5b100540@saldaea> X-ASG-Orig-Subj: Suspend does not work with v4.4-rc8 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:2kSjhqlK0Kw1JMzJGByIyNvPQbv7+/diV8ECM802X9rFF48eeJD qvXVDg8+aNHLfPSzVljYQaGg4qhVOGvTqd/koRgKP/RXUREnCH3UXdhZr8QAPClvRG58IeH k5lBhdRLMpC17hRlRrYDsaEycdvRoHbX+W6erCjUXW5Jdmd0UMn0VWkJWVxeBoLSjrN36RU BATXtPg0Ywmt9tAsQWQAg== X-UI-Out-Filterresults: notjunk:1;V01:K0:6tYB1zneba8=:sNWbiR7WxIiPTDirPhpK8H cE9UD2e65eP+gXOZprZ7u7xocDkc8T8tZwY5U9W6gyx0LE3dgt4w6d97JErDf0g9YUwth1emH L9uft5vUW1gRfsjfujXo6iSfHSuAFZBZtMwm5E07i8nI7YcBXTgsGDp6pPXCXrsI+Acv1Ov7Y tL1+JK1T5v38NAwQHzm0S5pLmabGAYSZY24PYHwK9yVcdYTznA/74/+bzWZwpFnB58W6P2pq+ gPCQcU+Z5UQMkxXSwXPgLA2SZGFpq4YX1j+p5LoZDfoixMb17dXMe6KSaXHoWhh9jAtaFK8kO D1zB/0dWndUOWslYWxOReKFyLDj4b0Dqk7KplEmR4sMKsq06jOhZrYI5wPgYq5SvH+SM2pXFa bNEzIDkL/i+Zx923ViyFrCJiUGAj4azzyp4Ssw4rJToKSWsrG2ldtxnn+MpeOBZMhBtHt5Jya JCkN0jsAoZY6YK5P3uG9twv64Gw5IMxDvFuSasM30VUkK93IfcW/p8pimzJPr3kOJ4f0kBVrB fUhrX286dy1dBNMjfJEI77uSnwLtmjIbXX8tpwkK4i7Z0D9XtFKobuUX5duYca5uZlRXivLbY FB/XnvpwFHH3EEWDYzjLVHFQ8SMR/owpaEV/i5eOFV+LFxrtLpwlosEEkzZG+7aBxNoeh7549 4g0WypLFYa90aWK1RE+kvykoqhC61Adx4X85SpPJkviXorpObMselbfb7MLI995rjkrJAhuuS UZ0uXn7FPpdsc5iHpvjxcgPpOnLQqyoh85yDpHwgtRm22mUCDCf06IbcbDYKy1k2pRcHXhsGN 9RP+npnuAAnKiGGO35+NvZ/je/BtQ== X-Barracuda-Connect: mout.web.de[212.227.17.12] X-Barracuda-Start-Time: 1451935573 X-Barracuda-Encrypted: DHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Hi, suspending (either to RAM or disk) fails with 4.4-rc8 with the following message while v4.3.3 worked fine: kernel: Freezing of tasks failed after 20.006 seconds (2 tasks refusing to freeze, wq_bu_busy=0): kernel: xfsaild/dm-1 S 0000000000014100 0 283 2 0x00000000 kernel: ffff880213f53e10 ffffffff8180e4c0 ffff880213f05040 0000000000000000 kernel: 0000000000000000 ffff880213f54000 0000000000000000 0000000000000000 kernel: ffff8800ca389e40 ffff8800ca392000 ffff880213f53ed0 ffffffff814cd8ac kernel: Call Trace: kernel: [] ? schedule+0x2c/0x70 kernel: [] ? xfsaild+0x4fd/0x5b0 kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 kernel: [] ? kthread+0xb8/0xd0 kernel: [] ? kthread_worker_fn+0x150/0x150 kernel: [] ? ret_from_fork+0x3f/0x70 kernel: [] ? kthread_worker_fn+0x150/0x150 kernel: xfsaild/sda1 S 0000000000014100 0 591 2 0x00000000 kernel: ffff88021193be10 ffff8802159c4dc0 ffff880213ab9340 0000000000000000 kernel: 0000000000000000 ffff88021193c000 0000000000000000 0000000000000000 kernel: ffff8800ca2a4240 ffff880214eea000 ffff88021193bed0 ffffffff814cd8ac kernel: Call Trace: kernel: [] ? schedule+0x2c/0x70 kernel: [] ? xfsaild+0x4fd/0x5b0 kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 kernel: [] ? kthread+0xb8/0xd0 kernel: [] ? kthread_worker_fn+0x150/0x150 kernel: [] ? ret_from_fork+0x3f/0x70 kernel: [] ? kthread_worker_fn+0x150/0x150 Please tell me what more information you need to be able to fix this issue. With best regards, Julian Wollrath From MAILER-DAEMON Mon Jan 4 13:36:34 2016 Return-Path: <> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.9 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLYTO, TO_MALFORMED,T_FREEMAIL_DOC_PDF autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 9CEE77F37 for ; Mon, 4 Jan 2016 13:36:34 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 5692F8F8035 for ; Mon, 4 Jan 2016 11:36:34 -0800 (PST) X-ASG-Debug-ID: 1451936189-04bdf026514527e0001-NocioJ Received: from mail.gaf.ni.ac.rs (mail.gaf.ni.ac.rs [160.99.31.9]) by cuda.sgi.com with ESMTP id khG8mOhaeD5cGtcN (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:36:31 -0800 (PST) X-Barracuda-Envelope-From: X-Barracuda-Apparent-Source-IP: 160.99.31.9 Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 0E0E68F315729; Mon, 4 Jan 2016 20:29:19 +0100 (CET) Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id JILdZ19_CQXK; Mon, 4 Jan 2016 20:29:18 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 9B72D8F3A824F; Mon, 4 Jan 2016 20:27:38 +0100 (CET) X-Virus-Scanned: amavisd-new at gaf.ni.ac.rs Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ODRxRiRrw92O; Mon, 4 Jan 2016 20:27:38 +0100 (CET) Received: from [100.67.150.48] (unknown [116.202.38.88]) by mail.gaf.ni.ac.rs (Postfix) with ESMTPSA id 4A0998F2F1C3E; Mon, 4 Jan 2016 20:25:22 +0100 (CET) Content-Type: multipart/mixed; boundary="===============1096080894==" MIME-Version: 1.0 Subject: Re To: Recipients <> X-ASG-Orig-Subj: Re From: "<"@LIVE.COM Date: Tue, 05 Jan 2016 00:55:08 +0530 Reply-To: cruzcelia2011@gmail.com Message-Id: <20160104192523.4A0998F2F1C3E@mail.gaf.ni.ac.rs> X-Barracuda-Connect: mail.gaf.ni.ac.rs[160.99.31.9] X-Barracuda-Start-Time: 1451936190 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 2.00 X-Barracuda-Spam-Status: No, SCORE=2.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0646, BSF_SC0_MV0646_4, EMPTY_ENV_FROM, NO_REAL_NAME, TO_MALFORMED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 TO_MALFORMED To: has a malformed address 0.00 EMPTY_ENV_FROM Empty Envelope From Address 0.00 NO_REAL_NAME From: does not include a real name 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.50 BSF_SC0_MV0646 Custom rule MV0646 1.50 BSF_SC0_MV0646_4 BSF_SC0_MV0646_4 You will not see this in a MIME-aware mail reader. --===============1096080894== Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body please kindly open the fill --===============1096080894== Content-Type: application/octet-stream MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="BUSINESS PROPOSAL.docx" UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0 VMtuwjAQvFfqP0S+Vomhh6qqCBz6OLZIpR9g7A1Y9Uv28vr7bgJEVQtBKuUSKVnvzOzsxIPR2pps CTFp70rWL3osAye90m5Wso/JS37PsoTCKWG8g5JtILHR8PpqMNkESBl1u1SyOWJ44DzJOViRCh/A UaXy0Qqk1zjjQchPMQN+2+vdcekdgsMcaww2HDxBJRYGs+c1fd4qiWASyx63B2uukokQjJYCSSlf OvWDJd8xFNTZnElzHdINyWD8IENdOU6w63sja6JWkI1FxFdhSQZf+ai48nJhaYaiG+aATl9VWkLb X6OF6CWkRJ5bU7QVK7Tb6z+qI+HGQPp/FVvcLnrSOY4+JE57OZsf6s0rUDlZESCihnZ1x0cHRLLs EsPvkLvGb1KAlHfgzbN/tgcNzEnKin6JiZgaOJvvV/Ja6JMiVjB9v5j738C7hLT5kz7+wYz9dVF3 H0gdb+634RcAAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJlbHMg ogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfLT9ab g5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+ImYz sKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kVyAdh b9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ5x687 HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAgBd29yZC9f cmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AKySzWrDMBCE74W+g9h7LTv9oYTIuZRArq37AIq9/qGyJLSbtn77CkNShwb34otgRmjmk7Sb7Xdv xCcG6pxVkCUpCLSlqzrbKHgvdnfPIIi1rbRxFhUMSLDNb282r2g0x0PUdp5ETLGkoGX2aympbLHX lDiPNu7ULvSaowyN9Lr80A3KVZo+yTDNgPwiU+wrBWFf3YMoBh+b/892dd2V+OLKY4+Wr1TILzy8 IXO8HMVYHRpkBRMzibQgr4OslgShPxQnZw4hWxSBBxM/8/wMNOq5+scl6zmOCP62j1KOazbH8LAk Q+0sF/pgJhxn6wQhLwY9/wEAAP//AwBQSwMEFAAGAAgAAAAhAKsR9BCDCgAAyx0AABEAAAB3b3Jk L2RvY3VtZW50LnhtbNRYzW7bRhC+F+g7DIQcWkCWKAcuEjVW4FpO40NTI7ZzX5FLcWFyl91dSuGt r9FbniWP0ifpN0tSliXFFgSjRU+0l9z5n++b0Zu3n4ucFtI6ZfRpbzSIeiR1bBKl56e925t3R696 5LzQiciNlqe9Wrre28n3371ZjhMTV4XUniBCu/ECbzPvy/Fw6OJMFsINTCk1XqbGFsLjXzsfFsLe VeVRbIpSeDVTufL18DiKfuq1Ysxpr7J63Io4KlRsjTOp5ytjk6Yqlu2ju2H30dvcnLYmB41DK3PY YLTLVOk6acWh0uBi1glZPObEosi775blPtoSK5bIR5E3Zi+NTUprYukcTqfNy5XEUfSY7jaALGJ1 Yx8THursLCmE0isxXB0b+V8lb4DkDRvdQxZ17whiMUEtzUxS87Ok5Ri1mHw87UXR+Un07mLa646m MhVV7vnNL+fHr6Yvw83yyvJF2zxmwyDt3OE5bE/x7D564svl2E9+tVJ6xNUN6CqXwkkqpS2Up0KS N6S0tyapYklF7WSe8lltKjKaj4yVA7okUZDA94mKRU4xCgxmCzTKrCYkLuXEGU1oKkIw7thQ35gb 7Ov8ZUePR6PXoyg4+oiHuM/dM3aliJGF0kon7UL2Jo6EY5VxZWVo1UJoMZeWlspn9Jjily9P3p1M 91A8uS6FVi6jq0ygzWNZeXZ7U/bTkd/tAZ0zUuiaw7qEM0rTpU6U4D+Oo9GIAC4INnssbJxRWdnS IGlJZZFEWmYKh5eUKBcbwJxMyGfC0w+b9h0U9MkNlAj6JK34MVgnFkLlYpbLlZ2D51A0s205H1wf QQKX2c2mPYcmJni+KeygKO5OPAd1q0KfNnZyLZFj5ahyeHJxhD5EmvKuNlx/w2w0iEkvLLpi7OsS DTS3orj2wvpegJOnAIbj+vULUr6P3Aud7C11d2TI+IxbGDiKfvDsrNDQL2KvFpLwbsbecwmi3asU 500zmJTOtFf0/vIT3eAzUdZ9eg+PrZc6QNIV2FlpB3C6mJlc0FShrZx0dOHjAe5AlWvj60oZK7Cx yPMaca4AZlAIyyhTfCSsFASN16YC1FwI5+nMKdEPsCfiGNZzgwI+0TDGCm9sTV7is1UvszAMHZRa U7RdnwECZlJqBrUFHrg+Ay7jwxmuMkhkAjFYKKc88t9ABdzxGWCdTBwL9tPxPSBFDHF8u3OLEZll JZhe8CcrA3qDDSrGDUMW4SA4jFBjKEFUKosRJ3h0q4NCVA18YL/PCmkRnYBbCEgwWCWBQ5gvWA3L RnKgzgM2+BKf5oZZI5EiR5JF6jnVEkERADIxhxsByGENcwesmkvcsls482SrPASV/T7nSr9EcVUh 6zsimG60APfUis/3JrMJ2vYjCPhl1KcoioZHzMEEcruTnsM02tTypPXf4MdoXjTVtRn4Ps0qJKYK +VLekRUWk2pINX+bqXnWlkkfRYdJIN8Cq8Ncv69GpPi3uuO/AGUvthxf17E/ZffpJAqhfa440u01 TU2eC+s2RR5m4dcvaMBNUYdmme6kLENv8fQFxAy0EMa3tbECYGDnwIj0HhkYBFxVloAzPg6DH2qh AiTk7TyCw8Iw1CBbgrRcoipKMEdoUgDiVMaymKF6c0bAGkMK46hcpZUxRpslibI0mC0hqJECudtD W4amZywJ8HY/By2lvHN///kXedUMqduN+e/BAzunnEPnWPSID84BX8PUhTnM4MBjVsI0zH3EAdvl KqIJPMckxZNloqwETGKHKCoNTOWdrZlgWcQ6XqJtZSyY/B6qylTBCUoVEgo+CqqZFnkoAEqHrLMs JrcmP82gHLKI1Bf3fQhfPvx+Q2IJgusge1vUOqOEfQBgbRWXCU+fnefNRgSM5KTKz2FwlZhSa4AA uFGlIHOEDITywHJwQWt3y8nQxlEFj8I3pojk524wCEfe1i3XItaeYwmHuNrmDGor1rxDITJVsjjk pdsVuFtSniy4ennWAK/yivNHBcpjElsoLD+ta6CqWGi2JmY8CMweRpEgdlZhZcX687AceUHj3v6v l79uSN49dgGL0aQcelQSUzjCxKHBn+BprHT8MwlKCW/X+q+pqd0k07GJBrsfBQoBNuE6C2zXnm82 yDLD9IPqECirBQoEUBOGnaZ613uiqa4gvivxzoNNgF3H6v3JmoPQsvUzkfUEvNwwMFu8Cx/QMc9j fNgKmjmQiYEnTc4tfMJE/WLUP9lU8zQJTcCt0ea1w0Ib0e1gxawwqFlnUXhNq7/eUrOHdX0kiS4/ fAyVAgwL4NHCoJZz41UDsMjqDIwolGbsAAhiCydsB0k/4Nc3ssK/d+Q5Q0ybOvxWshZXBo9S1CHG mMWz/yMQnIWZ/B8AAAD//6xWXW7bOBC+ysBA31xXTtLYzTYG6iQt+rCA0e4FKGkkEZZIlaSiuk+9 Rt/2LD3KnmS/oawkNookBQokkvgzw5lvvm/oUDG1zmad44ZNoEYZVbKjSnlqO5fhzTnJLs/4KJxt aGe7qTyo13VNOeddFuKWe4PM+kDK5LLN0dtX/UVYydOt3uJB+Pc6/3Q5SZLzk/nN4noS5zdxOX0l g/TK4y0mw2xYBaduua61KY/9PW1MX5v6wrcq48tJ69izu+XJivhryybXAdlPY7hI1AypOFvoMGSY IvlKOaSfcugZOzpPKtBp8oJe0iJ5MaW+0llFDSvj75EJasuyLA7Nz3/p4+AuTsN2Fqccq7reUW7J WJynUINgqbaeyXaBrISjPaWd14Y9PjhTHRYRHKaVIdu21oXO6LCjwjoEIQ4aObuxhjEpNZMyZLZp ldnN6J1HLF7pnFi5WrObxpxzRvVzssVwJGgRK4tj5kmSENDbcvB/jUenEl9le+HMblyN2WcIa+TC cLwQqLaZErqoejzQdE0KruHAve9ofWe5p1dg1yD3GKJqbAeExGJfIB5ALTnMDmgB5rTCi/Yh2c7W i+Wbm8k4dc2F6uogNFxfnSyvTwcaPkmn1EWKhl+z6iOpBlibQueiqFCBKr0O1aCFCjXxKJUgjZJm 7KRwklDnWql6v88oVrC0FqLTRplMA7xSaUP4O6DEbCRW2eHECCFYU3QOoLsobigXqgFbPHmLNbxF vjFGF/XsCDzEJmEY2HOA5Kjax9RZKx8+QUoMlWzQQNZwt30MpN667YBKwzNag0kqtbdMEMMU+Wy1 yaEKx186oBWjRVhb5nbIHVRuQPlMBY187sAGRjPa1IymJYryiEj0EJsQNgWFRrUnneB/nObIi9/r SyrP0VE86eIxfyfz+Zt58ow+92tS/aFYgcR/3384tA8DWQFazo+DfpL9YTWjfyplttJ+cvoAhqY1 ADjU33NYM+potT4O4mElno3cSqjiuFQu99M/4fEuvncQ6TdFV7Vy6tjxc/A6sBk7k+csbA7uwsP2 9BnrwsTXN4vk9fvInLb8/A3Q9JcTsCk5l0ZW4ft8ebqcRLW15d9KXAbbYv7sLIm9TpcVPI3D1IZg m/txzcWD1Qp9gN3lZJEsxbawFkS5G5ZdiMNkOC6ztcem/cUqJjGK3GYfHG6X/gK3NW90yBDl6Xk0 QvZD4vFyT22+ix8w6eQHyOp/AAAA//8DAFBLAwQUAAYACAAAACEAlrWt4pYGAABQGwAAFQAAAHdv cmQvdGhlbWUvdGhlbWUxLnhtbOxZT2/bNhS/D9h3IHRvYyd2Ggd1itixmy1NG8Ruhx5piZbYUKJA 0kl9G9rjgAHDumGHFdhth2FbgRbYpfs02TpsHdCvsEdSksVYXpI22IqtPiQS+eP7/x4fqavX7scM HRIhKU/aXv1yzUMk8XlAk7Dt3R72L615SCqcBJjxhLS9KZHetY3337uK11VEYoJgfSLXcduLlErX l5akD8NYXuYpSWBuzEWMFbyKcCkQ+AjoxmxpuVZbXYoxTTyU4BjI3hqPqU/QUJP0NnLiPQaviZJ6 wGdioEkTZ4XBBgd1jZBT2WUCHWLW9oBPwI+G5L7yEMNSwUTbq5mft7RxdQmvZ4uYWrC2tK5vftm6 bEFwsGx4inBUMK33G60rWwV9A2BqHtfr9bq9ekHPALDvg6ZWljLNRn+t3slplkD2cZ52t9asNVx8 if7KnMytTqfTbGWyWKIGZB8bc/i12mpjc9nBG5DFN+fwjc5mt7vq4A3I4lfn8P0rrdWGizegiNHk YA6tHdrvZ9QLyJiz7Ur4GsDXahl8hoJoKKJLsxjzRC2KtRjf46IPAA1kWNEEqWlKxtiHKO7ieCQo 1gzwOsGlGTvky7khzQtJX9BUtb0PUwwZMaP36vn3r54/RccPnh0/+On44cPjBz9aQs6qbZyE5VUv v/3sz8cfoz+efvPy0RfVeFnG//rDJ7/8/Hk1ENJnJs6LL5/89uzJi68+/f27RxXwTYFHZfiQxkSi m+QI7fMYFDNWcSUnI3G+FcMI0/KKzSSUOMGaSwX9nooc9M0pZpl3HDk6xLXgHQHlowp4fXLPEXgQ iYmiFZx3otgB7nLOOlxUWmFH8yqZeThJwmrmYlLG7WN8WMW7ixPHv71JCnUzD0tH8W5EHDH3GE4U DklCFNJz/ICQCu3uUurYdZf6gks+VuguRR1MK00ypCMnmmaLtmkMfplW6Qz+dmyzewd1OKvSeosc ukjICswqhB8S5pjxOp4oHFeRHOKYlQ1+A6uoSsjBVPhlXE8q8HRIGEe9gEhZteaWAH1LTt/BULEq 3b7LprGLFIoeVNG8gTkvI7f4QTfCcVqFHdAkKmM/kAcQohjtcVUF3+Vuhuh38ANOFrr7DiWOu0+v Brdp6Ig0CxA9MxHal1CqnQoc0+TvyjGjUI9tDFxcOYYC+OLrxxWR9bYW4k3Yk6oyYftE+V2EO1l0 u1wE9O2vuVt4kuwRCPP5jeddyX1Xcr3/fMldlM9nLbSz2gplV/cNtik2LXK8sEMeU8YGasrIDWma ZAn7RNCHQb3OnA5JcWJKI3jM6rqDCwU2a5Dg6iOqokGEU2iw654mEsqMdChRyiUc7MxwJW2NhyZd 2WNhUx8YbD2QWO3ywA6v6OH8XFCQMbtNaA6fOaMVTeCszFauZERB7ddhVtdCnZlb3YhmSp3DrVAZ fDivGgwW1oQGBEHbAlZehfO5Zg0HE8xIoO1u997cLcYLF+kiGeGAZD7Ses/7qG6clMeKuQmA2Knw kT7knWK1EreWJvsG3M7ipDK7xgJ2uffexEt5BM+8pPP2RDqypJycLEFHba/VXG56yMdp2xvDmRYe 4xS8LnXPh1kIF0O+EjbsT01mk+Uzb7ZyxdwkqMM1hbX7nMJOHUiFVFtYRjY0zFQWAizRnKz8y00w 60UpYCP9NaRYWYNg+NekADu6riXjMfFV2dmlEW07+5qVUj5RRAyi4AiN2ETsY3C/DlXQJ6ASriZM RdAvcI+mrW2m3OKcJV359srg7DhmaYSzcqtTNM9kCzd5XMhg3krigW6Vshvlzq+KSfkLUqUcxv8z VfR+AjcFK4H2gA/XuAIjna9tjwsVcahCaUT9voDGwdQOiBa4i4VpCCq4TDb/BTnU/23OWRomreHA p/ZpiASF/UhFgpA9KEsm+k4hVs/2LkuSZYRMRJXElakVe0QOCRvqGriq93YPRRDqpppkZcDgTsaf +55l0CjUTU4535waUuy9Ngf+6c7HJjMo5dZh09Dk9i9ErNhV7XqzPN97y4roiVmb1cizApiVtoJW lvavKcI5t1pbseY0Xm7mwoEX5zWGwaIhSuG+B+k/sP9R4TP7ZUJvqEO+D7UVwYcGTQzCBqL6km08 kC6QdnAEjZMdtMGkSVnTZq2Ttlq+WV9wp1vwPWFsLdlZ/H1OYxfNmcvOycWLNHZmYcfWdmyhqcGz J1MUhsb5QcY4xnzSKn914qN74OgtuN+fMCVNMME3JYGh9RyYPIDktxzN0o2/AAAA//8DAFBLAwQU AAYACAAAACEAoQmDWCUDAAAfBwAAEQAAAHdvcmQvc2V0dGluZ3MueG1snFXbbts4EH1fYP9B0PM6 lmQ7ToQ4RWNb3RZJW6zSD6CkkUyEN5CUFffrO5TEKul6g2KfTJ0zczicm2/ePXMWHEEbKsUmjC+i MABRyoqKZhN+e8xmV2FgLBEVYVLAJjyBCd/d/vnHTZcasBbNTIASwqRyE7ZapKY8ACdmxmmppZG1 nZWSp7KuaQnjTzh66E14sFal8/nodCEVCFSrpebEmgupm/nguZNly0HYeRJFl3MNjFgM2ByoMl6N /181vOrgRY5vPeLImbfr4ugty/G5ndTVT4/fCc85KC1LMAYzy9nwXE6o8DKG/Y7OkM97WmiiTy9E brFs36XkQZcq0CUmFGseReHcEZX8LO2OGsXI6Stp4E62WHZNwfQ0xiXr3BIL6G0UMNb3SMmAYHRd 2mjCOcGaDsggCTVpmX0kRW6lQqMjwfjXyXhjeSCalBZ0rkiJalsprJbM2/UBbSVXGvMxxIi9pIh1 0bYGsv09OcnW4lXzLp0obObKOBt3+EdK6wWj6G6bXO0Wg5ZjJyaJ4+t4jOs1s1isstXunM/ybn11 vT/HrPbraJWdYy6TeL8+q/Y+W27jsz7/HfV2FWX7s2rZ1fI6XrsIMDdjRnjqev2rvr0ZThnmO+BD UbaEF5qS4MFNA3rxtNBPd1R4vgCcSnjJ5G3hydlsIAwnjGVYU0/gIAxMhX21g7oXZg9EN5Nyn3Se 6rNoBfWnn2quYUF/0LJVg2qnifooKoT9hfFyOepRYe8p97hpi9x7CRyKFxR2+ZejdoLzKUFdanGP gcvQPRGN7xMQs2+5M8V+Yzp3uw4eiFLYvGhSNPEmZLQ52NhNhMUvHKCn/qNokpFLeg6/HNd/kNK9 DK3HgzMYjmg1HiZs4bHFhC09tpywlcdWE3bpsUuHHU64BXCMn3Cn+KPDa8mY7KD624Ob8F/QkARz IAqwrm7KceRk2gNYtB4Ijik844qBilr8G1G04uR5E67iJHHuozWuGxziV7aOc8bqFRpUxBJcWH2p Xjn3Tf5LLLjQoKTYkPmJF9NSuRgCZ9TYHBTuHys1PrlfTH/1ytM/2+0PAAAA//8DAFBLAwQUAAYA CAAAACEAb2NQm4sBAAAHBAAAEgAAAHdvcmQvZm9udFRhYmxlLnhtbKSS326CMBTG75fsHUjvJwX/ TI1onNPLXSzuAY5YpAltSU+V+fY7UORiZolkkDThO+3H6e98i9W3KoKLsCiNTlg04CwQOjVHqU8J +9rvXqYsQAf6CIXRImFXgWy1fH5aVPPMaIcBndc4twnLnSvnYYhpLhTgwJRCUy0zVoGjT3sKTZbJ VLyb9KyEdmHM+SS0ogBH/8Zclshat+oRt8rYY2lNKhCpWVV4PwVSs2XbXVDNNSjqegOFPFjZFErQ BkVEtQsUCeMx3/ExrfU74sN6ZWHtkOZgUbhuI/dyBkoW15uKlUT0hVK6NL/pF7ASDoXwJZQnKpzx wBO2jTjn8W7HvBIlbETCetMpMTXln1m7Z9gpNB5qrPFptkSzxocU8mlPNX2Gfj53JPZSCQw+RBV8 GgUe1T2RmE+IxJh41GSGvYjYxrch+CgRajxed/enm2xIeZ2Oovb+vYh4n8eJbEBRNOCPbNQEPIma SL9s9CexrkcYb39lg/PR2x2JJgmUqP9kow0JLn8AAAD//wMAUEsDBBQABgAIAAAAIQAoh3GlzwAA AB8BAAAUAAAAd29yZC93ZWJTZXR0aW5ncy54bWyMj8tOAzEMRfdI/MMoe5qBRYVGnamEUNlQqMRj n2Y8nUiJHdmB0H495rFhx/LaV8fHq/VHis07sATC3lwuWtMAehoDHnrz8ry5uDaNFIeji4TQmyOI WQ/nZ6vaVdg/QSnalEYpKB33Zi4ld9aKnyE5WVAG1N1EnFzRyAdL0xQ83JJ/S4DFXrXt0jJEV9RA 5pDF/NLqf2iVeMxMHkRUJMUfXnIBzaCOlEtI4QQb4humKsD2a6z3jo/4ur3/Ti5GqruHOw32z1vD JwAAAP//AwBQSwMEFAAGAAgAAAAhAPeMfebgAQAA2wMAABAACAFkb2NQcm9wcy9hcHAueG1sIKIE ASigAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnFPBbtswDL0P2D8Yvjd2nCUoAkXFkGLo YVsDxG3PnEzHwmRJkNSg2dePshtX2XqqT+Qj/fT0SLGbl15lR3ReGr3J57Myz1AL00h92OQP9ber 6zzzAXQDymjc5Cf0+Q3//IntnLHogkSfEYX2m7wLwa6LwosOe/AzKmuqtMb1ECh1h8K0rRR4a8Rz jzoUVVmuCnwJqBtsruxEmI+M62P4KGljRNTnH+uTJcGc1dhbBQH5zyhHsWICWG0CqFr2yBcl4VPG dnBAz+esGAP2ZFzj+bIkZAzZtgMHIpB7vLpe0d8JwL5aq6SAQMbyH1I4400bsvvBgiwSsCJtYWTL HsWzk+HEiSpN2XepSUq1YMUYkTYHBwe283wVBU4Z2wtQuKXL8xaUR1a8AewOIQ52B5IUs2NYH1EE 4zIv/9Boqzz7BR6jZZv8CE6CDmRdbBuTIVbWB8drGRRxU23MhzBtS2P5JbpIvRRcNkZw1ECFS3XD Cf6+pbuFd8TOU7GDhlFqIicJpzP+Yd2a3oI+0eFTRAb/9g+2NrdxXV49vASTuT/J0O0tCJrOYrFc phuQlNieFgUbGumZ8A1gd+S3U/FU2h59wObc838h7tTj+Fb5vJqV9A1LdMZoE6ZHxP8CAAD//wMA UEsDBBQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAgBZG9jUHJvcHMvY29yZS54bWwgogQBKKAAAQAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACMkstuwjAQRfeV+g+R94mTAC2ykiD1wQraSk0f6s6y B4gaP2S7BP6+TgIpqF10Ob7XZ+6Mnc12og62YGylZI6SKEYBSKZ4Jdc5einn4RQF1lHJaa0k5GgP Fs2Ky4uMacKUgSejNBhXgQ08SVrCdI42zmmCsWUbENRG3iG9uFJGUOdLs8aask+6BpzG8RUW4Cin juIWGOqBiA5Izgak/jJ1B+AMQw0CpLM4iRL843VghP3zQqecOEXl9trPdIh7yuasFwf3zlaDsWma qBl1MXz+BL8vF8/dqGEl210xQEXGGWEGqFOmqEGqrcrwyVG7vppat/SbXlXAb/bF4v7h8fUxw7+V 1mxgW7VvVEw6x1D6Pt1YfTPggQ9K+rGOytvo9q6coyKNk0mYpGEyLuNrMhmROP5oQ53db4P3B+IQ 7R/EdFqmMUnG58QjoOgSn3+W4hsAAP//AwBQSwMEFAAGAAgAAAAhAOYmwWoaBwAA+zkAAA8AAAB3 b3JkL3N0eWxlcy54bWy0m1FT2zgQx99v5r6Dx+80ISnJlWnaoRSuzLQcbWDuWbEVoqlj5SylQD/9 SStbGBvbu9g8EcvW/lba1X+dIL3/eL9Ngl88U0Kmi/DwzTgMeBrJWKS3i/Dm+vzgrzBQmqUxS2TK F+EDV+HHD3/+8f7uWOmHhKvAGEjVcbYIN1rvjkcjFW34lqk3csdTc28tsy3T5jK7Hcn1WkT8s4z2 W57q0WQ8no0ynjBt4GojdirMrd1hrN3JLN5lMuJKGW+3ibO3ZSINPxj3Yhl95mu2T7Syl9lVll/m V/DnXKZaBXfHTEVCXBvHzRC3IpXZl5NUidDc4UzpEyVY+eZZ3mbvb+yD5Zu+Z6R0yeAnEYtwZKHq t+n2iyWLcDIpWk6tE0/aEpbeFm08Pbi4LDuzCH3TythdhCw7WJ5YYyMYafG3NOLdk/GbK3BlxyIz d4bD1pqbGJqQWE4ibKwn81lx8WOfmAa21zKHgAEDK5s1l5VJN6E1gV66RDF3+fqrjH7yeKnNjUUI LNN4c3GVCZkJ/bAI372zTNO45FvxRcQxt3mZt92kGxHzfzc8vVE8fmz/fg5ZlluM5D7Vxv3ZHBIh UfHZfcR3NsuM6ZTZIF/aDok1q0occGgvHr1xDRUqNP5XIA9dDJ+lbDizKykA/1tBMOp9b9DEjqg8 ALBL8nXa38Tb/iaO+puA5O03F/P+Xhj97BsRlxulrMQHVcvIJV95HqbvWlLW9qhlUWePWtJ09qjl SGePWkp09qhlQGePWsA7e9Ti29mjFs7WHhED4apm0RRmA7Wwr4VOuO3fKkCHPaUuLzXBFcvYbcZ2 m8DW1qrbbWK53K80zlWQ05eL5VJnMr3tnBFTne3SfbEmn213G6aEeanpmPpJz6m/ZquEB39nIu5E Hbnkq40JXkyeLWFXCYv4RiYxz4Jrfu8iSuh/KYOle8vodK5nWL+K240OlhsouZ2wWcOkN8+Es/9V KJiD1sU0axhKl3FUDGcNedls/BuPxX5bTA3ibWTm9JwQ5goCXGyforc2RPXV1TkKGwDMEFy5oA8B 7CP8d8WFbt/GGOO/K0UvtI/w3xWuF9qH/GiPL1lpPrPsZ4BaXnPy2j2ViczW+6RYA53yMCevYI/A DYG8iL19lEjMySv4iXwGJ1Fkvrlh8pQci0cdJVDI4XAUWGz4sZCDUpG9Q8KIyAGqsCYEVj+tJYDI ovuD/xL2tydqMQCV9u+anct52jADpgSh3qG/76XufoeeNGgelnKRmp9LFA9wtGnDysPS8nxy9Y4Q 436FjwDqVwEJoH6lkABqyI/mdx5fE/GQ/sWRwCLLsq9ikHZoZZ6TldmDaCVgoLqJeP9qWL3NuVCv mwgKOUD1uomgkKNTqWW+biJYg9VNBKuhajTHqKyplEGR62YZ5N8EECMaRrwRoGHEGwEaRrwRoP7i 3Q0ZTrwRLLI2eE0tizcCBI9Qvup7UFm8ESCyNji1y38zKuoeWGn/cjuAeCMo5ADVxRtBIUenSbwR LHiEkgkVlpc6BGsY8UaAhhFvBGgY8UaAhhFvBGgY8UaA+ot3N2Q48UawyNrgNbUs3ggQWR48qCze CBA8QtGGZ8UbVv2rizeCQg5QXbwRFHJ0KoLqX1IRLHKAKiwv3ggWPEJJhpwFyU0Z1DDijRjRMOKN AA0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwRILI8eFBZvBEgsjY8K96wGF9dvBEUcoDq4o2gkKNT EVSvcwgWOUAVlhdvBAvypbd4I0DwyEtBlBENI96IEQ0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwR ILI8eFBZvBEgsjY8K96wRl5dvBEUcoDq4o2gkKNTEVQv3ggWOUAVlpc6BGsY8UaAIDF7izcCBI+8 AASriBKmYcQbMaJhxBsB6i/e3ZDhxBvBImuD19SyeCNAZHnwoLJ4I0BkbbD7bM1+UfT21MOGJMDu Myh2NaCBk4YgYYH5AH/wNc/MYSbevTukJ7AYIYHYkB7YIX6S8meA29g9bUgQNEqsEiFhS/cD7NIp HUSYzltOElz/cxp8cQdgav0gpZ7uvDGnh8rHheB4kj04ZPzUDztzZGdX7Cy31swBIXu0Kz8CBEfR LsyBoPxYj+1sz/mYB+FQVd4M/7fNqfDZHHuLi2fG46Oz+fjoPD/gBCbrTkQb40Vkzkq1OJFvhfe7 k2AjfNWlhv3y4NbjYY3CuXzf/OPblXvuye5N02TmsMFvbfeIt/gMe8hbZy+AR1y86w6aY1vgUpeH fr8VPK1XiTuIZj5cpDYU5uQf/G/NhTy+Z86suX/Kk+Qbg2NrWu6aH034Wru7h2OokxVTK6m13Db3 z2AbOXjynAEzxWVn3KUdRPPcp/vtimfmHFjL/F9KW1/gvNrTxHU7Yl24/coz3kNeY2f90bfik/rw PwAAAP//AwBQSwECLQAUAAYACAAAACEA3fyVN2YBAAAgBQAAEwAAAAAAAAAAAAAAAAAAAAAAW0Nv bnRlbnRfVHlwZXNdLnhtbFBLAQItABQABgAIAAAAIQAekRq38wAAAE4CAAALAAAAAAAAAAAAAAAA AJ8DAABfcmVscy8ucmVsc1BLAQItABQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAAAAAAAAAAAAAAA AMMGAAB3b3JkL19yZWxzL2RvY3VtZW50LnhtbC5yZWxzUEsBAi0AFAAGAAgAAAAhAKsR9BCDCgAA yx0AABEAAAAAAAAAAAAAAAAA/wgAAHdvcmQvZG9jdW1lbnQueG1sUEsBAi0AFAAGAAgAAAAhAJa1 reKWBgAAUBsAABUAAAAAAAAAAAAAAAAAsRMAAHdvcmQvdGhlbWUvdGhlbWUxLnhtbFBLAQItABQA BgAIAAAAIQChCYNYJQMAAB8HAAARAAAAAAAAAAAAAAAAAHoaAAB3b3JkL3NldHRpbmdzLnhtbFBL AQItABQABgAIAAAAIQBvY1CbiwEAAAcEAAASAAAAAAAAAAAAAAAAAM4dAAB3b3JkL2ZvbnRUYWJs ZS54bWxQSwECLQAUAAYACAAAACEAKIdxpc8AAAAfAQAAFAAAAAAAAAAAAAAAAACJHwAAd29yZC93 ZWJTZXR0aW5ncy54bWxQSwECLQAUAAYACAAAACEA94x95uABAADbAwAAEAAAAAAAAAAAAAAAAACK IAAAZG9jUHJvcHMvYXBwLnhtbFBLAQItABQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAAAAAAAAAAA AAAAAKAjAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQDmJsFqGgcAAPs5AAAPAAAA AAAAAAAAAAAAAC0mAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAAdC0AAAAA --===============1096080894==-- From david@fromorbit.com Mon Jan 4 13:40:48 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 42D327F37 for ; Mon, 4 Jan 2016 13:40:48 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id DEC44AC004 for ; Mon, 4 Jan 2016 11:40:47 -0800 (PST) X-ASG-Debug-ID: 1451936445-04bdf026514529d0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id QAvbNy9BbmyDhxrE for ; Mon, 04 Jan 2016 11:40:45 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DkCABpyopWPGu7LXleKAECgw+BP4ZggXmgSQEBAQEBAQaLVYk+hgkCAgEBAoEcTQEBAQEBAQcBAQEBQT+ENQEBBDocIxAIAw4KCSUPBSUDBxoTiC7BTgEBAQEGAQEBAQEeGYV1hUeIIYEbBYdbhxKIGY1IgWWERohZjjqFAio0g0UjgSgBAQE Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 06:10:15 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGAzG-0002Rc-UX; Tue, 05 Jan 2016 06:40:14 +1100 Date: Tue, 5 Jan 2016 06:40:14 +1100 From: Dave Chinner To: Julian Wollrath Cc: xfs@oss.sgi.com, Jiri Kosina Subject: Re: Suspend does not work with v4.4-rc8 Message-ID: <20160104194014.GD19802@dastard> X-ASG-Orig-Subj: Re: Suspend does not work with v4.4-rc8 References: <20160104202611.5b100540@saldaea> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104202611.5b100540@saldaea> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451936445 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 08:26:11PM +0100, Julian Wollrath wrote: > Hi, > > suspending (either to RAM or disk) fails with 4.4-rc8 with the > following message while v4.3.3 worked fine: > > kernel: Freezing of tasks failed after 20.006 seconds (2 tasks refusing to freeze, wq_bu_busy=0): > kernel: xfsaild/dm-1 S 0000000000014100 0 283 2 0x00000000 > kernel: ffff880213f53e10 ffffffff8180e4c0 ffff880213f05040 0000000000000000 > kernel: 0000000000000000 ffff880213f54000 0000000000000000 0000000000000000 > kernel: ffff8800ca389e40 ffff8800ca392000 ffff880213f53ed0 ffffffff814cd8ac > kernel: Call Trace: > kernel: [] ? schedule+0x2c/0x70 > kernel: [] ? xfsaild+0x4fd/0x5b0 > kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 > kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 > kernel: [] ? kthread+0xb8/0xd0 > kernel: [] ? kthread_worker_fn+0x150/0x150 > kernel: [] ? ret_from_fork+0x3f/0x70 > kernel: [] ? kthread_worker_fn+0x150/0x150 > kernel: xfsaild/sda1 S 0000000000014100 0 591 2 0x00000000 > kernel: ffff88021193be10 ffff8802159c4dc0 ffff880213ab9340 0000000000000000 > kernel: 0000000000000000 ffff88021193c000 0000000000000000 0000000000000000 > kernel: ffff8800ca2a4240 ffff880214eea000 ffff88021193bed0 ffffffff814cd8ac > kernel: Call Trace: > kernel: [] ? schedule+0x2c/0x70 > kernel: [] ? xfsaild+0x4fd/0x5b0 > kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 > kernel: [] ? xfs_trans_ail_cursor_first+0x80/0x80 > kernel: [] ? kthread+0xb8/0xd0 > kernel: [] ? kthread_worker_fn+0x150/0x150 > kernel: [] ? ret_from_fork+0x3f/0x70 > kernel: [] ? kthread_worker_fn+0x150/0x150 > > Please tell me what more information you need to be able to fix this > issue. The freezer detection is broken. The thread is sleeping in schedule until a wakeup occurs some time in the future, which means it cannot "enter then freezer" because it's not a running thread. This is a problem introduced by commit 24ba16b ("xfs: clear PF_NOFREEZE for xfsaild kthread"). Jiri, I'm tempted just to revert this change - if the freezer doesn't detect processes that are not in TASK_RUNNABLE state as frozeni or can't mark them as frozen, then this change will never work reliably for XFS.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From MAILER-DAEMON Mon Jan 4 13:43:04 2016 Return-Path: <> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.9 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLYTO, TO_MALFORMED,T_FREEMAIL_DOC_PDF autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 5AF2C7F37 for ; Mon, 4 Jan 2016 13:43:04 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 4921A8F8035 for ; Mon, 4 Jan 2016 11:43:04 -0800 (PST) X-ASG-Debug-ID: 1451936581-04cbb0101c1f8fd0001-NocioJ Received: from mail.gaf.ni.ac.rs (mail.gaf.ni.ac.rs [160.99.31.9]) by cuda.sgi.com with ESMTP id pN6oe33DT2FR706E (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:43:02 -0800 (PST) X-Barracuda-Envelope-From: X-Barracuda-Apparent-Source-IP: 160.99.31.9 Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 63AEC8F2F1C32; Mon, 4 Jan 2016 20:32:15 +0100 (CET) Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id wZkcjzMM0HJY; Mon, 4 Jan 2016 20:32:15 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id B33D48F377A96; Mon, 4 Jan 2016 20:32:12 +0100 (CET) X-Virus-Scanned: amavisd-new at gaf.ni.ac.rs Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id tSadQxvBtZRd; Mon, 4 Jan 2016 20:32:12 +0100 (CET) Received: from [100.67.150.48] (unknown [116.202.38.88]) by mail.gaf.ni.ac.rs (Postfix) with ESMTPSA id A38108F3C3D6E; Mon, 4 Jan 2016 20:31:52 +0100 (CET) Content-Type: multipart/mixed; boundary="===============2055748759==" MIME-Version: 1.0 Subject: Re To: Recipients <> X-ASG-Orig-Subj: Re From: "<"@LIVE.COM Date: Tue, 05 Jan 2016 01:01:51 +0530 Reply-To: cruzcelia2011@gmail.com Message-Id: <20160104193152.A38108F3C3D6E@mail.gaf.ni.ac.rs> X-Barracuda-Connect: mail.gaf.ni.ac.rs[160.99.31.9] X-Barracuda-Start-Time: 1451936581 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 2.00 X-Barracuda-Spam-Status: No, SCORE=2.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0646, BSF_SC0_MV0646_4, EMPTY_ENV_FROM, NO_REAL_NAME, TO_MALFORMED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25827 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 TO_MALFORMED To: has a malformed address 0.00 EMPTY_ENV_FROM Empty Envelope From Address 0.00 NO_REAL_NAME From: does not include a real name 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.50 BSF_SC0_MV0646 Custom rule MV0646 1.50 BSF_SC0_MV0646_4 BSF_SC0_MV0646_4 You will not see this in a MIME-aware mail reader. --===============2055748759== Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body please kindly open the fill --===============2055748759== Content-Type: application/octet-stream MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="BUSINESS PROPOSAL.docx" UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0 VMtuwjAQvFfqP0S+Vomhh6qqCBz6OLZIpR9g7A1Y9Uv28vr7bgJEVQtBKuUSKVnvzOzsxIPR2pps CTFp70rWL3osAye90m5Wso/JS37PsoTCKWG8g5JtILHR8PpqMNkESBl1u1SyOWJ44DzJOViRCh/A UaXy0Qqk1zjjQchPMQN+2+vdcekdgsMcaww2HDxBJRYGs+c1fd4qiWASyx63B2uukokQjJYCSSlf OvWDJd8xFNTZnElzHdINyWD8IENdOU6w63sja6JWkI1FxFdhSQZf+ai48nJhaYaiG+aATl9VWkLb X6OF6CWkRJ5bU7QVK7Tb6z+qI+HGQPp/FVvcLnrSOY4+JE57OZsf6s0rUDlZESCihnZ1x0cHRLLs EsPvkLvGb1KAlHfgzbN/tgcNzEnKin6JiZgaOJvvV/Ja6JMiVjB9v5j738C7hLT5kz7+wYz9dVF3 H0gdb+634RcAAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJlbHMg ogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfLT9ab g5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+ImYz sKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kVyAdh b9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ5x687 HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAgBd29yZC9f cmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AKySzWrDMBCE74W+g9h7LTv9oYTIuZRArq37AIq9/qGyJLSbtn77CkNShwb34otgRmjmk7Sb7Xdv xCcG6pxVkCUpCLSlqzrbKHgvdnfPIIi1rbRxFhUMSLDNb282r2g0x0PUdp5ETLGkoGX2aympbLHX lDiPNu7ULvSaowyN9Lr80A3KVZo+yTDNgPwiU+wrBWFf3YMoBh+b/892dd2V+OLKY4+Wr1TILzy8 IXO8HMVYHRpkBRMzibQgr4OslgShPxQnZw4hWxSBBxM/8/wMNOq5+scl6zmOCP62j1KOazbH8LAk Q+0sF/pgJhxn6wQhLwY9/wEAAP//AwBQSwMEFAAGAAgAAAAhAKsR9BCDCgAAyx0AABEAAAB3b3Jk L2RvY3VtZW50LnhtbNRYzW7bRhC+F+g7DIQcWkCWKAcuEjVW4FpO40NTI7ZzX5FLcWFyl91dSuGt r9FbniWP0ifpN0tSliXFFgSjRU+0l9z5n++b0Zu3n4ucFtI6ZfRpbzSIeiR1bBKl56e925t3R696 5LzQiciNlqe9Wrre28n3371ZjhMTV4XUniBCu/ECbzPvy/Fw6OJMFsINTCk1XqbGFsLjXzsfFsLe VeVRbIpSeDVTufL18DiKfuq1Ysxpr7J63Io4KlRsjTOp5ytjk6Yqlu2ju2H30dvcnLYmB41DK3PY YLTLVOk6acWh0uBi1glZPObEosi775blPtoSK5bIR5E3Zi+NTUprYukcTqfNy5XEUfSY7jaALGJ1 Yx8THursLCmE0isxXB0b+V8lb4DkDRvdQxZ17whiMUEtzUxS87Ok5Ri1mHw87UXR+Un07mLa646m MhVV7vnNL+fHr6Yvw83yyvJF2zxmwyDt3OE5bE/x7D564svl2E9+tVJ6xNUN6CqXwkkqpS2Up0KS N6S0tyapYklF7WSe8lltKjKaj4yVA7okUZDA94mKRU4xCgxmCzTKrCYkLuXEGU1oKkIw7thQ35gb 7Ov8ZUePR6PXoyg4+oiHuM/dM3aliJGF0kon7UL2Jo6EY5VxZWVo1UJoMZeWlspn9Jjily9P3p1M 91A8uS6FVi6jq0ygzWNZeXZ7U/bTkd/tAZ0zUuiaw7qEM0rTpU6U4D+Oo9GIAC4INnssbJxRWdnS IGlJZZFEWmYKh5eUKBcbwJxMyGfC0w+b9h0U9MkNlAj6JK34MVgnFkLlYpbLlZ2D51A0s205H1wf QQKX2c2mPYcmJni+KeygKO5OPAd1q0KfNnZyLZFj5ahyeHJxhD5EmvKuNlx/w2w0iEkvLLpi7OsS DTS3orj2wvpegJOnAIbj+vULUr6P3Aud7C11d2TI+IxbGDiKfvDsrNDQL2KvFpLwbsbecwmi3asU 500zmJTOtFf0/vIT3eAzUdZ9eg+PrZc6QNIV2FlpB3C6mJlc0FShrZx0dOHjAe5AlWvj60oZK7Cx yPMaca4AZlAIyyhTfCSsFASN16YC1FwI5+nMKdEPsCfiGNZzgwI+0TDGCm9sTV7is1UvszAMHZRa U7RdnwECZlJqBrUFHrg+Ay7jwxmuMkhkAjFYKKc88t9ABdzxGWCdTBwL9tPxPSBFDHF8u3OLEZll JZhe8CcrA3qDDSrGDUMW4SA4jFBjKEFUKosRJ3h0q4NCVA18YL/PCmkRnYBbCEgwWCWBQ5gvWA3L RnKgzgM2+BKf5oZZI5EiR5JF6jnVEkERADIxhxsByGENcwesmkvcsls482SrPASV/T7nSr9EcVUh 6zsimG60APfUis/3JrMJ2vYjCPhl1KcoioZHzMEEcruTnsM02tTypPXf4MdoXjTVtRn4Ps0qJKYK +VLekRUWk2pINX+bqXnWlkkfRYdJIN8Cq8Ncv69GpPi3uuO/AGUvthxf17E/ZffpJAqhfa440u01 TU2eC+s2RR5m4dcvaMBNUYdmme6kLENv8fQFxAy0EMa3tbECYGDnwIj0HhkYBFxVloAzPg6DH2qh AiTk7TyCw8Iw1CBbgrRcoipKMEdoUgDiVMaymKF6c0bAGkMK46hcpZUxRpslibI0mC0hqJECudtD W4amZywJ8HY/By2lvHN///kXedUMqduN+e/BAzunnEPnWPSID84BX8PUhTnM4MBjVsI0zH3EAdvl KqIJPMckxZNloqwETGKHKCoNTOWdrZlgWcQ6XqJtZSyY/B6qylTBCUoVEgo+CqqZFnkoAEqHrLMs JrcmP82gHLKI1Bf3fQhfPvx+Q2IJgusge1vUOqOEfQBgbRWXCU+fnefNRgSM5KTKz2FwlZhSa4AA uFGlIHOEDITywHJwQWt3y8nQxlEFj8I3pojk524wCEfe1i3XItaeYwmHuNrmDGor1rxDITJVsjjk pdsVuFtSniy4ennWAK/yivNHBcpjElsoLD+ta6CqWGi2JmY8CMweRpEgdlZhZcX687AceUHj3v6v l79uSN49dgGL0aQcelQSUzjCxKHBn+BprHT8MwlKCW/X+q+pqd0k07GJBrsfBQoBNuE6C2zXnm82 yDLD9IPqECirBQoEUBOGnaZ613uiqa4gvivxzoNNgF3H6v3JmoPQsvUzkfUEvNwwMFu8Cx/QMc9j fNgKmjmQiYEnTc4tfMJE/WLUP9lU8zQJTcCt0ea1w0Ib0e1gxawwqFlnUXhNq7/eUrOHdX0kiS4/ fAyVAgwL4NHCoJZz41UDsMjqDIwolGbsAAhiCydsB0k/4Nc3ssK/d+Q5Q0ybOvxWshZXBo9S1CHG mMWz/yMQnIWZ/B8AAAD//6xWXW7bOBC+ysBA31xXTtLYzTYG6iQt+rCA0e4FKGkkEZZIlaSiuk+9 Rt/2LD3KnmS/oawkNookBQokkvgzw5lvvm/oUDG1zmad44ZNoEYZVbKjSnlqO5fhzTnJLs/4KJxt aGe7qTyo13VNOeddFuKWe4PM+kDK5LLN0dtX/UVYydOt3uJB+Pc6/3Q5SZLzk/nN4noS5zdxOX0l g/TK4y0mw2xYBaduua61KY/9PW1MX5v6wrcq48tJ69izu+XJivhryybXAdlPY7hI1AypOFvoMGSY IvlKOaSfcugZOzpPKtBp8oJe0iJ5MaW+0llFDSvj75EJasuyLA7Nz3/p4+AuTsN2Fqccq7reUW7J WJynUINgqbaeyXaBrISjPaWd14Y9PjhTHRYRHKaVIdu21oXO6LCjwjoEIQ4aObuxhjEpNZMyZLZp ldnN6J1HLF7pnFi5WrObxpxzRvVzssVwJGgRK4tj5kmSENDbcvB/jUenEl9le+HMblyN2WcIa+TC cLwQqLaZErqoejzQdE0KruHAve9ofWe5p1dg1yD3GKJqbAeExGJfIB5ALTnMDmgB5rTCi/Yh2c7W i+Wbm8k4dc2F6uogNFxfnSyvTwcaPkmn1EWKhl+z6iOpBlibQueiqFCBKr0O1aCFCjXxKJUgjZJm 7KRwklDnWql6v88oVrC0FqLTRplMA7xSaUP4O6DEbCRW2eHECCFYU3QOoLsobigXqgFbPHmLNbxF vjFGF/XsCDzEJmEY2HOA5Kjax9RZKx8+QUoMlWzQQNZwt30MpN667YBKwzNag0kqtbdMEMMU+Wy1 yaEKx186oBWjRVhb5nbIHVRuQPlMBY187sAGRjPa1IymJYryiEj0EJsQNgWFRrUnneB/nObIi9/r SyrP0VE86eIxfyfz+Zt58ow+92tS/aFYgcR/3384tA8DWQFazo+DfpL9YTWjfyplttJ+cvoAhqY1 ADjU33NYM+potT4O4mElno3cSqjiuFQu99M/4fEuvncQ6TdFV7Vy6tjxc/A6sBk7k+csbA7uwsP2 9BnrwsTXN4vk9fvInLb8/A3Q9JcTsCk5l0ZW4ft8ebqcRLW15d9KXAbbYv7sLIm9TpcVPI3D1IZg m/txzcWD1Qp9gN3lZJEsxbawFkS5G5ZdiMNkOC6ztcem/cUqJjGK3GYfHG6X/gK3NW90yBDl6Xk0 QvZD4vFyT22+ix8w6eQHyOp/AAAA//8DAFBLAwQUAAYACAAAACEAlrWt4pYGAABQGwAAFQAAAHdv cmQvdGhlbWUvdGhlbWUxLnhtbOxZT2/bNhS/D9h3IHRvYyd2Ggd1itixmy1NG8Ruhx5piZbYUKJA 0kl9G9rjgAHDumGHFdhth2FbgRbYpfs02TpsHdCvsEdSksVYXpI22IqtPiQS+eP7/x4fqavX7scM HRIhKU/aXv1yzUMk8XlAk7Dt3R72L615SCqcBJjxhLS9KZHetY3337uK11VEYoJgfSLXcduLlErX l5akD8NYXuYpSWBuzEWMFbyKcCkQ+AjoxmxpuVZbXYoxTTyU4BjI3hqPqU/QUJP0NnLiPQaviZJ6 wGdioEkTZ4XBBgd1jZBT2WUCHWLW9oBPwI+G5L7yEMNSwUTbq5mft7RxdQmvZ4uYWrC2tK5vftm6 bEFwsGx4inBUMK33G60rWwV9A2BqHtfr9bq9ekHPALDvg6ZWljLNRn+t3slplkD2cZ52t9asNVx8 if7KnMytTqfTbGWyWKIGZB8bc/i12mpjc9nBG5DFN+fwjc5mt7vq4A3I4lfn8P0rrdWGizegiNHk YA6tHdrvZ9QLyJiz7Ur4GsDXahl8hoJoKKJLsxjzRC2KtRjf46IPAA1kWNEEqWlKxtiHKO7ieCQo 1gzwOsGlGTvky7khzQtJX9BUtb0PUwwZMaP36vn3r54/RccPnh0/+On44cPjBz9aQs6qbZyE5VUv v/3sz8cfoz+efvPy0RfVeFnG//rDJ7/8/Hk1ENJnJs6LL5/89uzJi68+/f27RxXwTYFHZfiQxkSi m+QI7fMYFDNWcSUnI3G+FcMI0/KKzSSUOMGaSwX9nooc9M0pZpl3HDk6xLXgHQHlowp4fXLPEXgQ iYmiFZx3otgB7nLOOlxUWmFH8yqZeThJwmrmYlLG7WN8WMW7ixPHv71JCnUzD0tH8W5EHDH3GE4U DklCFNJz/ICQCu3uUurYdZf6gks+VuguRR1MK00ypCMnmmaLtmkMfplW6Qz+dmyzewd1OKvSeosc ukjICswqhB8S5pjxOp4oHFeRHOKYlQ1+A6uoSsjBVPhlXE8q8HRIGEe9gEhZteaWAH1LTt/BULEq 3b7LprGLFIoeVNG8gTkvI7f4QTfCcVqFHdAkKmM/kAcQohjtcVUF3+Vuhuh38ANOFrr7DiWOu0+v Brdp6Ig0CxA9MxHal1CqnQoc0+TvyjGjUI9tDFxcOYYC+OLrxxWR9bYW4k3Yk6oyYftE+V2EO1l0 u1wE9O2vuVt4kuwRCPP5jeddyX1Xcr3/fMldlM9nLbSz2gplV/cNtik2LXK8sEMeU8YGasrIDWma ZAn7RNCHQb3OnA5JcWJKI3jM6rqDCwU2a5Dg6iOqokGEU2iw654mEsqMdChRyiUc7MxwJW2NhyZd 2WNhUx8YbD2QWO3ywA6v6OH8XFCQMbtNaA6fOaMVTeCszFauZERB7ddhVtdCnZlb3YhmSp3DrVAZ fDivGgwW1oQGBEHbAlZehfO5Zg0HE8xIoO1u997cLcYLF+kiGeGAZD7Ses/7qG6clMeKuQmA2Knw kT7knWK1EreWJvsG3M7ipDK7xgJ2uffexEt5BM+8pPP2RDqypJycLEFHba/VXG56yMdp2xvDmRYe 4xS8LnXPh1kIF0O+EjbsT01mk+Uzb7ZyxdwkqMM1hbX7nMJOHUiFVFtYRjY0zFQWAizRnKz8y00w 60UpYCP9NaRYWYNg+NekADu6riXjMfFV2dmlEW07+5qVUj5RRAyi4AiN2ETsY3C/DlXQJ6ASriZM RdAvcI+mrW2m3OKcJV359srg7DhmaYSzcqtTNM9kCzd5XMhg3krigW6Vshvlzq+KSfkLUqUcxv8z VfR+AjcFK4H2gA/XuAIjna9tjwsVcahCaUT9voDGwdQOiBa4i4VpCCq4TDb/BTnU/23OWRomreHA p/ZpiASF/UhFgpA9KEsm+k4hVs/2LkuSZYRMRJXElakVe0QOCRvqGriq93YPRRDqpppkZcDgTsaf +55l0CjUTU4535waUuy9Ngf+6c7HJjMo5dZh09Dk9i9ErNhV7XqzPN97y4roiVmb1cizApiVtoJW lvavKcI5t1pbseY0Xm7mwoEX5zWGwaIhSuG+B+k/sP9R4TP7ZUJvqEO+D7UVwYcGTQzCBqL6km08 kC6QdnAEjZMdtMGkSVnTZq2Ttlq+WV9wp1vwPWFsLdlZ/H1OYxfNmcvOycWLNHZmYcfWdmyhqcGz J1MUhsb5QcY4xnzSKn914qN74OgtuN+fMCVNMME3JYGh9RyYPIDktxzN0o2/AAAA//8DAFBLAwQU AAYACAAAACEAoQmDWCUDAAAfBwAAEQAAAHdvcmQvc2V0dGluZ3MueG1snFXbbts4EH1fYP9B0PM6 lmQ7ToQ4RWNb3RZJW6zSD6CkkUyEN5CUFffrO5TEKul6g2KfTJ0zczicm2/ePXMWHEEbKsUmjC+i MABRyoqKZhN+e8xmV2FgLBEVYVLAJjyBCd/d/vnHTZcasBbNTIASwqRyE7ZapKY8ACdmxmmppZG1 nZWSp7KuaQnjTzh66E14sFal8/nodCEVCFSrpebEmgupm/nguZNly0HYeRJFl3MNjFgM2ByoMl6N /181vOrgRY5vPeLImbfr4ugty/G5ndTVT4/fCc85KC1LMAYzy9nwXE6o8DKG/Y7OkM97WmiiTy9E brFs36XkQZcq0CUmFGseReHcEZX8LO2OGsXI6Stp4E62WHZNwfQ0xiXr3BIL6G0UMNb3SMmAYHRd 2mjCOcGaDsggCTVpmX0kRW6lQqMjwfjXyXhjeSCalBZ0rkiJalsprJbM2/UBbSVXGvMxxIi9pIh1 0bYGsv09OcnW4lXzLp0obObKOBt3+EdK6wWj6G6bXO0Wg5ZjJyaJ4+t4jOs1s1isstXunM/ybn11 vT/HrPbraJWdYy6TeL8+q/Y+W27jsz7/HfV2FWX7s2rZ1fI6XrsIMDdjRnjqev2rvr0ZThnmO+BD UbaEF5qS4MFNA3rxtNBPd1R4vgCcSnjJ5G3hydlsIAwnjGVYU0/gIAxMhX21g7oXZg9EN5Nyn3Se 6rNoBfWnn2quYUF/0LJVg2qnifooKoT9hfFyOepRYe8p97hpi9x7CRyKFxR2+ZejdoLzKUFdanGP gcvQPRGN7xMQs2+5M8V+Yzp3uw4eiFLYvGhSNPEmZLQ52NhNhMUvHKCn/qNokpFLeg6/HNd/kNK9 DK3HgzMYjmg1HiZs4bHFhC09tpywlcdWE3bpsUuHHU64BXCMn3Cn+KPDa8mY7KD624Ob8F/QkARz IAqwrm7KceRk2gNYtB4Ijik844qBilr8G1G04uR5E67iJHHuozWuGxziV7aOc8bqFRpUxBJcWH2p Xjn3Tf5LLLjQoKTYkPmJF9NSuRgCZ9TYHBTuHys1PrlfTH/1ytM/2+0PAAAA//8DAFBLAwQUAAYA CAAAACEAb2NQm4sBAAAHBAAAEgAAAHdvcmQvZm9udFRhYmxlLnhtbKSS326CMBTG75fsHUjvJwX/ TI1onNPLXSzuAY5YpAltSU+V+fY7UORiZolkkDThO+3H6e98i9W3KoKLsCiNTlg04CwQOjVHqU8J +9rvXqYsQAf6CIXRImFXgWy1fH5aVPPMaIcBndc4twnLnSvnYYhpLhTgwJRCUy0zVoGjT3sKTZbJ VLyb9KyEdmHM+SS0ogBH/8Zclshat+oRt8rYY2lNKhCpWVV4PwVSs2XbXVDNNSjqegOFPFjZFErQ BkVEtQsUCeMx3/ExrfU74sN6ZWHtkOZgUbhuI/dyBkoW15uKlUT0hVK6NL/pF7ASDoXwJZQnKpzx wBO2jTjn8W7HvBIlbETCetMpMTXln1m7Z9gpNB5qrPFptkSzxocU8mlPNX2Gfj53JPZSCQw+RBV8 GgUe1T2RmE+IxJh41GSGvYjYxrch+CgRajxed/enm2xIeZ2Oovb+vYh4n8eJbEBRNOCPbNQEPIma SL9s9CexrkcYb39lg/PR2x2JJgmUqP9kow0JLn8AAAD//wMAUEsDBBQABgAIAAAAIQAoh3GlzwAA AB8BAAAUAAAAd29yZC93ZWJTZXR0aW5ncy54bWyMj8tOAzEMRfdI/MMoe5qBRYVGnamEUNlQqMRj n2Y8nUiJHdmB0H495rFhx/LaV8fHq/VHis07sATC3lwuWtMAehoDHnrz8ry5uDaNFIeji4TQmyOI WQ/nZ6vaVdg/QSnalEYpKB33Zi4ld9aKnyE5WVAG1N1EnFzRyAdL0xQ83JJ/S4DFXrXt0jJEV9RA 5pDF/NLqf2iVeMxMHkRUJMUfXnIBzaCOlEtI4QQb4humKsD2a6z3jo/4ur3/Ti5GqruHOw32z1vD JwAAAP//AwBQSwMEFAAGAAgAAAAhAPeMfebgAQAA2wMAABAACAFkb2NQcm9wcy9hcHAueG1sIKIE ASigAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnFPBbtswDL0P2D8Yvjd2nCUoAkXFkGLo YVsDxG3PnEzHwmRJkNSg2dePshtX2XqqT+Qj/fT0SLGbl15lR3ReGr3J57Myz1AL00h92OQP9ber 6zzzAXQDymjc5Cf0+Q3//IntnLHogkSfEYX2m7wLwa6LwosOe/AzKmuqtMb1ECh1h8K0rRR4a8Rz jzoUVVmuCnwJqBtsruxEmI+M62P4KGljRNTnH+uTJcGc1dhbBQH5zyhHsWICWG0CqFr2yBcl4VPG dnBAz+esGAP2ZFzj+bIkZAzZtgMHIpB7vLpe0d8JwL5aq6SAQMbyH1I4400bsvvBgiwSsCJtYWTL HsWzk+HEiSpN2XepSUq1YMUYkTYHBwe283wVBU4Z2wtQuKXL8xaUR1a8AewOIQ52B5IUs2NYH1EE 4zIv/9Boqzz7BR6jZZv8CE6CDmRdbBuTIVbWB8drGRRxU23MhzBtS2P5JbpIvRRcNkZw1ECFS3XD Cf6+pbuFd8TOU7GDhlFqIicJpzP+Yd2a3oI+0eFTRAb/9g+2NrdxXV49vASTuT/J0O0tCJrOYrFc phuQlNieFgUbGumZ8A1gd+S3U/FU2h59wObc838h7tTj+Fb5vJqV9A1LdMZoE6ZHxP8CAAD//wMA UEsDBBQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAgBZG9jUHJvcHMvY29yZS54bWwgogQBKKAAAQAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACMkstuwjAQRfeV+g+R94mTAC2ykiD1wQraSk0f6s6y B4gaP2S7BP6+TgIpqF10Ob7XZ+6Mnc12og62YGylZI6SKEYBSKZ4Jdc5einn4RQF1lHJaa0k5GgP Fs2Ky4uMacKUgSejNBhXgQ08SVrCdI42zmmCsWUbENRG3iG9uFJGUOdLs8aask+6BpzG8RUW4Cin juIWGOqBiA5Izgak/jJ1B+AMQw0CpLM4iRL843VghP3zQqecOEXl9trPdIh7yuasFwf3zlaDsWma qBl1MXz+BL8vF8/dqGEl210xQEXGGWEGqFOmqEGqrcrwyVG7vppat/SbXlXAb/bF4v7h8fUxw7+V 1mxgW7VvVEw6x1D6Pt1YfTPggQ9K+rGOytvo9q6coyKNk0mYpGEyLuNrMhmROP5oQ53db4P3B+IQ 7R/EdFqmMUnG58QjoOgSn3+W4hsAAP//AwBQSwMEFAAGAAgAAAAhAOYmwWoaBwAA+zkAAA8AAAB3 b3JkL3N0eWxlcy54bWy0m1FT2zgQx99v5r6Dx+80ISnJlWnaoRSuzLQcbWDuWbEVoqlj5SylQD/9 SStbGBvbu9g8EcvW/lba1X+dIL3/eL9Ngl88U0Kmi/DwzTgMeBrJWKS3i/Dm+vzgrzBQmqUxS2TK F+EDV+HHD3/+8f7uWOmHhKvAGEjVcbYIN1rvjkcjFW34lqk3csdTc28tsy3T5jK7Hcn1WkT8s4z2 W57q0WQ8no0ynjBt4GojdirMrd1hrN3JLN5lMuJKGW+3ibO3ZSINPxj3Yhl95mu2T7Syl9lVll/m V/DnXKZaBXfHTEVCXBvHzRC3IpXZl5NUidDc4UzpEyVY+eZZ3mbvb+yD5Zu+Z6R0yeAnEYtwZKHq t+n2iyWLcDIpWk6tE0/aEpbeFm08Pbi4LDuzCH3TythdhCw7WJ5YYyMYafG3NOLdk/GbK3BlxyIz d4bD1pqbGJqQWE4ibKwn81lx8WOfmAa21zKHgAEDK5s1l5VJN6E1gV66RDF3+fqrjH7yeKnNjUUI LNN4c3GVCZkJ/bAI372zTNO45FvxRcQxt3mZt92kGxHzfzc8vVE8fmz/fg5ZlluM5D7Vxv3ZHBIh UfHZfcR3NsuM6ZTZIF/aDok1q0occGgvHr1xDRUqNP5XIA9dDJ+lbDizKykA/1tBMOp9b9DEjqg8 ALBL8nXa38Tb/iaO+puA5O03F/P+Xhj97BsRlxulrMQHVcvIJV95HqbvWlLW9qhlUWePWtJ09qjl SGePWkp09qhlQGePWsA7e9Ti29mjFs7WHhED4apm0RRmA7Wwr4VOuO3fKkCHPaUuLzXBFcvYbcZ2 m8DW1qrbbWK53K80zlWQ05eL5VJnMr3tnBFTne3SfbEmn213G6aEeanpmPpJz6m/ZquEB39nIu5E Hbnkq40JXkyeLWFXCYv4RiYxz4Jrfu8iSuh/KYOle8vodK5nWL+K240OlhsouZ2wWcOkN8+Es/9V KJiD1sU0axhKl3FUDGcNedls/BuPxX5bTA3ibWTm9JwQ5goCXGyforc2RPXV1TkKGwDMEFy5oA8B 7CP8d8WFbt/GGOO/K0UvtI/w3xWuF9qH/GiPL1lpPrPsZ4BaXnPy2j2ViczW+6RYA53yMCevYI/A DYG8iL19lEjMySv4iXwGJ1Fkvrlh8pQci0cdJVDI4XAUWGz4sZCDUpG9Q8KIyAGqsCYEVj+tJYDI ovuD/xL2tydqMQCV9u+anct52jADpgSh3qG/76XufoeeNGgelnKRmp9LFA9wtGnDysPS8nxy9Y4Q 436FjwDqVwEJoH6lkABqyI/mdx5fE/GQ/sWRwCLLsq9ikHZoZZ6TldmDaCVgoLqJeP9qWL3NuVCv mwgKOUD1uomgkKNTqWW+biJYg9VNBKuhajTHqKyplEGR62YZ5N8EECMaRrwRoGHEGwEaRrwRoP7i 3Q0ZTrwRLLI2eE0tizcCBI9Qvup7UFm8ESCyNji1y38zKuoeWGn/cjuAeCMo5ADVxRtBIUenSbwR LHiEkgkVlpc6BGsY8UaAhhFvBGgY8UaAhhFvBGgY8UaA+ot3N2Q48UawyNrgNbUs3ggQWR48qCze CBA8QtGGZ8UbVv2rizeCQg5QXbwRFHJ0KoLqX1IRLHKAKiwv3ggWPEJJhpwFyU0Z1DDijRjRMOKN AA0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwRILI8eFBZvBEgsjY8K96wGF9dvBEUcoDq4o2gkKNT EVSvcwgWOUAVlhdvBAvypbd4I0DwyEtBlBENI96IEQ0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwR ILI8eFBZvBEgsjY8K96wRl5dvBEUcoDq4o2gkKNTEVQv3ggWOUAVlpc6BGsY8UaAIDF7izcCBI+8 AASriBKmYcQbMaJhxBsB6i/e3ZDhxBvBImuD19SyeCNAZHnwoLJ4I0BkbbD7bM1+UfT21MOGJMDu Myh2NaCBk4YgYYH5AH/wNc/MYSbevTukJ7AYIYHYkB7YIX6S8meA29g9bUgQNEqsEiFhS/cD7NIp HUSYzltOElz/cxp8cQdgav0gpZ7uvDGnh8rHheB4kj04ZPzUDztzZGdX7Cy31swBIXu0Kz8CBEfR LsyBoPxYj+1sz/mYB+FQVd4M/7fNqfDZHHuLi2fG46Oz+fjoPD/gBCbrTkQb40Vkzkq1OJFvhfe7 k2AjfNWlhv3y4NbjYY3CuXzf/OPblXvuye5N02TmsMFvbfeIt/gMe8hbZy+AR1y86w6aY1vgUpeH fr8VPK1XiTuIZj5cpDYU5uQf/G/NhTy+Z86suX/Kk+Qbg2NrWu6aH034Wru7h2OokxVTK6m13Db3 z2AbOXjynAEzxWVn3KUdRPPcp/vtimfmHFjL/F9KW1/gvNrTxHU7Yl24/coz3kNeY2f90bfik/rw PwAAAP//AwBQSwECLQAUAAYACAAAACEA3fyVN2YBAAAgBQAAEwAAAAAAAAAAAAAAAAAAAAAAW0Nv bnRlbnRfVHlwZXNdLnhtbFBLAQItABQABgAIAAAAIQAekRq38wAAAE4CAAALAAAAAAAAAAAAAAAA AJ8DAABfcmVscy8ucmVsc1BLAQItABQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAAAAAAAAAAAAAAA AMMGAAB3b3JkL19yZWxzL2RvY3VtZW50LnhtbC5yZWxzUEsBAi0AFAAGAAgAAAAhAKsR9BCDCgAA yx0AABEAAAAAAAAAAAAAAAAA/wgAAHdvcmQvZG9jdW1lbnQueG1sUEsBAi0AFAAGAAgAAAAhAJa1 reKWBgAAUBsAABUAAAAAAAAAAAAAAAAAsRMAAHdvcmQvdGhlbWUvdGhlbWUxLnhtbFBLAQItABQA BgAIAAAAIQChCYNYJQMAAB8HAAARAAAAAAAAAAAAAAAAAHoaAAB3b3JkL3NldHRpbmdzLnhtbFBL AQItABQABgAIAAAAIQBvY1CbiwEAAAcEAAASAAAAAAAAAAAAAAAAAM4dAAB3b3JkL2ZvbnRUYWJs ZS54bWxQSwECLQAUAAYACAAAACEAKIdxpc8AAAAfAQAAFAAAAAAAAAAAAAAAAACJHwAAd29yZC93 ZWJTZXR0aW5ncy54bWxQSwECLQAUAAYACAAAACEA94x95uABAADbAwAAEAAAAAAAAAAAAAAAAACK IAAAZG9jUHJvcHMvYXBwLnhtbFBLAQItABQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAAAAAAAAAAA AAAAAKAjAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQDmJsFqGgcAAPs5AAAPAAAA AAAAAAAAAAAAAC0mAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAAdC0AAAAA --===============2055748759==-- From darrick.wong@oracle.com Mon Jan 4 13:47:23 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 738967F37 for ; Mon, 4 Jan 2016 13:47:23 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 4370A8F8033 for ; Mon, 4 Jan 2016 11:47:23 -0800 (PST) X-ASG-Debug-ID: 1451936841-04bdf02650452c40001-NocioJ Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) by cuda.sgi.com with ESMTP id tAFFWHZItvdqmJ34 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:47:21 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 156.151.31.81 Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04JlG2c010922 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 19:47:17 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u04JlGm1031101 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 19:47:16 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04JlFh7015076; Mon, 4 Jan 2016 19:47:15 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 11:47:14 -0800 Date: Mon, 4 Jan 2016 11:47:08 -0800 From: "Darrick J. Wong" To: Dave Chinner Cc: linux-ext4@vger.kernel.org, linux-btrfs@vger.kernel.org, fstests@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [PATCH 06/11] reflink: more tests Message-ID: <20160104194708.GA28357@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 06/11] reflink: more tests References: <20151219091052.15320.94973.stgit@birch.djwong.org> <20151219091131.15320.12391.stgit@birch.djwong.org> <20151221063111.GO19802@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151221063111.GO19802@dastard> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Barracuda-Connect: userp1040.oracle.com[156.151.31.81] X-Barracuda-Start-Time: 1451936841 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Mon, Dec 21, 2015 at 05:31:11PM +1100, Dave Chinner wrote: > On Sat, Dec 19, 2015 at 01:11:31AM -0800, Darrick J. Wong wrote: > > Add more tests for unaligned copy-on-write things, and explicitly > > test the ability to pass "len == 0" to mean reflink/dedupe all > > the way to the end of the file". > > > > Signed-off-by: Darrick J. Wong > ..... > > +_cleanup() > > +{ > > + cd / > > + rm -rf "$tmp".* "$TESTDIR" > > +} > ..... > > + > > +TESTDIR="$TEST_DIR/test-$seq" > > +rm -rf "$TESTDIR" > > +mkdir "$TESTDIR" > > This use of TESTDIR is highly confusing. When I see TESTDIR I think > that it's the mount point of the test device, not potentially some > directory on the scratch device. > > I see that this occurs through lots of tests, not just these new > ones. Can you do a pass across the tests with, say, sed and rename > all these to something less confusing? e.g. "testdir", in lower > case, makes it clear that it's a local variable, not the global test > device mount point.... (i.e. test local variables are lower case, > exported test harnes variables are upper case ;) > > Separate patch/pull req is fine. No problem. Will give it a spin on the test harness before sending. (Yikes, it's a 272K patch...) --D > > Cheers, > > dave. > -- > Dave Chinner > david@fromorbit.com From darrick.wong@oracle.com Mon Jan 4 13:58:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 633057F37 for ; Mon, 4 Jan 2016 13:58:18 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id F3C57AC006 for ; Mon, 4 Jan 2016 11:58:17 -0800 (PST) X-ASG-Debug-ID: 1451937495-04cb6c303042bdf0001-NocioJ Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) by cuda.sgi.com with ESMTP id iJ7lxcKU0tFkAtRt (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 11:58:16 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 156.151.31.81 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04Jvq5H023823 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 19:57:52 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u04JvpVl005911 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 19:57:52 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04JvpqS020052; Mon, 4 Jan 2016 19:57:51 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 11:57:51 -0800 Date: Mon, 4 Jan 2016 11:57:48 -0800 From: "Darrick J. Wong" To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [PATCH 02/76] xfs: fix log ticket type printing Message-ID: <20160104195748.GC28357@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 02/76] xfs: fix log ticket type printing References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085636.12713.59155.stgit@birch.djwong.org> <20160103121353.GA28429@infradead.org> <20160103212905.GY19802@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103212905.GY19802@dastard> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0022.oracle.com [156.151.31.74] X-Barracuda-Connect: userp1040.oracle.com[156.151.31.81] X-Barracuda-Start-Time: 1451937496 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Mon, Jan 04, 2016 at 08:29:05AM +1100, Dave Chinner wrote: > On Sun, Jan 03, 2016 at 04:13:53AM -0800, Christoph Hellwig wrote: > > On Sat, Dec 19, 2015 at 12:56:36AM -0800, Darrick J. Wong wrote: > > > Update the log ticket reservation type printing code to reflect > > > all the types of log tickets, to avoid incorrect debug output and > > > avoid running off the end of the array. > > > > This looks okay, but I think we should use name array initializers > > to make sure it doesn't got out of sync ever. > > Good idea, separate patch is fine. Will do. --D > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com From MAILER-DAEMON Mon Jan 4 14:07:07 2016 Return-Path: <> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.9 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLYTO, TO_MALFORMED,T_FREEMAIL_DOC_PDF autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 3FE0C7F37 for ; Mon, 4 Jan 2016 14:07:07 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 218EE304039 for ; Mon, 4 Jan 2016 12:07:04 -0800 (PST) X-ASG-Debug-ID: 1451938019-04cb6c303242c120001-NocioJ Received: from mail.gaf.ni.ac.rs (mail.gaf.ni.ac.rs [160.99.31.9]) by cuda.sgi.com with ESMTP id gC71a6wp0QOSoQcH (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 12:07:00 -0800 (PST) X-Barracuda-Envelope-From: X-Barracuda-Apparent-Source-IP: 160.99.31.9 Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id A56A2811C449B; Mon, 4 Jan 2016 21:05:56 +0100 (CET) Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id FII5rwOfGjGL; Mon, 4 Jan 2016 21:05:56 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 2FAC180C673F4; Mon, 4 Jan 2016 21:05:56 +0100 (CET) X-Virus-Scanned: amavisd-new at gaf.ni.ac.rs Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id rjmkrAXBPKu8; Mon, 4 Jan 2016 21:05:56 +0100 (CET) Received: from [100.67.150.48] (unknown [116.202.38.88]) by mail.gaf.ni.ac.rs (Postfix) with ESMTPSA id 29C4182457A74; Mon, 4 Jan 2016 21:05:34 +0100 (CET) Content-Type: multipart/mixed; boundary="===============0921340515==" MIME-Version: 1.0 Subject: Re To: Recipients <> X-ASG-Orig-Subj: Re From: "<"@LIVE.COM Date: Tue, 05 Jan 2016 01:35:14 +0530 Reply-To: cruzcelia2011@gmail.com Message-Id: <20160104200535.29C4182457A74@mail.gaf.ni.ac.rs> X-Barracuda-Connect: mail.gaf.ni.ac.rs[160.99.31.9] X-Barracuda-Start-Time: 1451938020 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 2.00 X-Barracuda-Spam-Status: No, SCORE=2.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0646, BSF_SC0_MV0646_4, EMPTY_ENV_FROM, NO_REAL_NAME, TO_MALFORMED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25828 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 TO_MALFORMED To: has a malformed address 0.00 EMPTY_ENV_FROM Empty Envelope From Address 0.00 NO_REAL_NAME From: does not include a real name 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.50 BSF_SC0_MV0646 Custom rule MV0646 1.50 BSF_SC0_MV0646_4 BSF_SC0_MV0646_4 You will not see this in a MIME-aware mail reader. --===============0921340515== Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body please kindly open the fill --===============0921340515== Content-Type: application/octet-stream MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="BUSINESS PROPOSAL.docx" UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0 VMtuwjAQvFfqP0S+Vomhh6qqCBz6OLZIpR9g7A1Y9Uv28vr7bgJEVQtBKuUSKVnvzOzsxIPR2pps CTFp70rWL3osAye90m5Wso/JS37PsoTCKWG8g5JtILHR8PpqMNkESBl1u1SyOWJ44DzJOViRCh/A UaXy0Qqk1zjjQchPMQN+2+vdcekdgsMcaww2HDxBJRYGs+c1fd4qiWASyx63B2uukokQjJYCSSlf OvWDJd8xFNTZnElzHdINyWD8IENdOU6w63sja6JWkI1FxFdhSQZf+ai48nJhaYaiG+aATl9VWkLb X6OF6CWkRJ5bU7QVK7Tb6z+qI+HGQPp/FVvcLnrSOY4+JE57OZsf6s0rUDlZESCihnZ1x0cHRLLs EsPvkLvGb1KAlHfgzbN/tgcNzEnKin6JiZgaOJvvV/Ja6JMiVjB9v5j738C7hLT5kz7+wYz9dVF3 H0gdb+634RcAAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJlbHMg ogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfLT9ab g5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+ImYz sKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kVyAdh b9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ5x687 HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAgBd29yZC9f cmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AKySzWrDMBCE74W+g9h7LTv9oYTIuZRArq37AIq9/qGyJLSbtn77CkNShwb34otgRmjmk7Sb7Xdv xCcG6pxVkCUpCLSlqzrbKHgvdnfPIIi1rbRxFhUMSLDNb282r2g0x0PUdp5ETLGkoGX2aympbLHX lDiPNu7ULvSaowyN9Lr80A3KVZo+yTDNgPwiU+wrBWFf3YMoBh+b/892dd2V+OLKY4+Wr1TILzy8 IXO8HMVYHRpkBRMzibQgr4OslgShPxQnZw4hWxSBBxM/8/wMNOq5+scl6zmOCP62j1KOazbH8LAk Q+0sF/pgJhxn6wQhLwY9/wEAAP//AwBQSwMEFAAGAAgAAAAhAKsR9BCDCgAAyx0AABEAAAB3b3Jk L2RvY3VtZW50LnhtbNRYzW7bRhC+F+g7DIQcWkCWKAcuEjVW4FpO40NTI7ZzX5FLcWFyl91dSuGt r9FbniWP0ifpN0tSliXFFgSjRU+0l9z5n++b0Zu3n4ucFtI6ZfRpbzSIeiR1bBKl56e925t3R696 5LzQiciNlqe9Wrre28n3371ZjhMTV4XUniBCu/ECbzPvy/Fw6OJMFsINTCk1XqbGFsLjXzsfFsLe VeVRbIpSeDVTufL18DiKfuq1Ysxpr7J63Io4KlRsjTOp5ytjk6Yqlu2ju2H30dvcnLYmB41DK3PY YLTLVOk6acWh0uBi1glZPObEosi775blPtoSK5bIR5E3Zi+NTUprYukcTqfNy5XEUfSY7jaALGJ1 Yx8THursLCmE0isxXB0b+V8lb4DkDRvdQxZ17whiMUEtzUxS87Ok5Ri1mHw87UXR+Un07mLa646m MhVV7vnNL+fHr6Yvw83yyvJF2zxmwyDt3OE5bE/x7D564svl2E9+tVJ6xNUN6CqXwkkqpS2Up0KS N6S0tyapYklF7WSe8lltKjKaj4yVA7okUZDA94mKRU4xCgxmCzTKrCYkLuXEGU1oKkIw7thQ35gb 7Ov8ZUePR6PXoyg4+oiHuM/dM3aliJGF0kon7UL2Jo6EY5VxZWVo1UJoMZeWlspn9Jjily9P3p1M 91A8uS6FVi6jq0ygzWNZeXZ7U/bTkd/tAZ0zUuiaw7qEM0rTpU6U4D+Oo9GIAC4INnssbJxRWdnS IGlJZZFEWmYKh5eUKBcbwJxMyGfC0w+b9h0U9MkNlAj6JK34MVgnFkLlYpbLlZ2D51A0s205H1wf QQKX2c2mPYcmJni+KeygKO5OPAd1q0KfNnZyLZFj5ahyeHJxhD5EmvKuNlx/w2w0iEkvLLpi7OsS DTS3orj2wvpegJOnAIbj+vULUr6P3Aud7C11d2TI+IxbGDiKfvDsrNDQL2KvFpLwbsbecwmi3asU 500zmJTOtFf0/vIT3eAzUdZ9eg+PrZc6QNIV2FlpB3C6mJlc0FShrZx0dOHjAe5AlWvj60oZK7Cx yPMaca4AZlAIyyhTfCSsFASN16YC1FwI5+nMKdEPsCfiGNZzgwI+0TDGCm9sTV7is1UvszAMHZRa U7RdnwECZlJqBrUFHrg+Ay7jwxmuMkhkAjFYKKc88t9ABdzxGWCdTBwL9tPxPSBFDHF8u3OLEZll JZhe8CcrA3qDDSrGDUMW4SA4jFBjKEFUKosRJ3h0q4NCVA18YL/PCmkRnYBbCEgwWCWBQ5gvWA3L RnKgzgM2+BKf5oZZI5EiR5JF6jnVEkERADIxhxsByGENcwesmkvcsls482SrPASV/T7nSr9EcVUh 6zsimG60APfUis/3JrMJ2vYjCPhl1KcoioZHzMEEcruTnsM02tTypPXf4MdoXjTVtRn4Ps0qJKYK +VLekRUWk2pINX+bqXnWlkkfRYdJIN8Cq8Ncv69GpPi3uuO/AGUvthxf17E/ZffpJAqhfa440u01 TU2eC+s2RR5m4dcvaMBNUYdmme6kLENv8fQFxAy0EMa3tbECYGDnwIj0HhkYBFxVloAzPg6DH2qh AiTk7TyCw8Iw1CBbgrRcoipKMEdoUgDiVMaymKF6c0bAGkMK46hcpZUxRpslibI0mC0hqJECudtD W4amZywJ8HY/By2lvHN///kXedUMqduN+e/BAzunnEPnWPSID84BX8PUhTnM4MBjVsI0zH3EAdvl KqIJPMckxZNloqwETGKHKCoNTOWdrZlgWcQ6XqJtZSyY/B6qylTBCUoVEgo+CqqZFnkoAEqHrLMs JrcmP82gHLKI1Bf3fQhfPvx+Q2IJgusge1vUOqOEfQBgbRWXCU+fnefNRgSM5KTKz2FwlZhSa4AA uFGlIHOEDITywHJwQWt3y8nQxlEFj8I3pojk524wCEfe1i3XItaeYwmHuNrmDGor1rxDITJVsjjk pdsVuFtSniy4ennWAK/yivNHBcpjElsoLD+ta6CqWGi2JmY8CMweRpEgdlZhZcX687AceUHj3v6v l79uSN49dgGL0aQcelQSUzjCxKHBn+BprHT8MwlKCW/X+q+pqd0k07GJBrsfBQoBNuE6C2zXnm82 yDLD9IPqECirBQoEUBOGnaZ613uiqa4gvivxzoNNgF3H6v3JmoPQsvUzkfUEvNwwMFu8Cx/QMc9j fNgKmjmQiYEnTc4tfMJE/WLUP9lU8zQJTcCt0ea1w0Ib0e1gxawwqFlnUXhNq7/eUrOHdX0kiS4/ fAyVAgwL4NHCoJZz41UDsMjqDIwolGbsAAhiCydsB0k/4Nc3ssK/d+Q5Q0ybOvxWshZXBo9S1CHG mMWz/yMQnIWZ/B8AAAD//6xWXW7bOBC+ysBA31xXTtLYzTYG6iQt+rCA0e4FKGkkEZZIlaSiuk+9 Rt/2LD3KnmS/oawkNookBQokkvgzw5lvvm/oUDG1zmad44ZNoEYZVbKjSnlqO5fhzTnJLs/4KJxt aGe7qTyo13VNOeddFuKWe4PM+kDK5LLN0dtX/UVYydOt3uJB+Pc6/3Q5SZLzk/nN4noS5zdxOX0l g/TK4y0mw2xYBaduua61KY/9PW1MX5v6wrcq48tJ69izu+XJivhryybXAdlPY7hI1AypOFvoMGSY IvlKOaSfcugZOzpPKtBp8oJe0iJ5MaW+0llFDSvj75EJasuyLA7Nz3/p4+AuTsN2Fqccq7reUW7J WJynUINgqbaeyXaBrISjPaWd14Y9PjhTHRYRHKaVIdu21oXO6LCjwjoEIQ4aObuxhjEpNZMyZLZp ldnN6J1HLF7pnFi5WrObxpxzRvVzssVwJGgRK4tj5kmSENDbcvB/jUenEl9le+HMblyN2WcIa+TC cLwQqLaZErqoejzQdE0KruHAve9ofWe5p1dg1yD3GKJqbAeExGJfIB5ALTnMDmgB5rTCi/Yh2c7W i+Wbm8k4dc2F6uogNFxfnSyvTwcaPkmn1EWKhl+z6iOpBlibQueiqFCBKr0O1aCFCjXxKJUgjZJm 7KRwklDnWql6v88oVrC0FqLTRplMA7xSaUP4O6DEbCRW2eHECCFYU3QOoLsobigXqgFbPHmLNbxF vjFGF/XsCDzEJmEY2HOA5Kjax9RZKx8+QUoMlWzQQNZwt30MpN667YBKwzNag0kqtbdMEMMU+Wy1 yaEKx186oBWjRVhb5nbIHVRuQPlMBY187sAGRjPa1IymJYryiEj0EJsQNgWFRrUnneB/nObIi9/r SyrP0VE86eIxfyfz+Zt58ow+92tS/aFYgcR/3384tA8DWQFazo+DfpL9YTWjfyplttJ+cvoAhqY1 ADjU33NYM+potT4O4mElno3cSqjiuFQu99M/4fEuvncQ6TdFV7Vy6tjxc/A6sBk7k+csbA7uwsP2 9BnrwsTXN4vk9fvInLb8/A3Q9JcTsCk5l0ZW4ft8ebqcRLW15d9KXAbbYv7sLIm9TpcVPI3D1IZg m/txzcWD1Qp9gN3lZJEsxbawFkS5G5ZdiMNkOC6ztcem/cUqJjGK3GYfHG6X/gK3NW90yBDl6Xk0 QvZD4vFyT22+ix8w6eQHyOp/AAAA//8DAFBLAwQUAAYACAAAACEAlrWt4pYGAABQGwAAFQAAAHdv cmQvdGhlbWUvdGhlbWUxLnhtbOxZT2/bNhS/D9h3IHRvYyd2Ggd1itixmy1NG8Ruhx5piZbYUKJA 0kl9G9rjgAHDumGHFdhth2FbgRbYpfs02TpsHdCvsEdSksVYXpI22IqtPiQS+eP7/x4fqavX7scM HRIhKU/aXv1yzUMk8XlAk7Dt3R72L615SCqcBJjxhLS9KZHetY3337uK11VEYoJgfSLXcduLlErX l5akD8NYXuYpSWBuzEWMFbyKcCkQ+AjoxmxpuVZbXYoxTTyU4BjI3hqPqU/QUJP0NnLiPQaviZJ6 wGdioEkTZ4XBBgd1jZBT2WUCHWLW9oBPwI+G5L7yEMNSwUTbq5mft7RxdQmvZ4uYWrC2tK5vftm6 bEFwsGx4inBUMK33G60rWwV9A2BqHtfr9bq9ekHPALDvg6ZWljLNRn+t3slplkD2cZ52t9asNVx8 if7KnMytTqfTbGWyWKIGZB8bc/i12mpjc9nBG5DFN+fwjc5mt7vq4A3I4lfn8P0rrdWGizegiNHk YA6tHdrvZ9QLyJiz7Ur4GsDXahl8hoJoKKJLsxjzRC2KtRjf46IPAA1kWNEEqWlKxtiHKO7ieCQo 1gzwOsGlGTvky7khzQtJX9BUtb0PUwwZMaP36vn3r54/RccPnh0/+On44cPjBz9aQs6qbZyE5VUv v/3sz8cfoz+efvPy0RfVeFnG//rDJ7/8/Hk1ENJnJs6LL5/89uzJi68+/f27RxXwTYFHZfiQxkSi m+QI7fMYFDNWcSUnI3G+FcMI0/KKzSSUOMGaSwX9nooc9M0pZpl3HDk6xLXgHQHlowp4fXLPEXgQ iYmiFZx3otgB7nLOOlxUWmFH8yqZeThJwmrmYlLG7WN8WMW7ixPHv71JCnUzD0tH8W5EHDH3GE4U DklCFNJz/ICQCu3uUurYdZf6gks+VuguRR1MK00ypCMnmmaLtmkMfplW6Qz+dmyzewd1OKvSeosc ukjICswqhB8S5pjxOp4oHFeRHOKYlQ1+A6uoSsjBVPhlXE8q8HRIGEe9gEhZteaWAH1LTt/BULEq 3b7LprGLFIoeVNG8gTkvI7f4QTfCcVqFHdAkKmM/kAcQohjtcVUF3+Vuhuh38ANOFrr7DiWOu0+v Brdp6Ig0CxA9MxHal1CqnQoc0+TvyjGjUI9tDFxcOYYC+OLrxxWR9bYW4k3Yk6oyYftE+V2EO1l0 u1wE9O2vuVt4kuwRCPP5jeddyX1Xcr3/fMldlM9nLbSz2gplV/cNtik2LXK8sEMeU8YGasrIDWma ZAn7RNCHQb3OnA5JcWJKI3jM6rqDCwU2a5Dg6iOqokGEU2iw654mEsqMdChRyiUc7MxwJW2NhyZd 2WNhUx8YbD2QWO3ywA6v6OH8XFCQMbtNaA6fOaMVTeCszFauZERB7ddhVtdCnZlb3YhmSp3DrVAZ fDivGgwW1oQGBEHbAlZehfO5Zg0HE8xIoO1u997cLcYLF+kiGeGAZD7Ses/7qG6clMeKuQmA2Knw kT7knWK1EreWJvsG3M7ipDK7xgJ2uffexEt5BM+8pPP2RDqypJycLEFHba/VXG56yMdp2xvDmRYe 4xS8LnXPh1kIF0O+EjbsT01mk+Uzb7ZyxdwkqMM1hbX7nMJOHUiFVFtYRjY0zFQWAizRnKz8y00w 60UpYCP9NaRYWYNg+NekADu6riXjMfFV2dmlEW07+5qVUj5RRAyi4AiN2ETsY3C/DlXQJ6ASriZM RdAvcI+mrW2m3OKcJV359srg7DhmaYSzcqtTNM9kCzd5XMhg3krigW6Vshvlzq+KSfkLUqUcxv8z VfR+AjcFK4H2gA/XuAIjna9tjwsVcahCaUT9voDGwdQOiBa4i4VpCCq4TDb/BTnU/23OWRomreHA p/ZpiASF/UhFgpA9KEsm+k4hVs/2LkuSZYRMRJXElakVe0QOCRvqGriq93YPRRDqpppkZcDgTsaf +55l0CjUTU4535waUuy9Ngf+6c7HJjMo5dZh09Dk9i9ErNhV7XqzPN97y4roiVmb1cizApiVtoJW lvavKcI5t1pbseY0Xm7mwoEX5zWGwaIhSuG+B+k/sP9R4TP7ZUJvqEO+D7UVwYcGTQzCBqL6km08 kC6QdnAEjZMdtMGkSVnTZq2Ttlq+WV9wp1vwPWFsLdlZ/H1OYxfNmcvOycWLNHZmYcfWdmyhqcGz J1MUhsb5QcY4xnzSKn914qN74OgtuN+fMCVNMME3JYGh9RyYPIDktxzN0o2/AAAA//8DAFBLAwQU AAYACAAAACEAoQmDWCUDAAAfBwAAEQAAAHdvcmQvc2V0dGluZ3MueG1snFXbbts4EH1fYP9B0PM6 lmQ7ToQ4RWNb3RZJW6zSD6CkkUyEN5CUFffrO5TEKul6g2KfTJ0zczicm2/ePXMWHEEbKsUmjC+i MABRyoqKZhN+e8xmV2FgLBEVYVLAJjyBCd/d/vnHTZcasBbNTIASwqRyE7ZapKY8ACdmxmmppZG1 nZWSp7KuaQnjTzh66E14sFal8/nodCEVCFSrpebEmgupm/nguZNly0HYeRJFl3MNjFgM2ByoMl6N /181vOrgRY5vPeLImbfr4ugty/G5ndTVT4/fCc85KC1LMAYzy9nwXE6o8DKG/Y7OkM97WmiiTy9E brFs36XkQZcq0CUmFGseReHcEZX8LO2OGsXI6Stp4E62WHZNwfQ0xiXr3BIL6G0UMNb3SMmAYHRd 2mjCOcGaDsggCTVpmX0kRW6lQqMjwfjXyXhjeSCalBZ0rkiJalsprJbM2/UBbSVXGvMxxIi9pIh1 0bYGsv09OcnW4lXzLp0obObKOBt3+EdK6wWj6G6bXO0Wg5ZjJyaJ4+t4jOs1s1isstXunM/ybn11 vT/HrPbraJWdYy6TeL8+q/Y+W27jsz7/HfV2FWX7s2rZ1fI6XrsIMDdjRnjqev2rvr0ZThnmO+BD UbaEF5qS4MFNA3rxtNBPd1R4vgCcSnjJ5G3hydlsIAwnjGVYU0/gIAxMhX21g7oXZg9EN5Nyn3Se 6rNoBfWnn2quYUF/0LJVg2qnifooKoT9hfFyOepRYe8p97hpi9x7CRyKFxR2+ZejdoLzKUFdanGP gcvQPRGN7xMQs2+5M8V+Yzp3uw4eiFLYvGhSNPEmZLQ52NhNhMUvHKCn/qNokpFLeg6/HNd/kNK9 DK3HgzMYjmg1HiZs4bHFhC09tpywlcdWE3bpsUuHHU64BXCMn3Cn+KPDa8mY7KD624Ob8F/QkARz IAqwrm7KceRk2gNYtB4Ijik844qBilr8G1G04uR5E67iJHHuozWuGxziV7aOc8bqFRpUxBJcWH2p Xjn3Tf5LLLjQoKTYkPmJF9NSuRgCZ9TYHBTuHys1PrlfTH/1ytM/2+0PAAAA//8DAFBLAwQUAAYA CAAAACEAb2NQm4sBAAAHBAAAEgAAAHdvcmQvZm9udFRhYmxlLnhtbKSS326CMBTG75fsHUjvJwX/ TI1onNPLXSzuAY5YpAltSU+V+fY7UORiZolkkDThO+3H6e98i9W3KoKLsCiNTlg04CwQOjVHqU8J +9rvXqYsQAf6CIXRImFXgWy1fH5aVPPMaIcBndc4twnLnSvnYYhpLhTgwJRCUy0zVoGjT3sKTZbJ VLyb9KyEdmHM+SS0ogBH/8Zclshat+oRt8rYY2lNKhCpWVV4PwVSs2XbXVDNNSjqegOFPFjZFErQ BkVEtQsUCeMx3/ExrfU74sN6ZWHtkOZgUbhuI/dyBkoW15uKlUT0hVK6NL/pF7ASDoXwJZQnKpzx wBO2jTjn8W7HvBIlbETCetMpMTXln1m7Z9gpNB5qrPFptkSzxocU8mlPNX2Gfj53JPZSCQw+RBV8 GgUe1T2RmE+IxJh41GSGvYjYxrch+CgRajxed/enm2xIeZ2Oovb+vYh4n8eJbEBRNOCPbNQEPIma SL9s9CexrkcYb39lg/PR2x2JJgmUqP9kow0JLn8AAAD//wMAUEsDBBQABgAIAAAAIQAoh3GlzwAA AB8BAAAUAAAAd29yZC93ZWJTZXR0aW5ncy54bWyMj8tOAzEMRfdI/MMoe5qBRYVGnamEUNlQqMRj n2Y8nUiJHdmB0H495rFhx/LaV8fHq/VHis07sATC3lwuWtMAehoDHnrz8ry5uDaNFIeji4TQmyOI WQ/nZ6vaVdg/QSnalEYpKB33Zi4ld9aKnyE5WVAG1N1EnFzRyAdL0xQ83JJ/S4DFXrXt0jJEV9RA 5pDF/NLqf2iVeMxMHkRUJMUfXnIBzaCOlEtI4QQb4humKsD2a6z3jo/4ur3/Ti5GqruHOw32z1vD JwAAAP//AwBQSwMEFAAGAAgAAAAhAPeMfebgAQAA2wMAABAACAFkb2NQcm9wcy9hcHAueG1sIKIE ASigAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnFPBbtswDL0P2D8Yvjd2nCUoAkXFkGLo YVsDxG3PnEzHwmRJkNSg2dePshtX2XqqT+Qj/fT0SLGbl15lR3ReGr3J57Myz1AL00h92OQP9ber 6zzzAXQDymjc5Cf0+Q3//IntnLHogkSfEYX2m7wLwa6LwosOe/AzKmuqtMb1ECh1h8K0rRR4a8Rz jzoUVVmuCnwJqBtsruxEmI+M62P4KGljRNTnH+uTJcGc1dhbBQH5zyhHsWICWG0CqFr2yBcl4VPG dnBAz+esGAP2ZFzj+bIkZAzZtgMHIpB7vLpe0d8JwL5aq6SAQMbyH1I4400bsvvBgiwSsCJtYWTL HsWzk+HEiSpN2XepSUq1YMUYkTYHBwe283wVBU4Z2wtQuKXL8xaUR1a8AewOIQ52B5IUs2NYH1EE 4zIv/9Boqzz7BR6jZZv8CE6CDmRdbBuTIVbWB8drGRRxU23MhzBtS2P5JbpIvRRcNkZw1ECFS3XD Cf6+pbuFd8TOU7GDhlFqIicJpzP+Yd2a3oI+0eFTRAb/9g+2NrdxXV49vASTuT/J0O0tCJrOYrFc phuQlNieFgUbGumZ8A1gd+S3U/FU2h59wObc838h7tTj+Fb5vJqV9A1LdMZoE6ZHxP8CAAD//wMA UEsDBBQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAgBZG9jUHJvcHMvY29yZS54bWwgogQBKKAAAQAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACMkstuwjAQRfeV+g+R94mTAC2ykiD1wQraSk0f6s6y B4gaP2S7BP6+TgIpqF10Ob7XZ+6Mnc12og62YGylZI6SKEYBSKZ4Jdc5einn4RQF1lHJaa0k5GgP Fs2Ky4uMacKUgSejNBhXgQ08SVrCdI42zmmCsWUbENRG3iG9uFJGUOdLs8aask+6BpzG8RUW4Cin juIWGOqBiA5Izgak/jJ1B+AMQw0CpLM4iRL843VghP3zQqecOEXl9trPdIh7yuasFwf3zlaDsWma qBl1MXz+BL8vF8/dqGEl210xQEXGGWEGqFOmqEGqrcrwyVG7vppat/SbXlXAb/bF4v7h8fUxw7+V 1mxgW7VvVEw6x1D6Pt1YfTPggQ9K+rGOytvo9q6coyKNk0mYpGEyLuNrMhmROP5oQ53db4P3B+IQ 7R/EdFqmMUnG58QjoOgSn3+W4hsAAP//AwBQSwMEFAAGAAgAAAAhAOYmwWoaBwAA+zkAAA8AAAB3 b3JkL3N0eWxlcy54bWy0m1FT2zgQx99v5r6Dx+80ISnJlWnaoRSuzLQcbWDuWbEVoqlj5SylQD/9 SStbGBvbu9g8EcvW/lba1X+dIL3/eL9Ngl88U0Kmi/DwzTgMeBrJWKS3i/Dm+vzgrzBQmqUxS2TK F+EDV+HHD3/+8f7uWOmHhKvAGEjVcbYIN1rvjkcjFW34lqk3csdTc28tsy3T5jK7Hcn1WkT8s4z2 W57q0WQ8no0ynjBt4GojdirMrd1hrN3JLN5lMuJKGW+3ibO3ZSINPxj3Yhl95mu2T7Syl9lVll/m V/DnXKZaBXfHTEVCXBvHzRC3IpXZl5NUidDc4UzpEyVY+eZZ3mbvb+yD5Zu+Z6R0yeAnEYtwZKHq t+n2iyWLcDIpWk6tE0/aEpbeFm08Pbi4LDuzCH3TythdhCw7WJ5YYyMYafG3NOLdk/GbK3BlxyIz d4bD1pqbGJqQWE4ibKwn81lx8WOfmAa21zKHgAEDK5s1l5VJN6E1gV66RDF3+fqrjH7yeKnNjUUI LNN4c3GVCZkJ/bAI372zTNO45FvxRcQxt3mZt92kGxHzfzc8vVE8fmz/fg5ZlluM5D7Vxv3ZHBIh UfHZfcR3NsuM6ZTZIF/aDok1q0occGgvHr1xDRUqNP5XIA9dDJ+lbDizKykA/1tBMOp9b9DEjqg8 ALBL8nXa38Tb/iaO+puA5O03F/P+Xhj97BsRlxulrMQHVcvIJV95HqbvWlLW9qhlUWePWtJ09qjl SGePWkp09qhlQGePWsA7e9Ti29mjFs7WHhED4apm0RRmA7Wwr4VOuO3fKkCHPaUuLzXBFcvYbcZ2 m8DW1qrbbWK53K80zlWQ05eL5VJnMr3tnBFTne3SfbEmn213G6aEeanpmPpJz6m/ZquEB39nIu5E Hbnkq40JXkyeLWFXCYv4RiYxz4Jrfu8iSuh/KYOle8vodK5nWL+K240OlhsouZ2wWcOkN8+Es/9V KJiD1sU0axhKl3FUDGcNedls/BuPxX5bTA3ibWTm9JwQ5goCXGyforc2RPXV1TkKGwDMEFy5oA8B 7CP8d8WFbt/GGOO/K0UvtI/w3xWuF9qH/GiPL1lpPrPsZ4BaXnPy2j2ViczW+6RYA53yMCevYI/A DYG8iL19lEjMySv4iXwGJ1Fkvrlh8pQci0cdJVDI4XAUWGz4sZCDUpG9Q8KIyAGqsCYEVj+tJYDI ovuD/xL2tydqMQCV9u+anct52jADpgSh3qG/76XufoeeNGgelnKRmp9LFA9wtGnDysPS8nxy9Y4Q 436FjwDqVwEJoH6lkABqyI/mdx5fE/GQ/sWRwCLLsq9ikHZoZZ6TldmDaCVgoLqJeP9qWL3NuVCv mwgKOUD1uomgkKNTqWW+biJYg9VNBKuhajTHqKyplEGR62YZ5N8EECMaRrwRoGHEGwEaRrwRoP7i 3Q0ZTrwRLLI2eE0tizcCBI9Qvup7UFm8ESCyNji1y38zKuoeWGn/cjuAeCMo5ADVxRtBIUenSbwR LHiEkgkVlpc6BGsY8UaAhhFvBGgY8UaAhhFvBGgY8UaA+ot3N2Q48UawyNrgNbUs3ggQWR48qCze CBA8QtGGZ8UbVv2rizeCQg5QXbwRFHJ0KoLqX1IRLHKAKiwv3ggWPEJJhpwFyU0Z1DDijRjRMOKN AA0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwRILI8eFBZvBEgsjY8K96wGF9dvBEUcoDq4o2gkKNT EVSvcwgWOUAVlhdvBAvypbd4I0DwyEtBlBENI96IEQ0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwR ILI8eFBZvBEgsjY8K96wRl5dvBEUcoDq4o2gkKNTEVQv3ggWOUAVlpc6BGsY8UaAIDF7izcCBI+8 AASriBKmYcQbMaJhxBsB6i/e3ZDhxBvBImuD19SyeCNAZHnwoLJ4I0BkbbD7bM1+UfT21MOGJMDu Myh2NaCBk4YgYYH5AH/wNc/MYSbevTukJ7AYIYHYkB7YIX6S8meA29g9bUgQNEqsEiFhS/cD7NIp HUSYzltOElz/cxp8cQdgav0gpZ7uvDGnh8rHheB4kj04ZPzUDztzZGdX7Cy31swBIXu0Kz8CBEfR LsyBoPxYj+1sz/mYB+FQVd4M/7fNqfDZHHuLi2fG46Oz+fjoPD/gBCbrTkQb40Vkzkq1OJFvhfe7 k2AjfNWlhv3y4NbjYY3CuXzf/OPblXvuye5N02TmsMFvbfeIt/gMe8hbZy+AR1y86w6aY1vgUpeH fr8VPK1XiTuIZj5cpDYU5uQf/G/NhTy+Z86suX/Kk+Qbg2NrWu6aH034Wru7h2OokxVTK6m13Db3 z2AbOXjynAEzxWVn3KUdRPPcp/vtimfmHFjL/F9KW1/gvNrTxHU7Yl24/coz3kNeY2f90bfik/rw PwAAAP//AwBQSwECLQAUAAYACAAAACEA3fyVN2YBAAAgBQAAEwAAAAAAAAAAAAAAAAAAAAAAW0Nv bnRlbnRfVHlwZXNdLnhtbFBLAQItABQABgAIAAAAIQAekRq38wAAAE4CAAALAAAAAAAAAAAAAAAA AJ8DAABfcmVscy8ucmVsc1BLAQItABQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAAAAAAAAAAAAAAA AMMGAAB3b3JkL19yZWxzL2RvY3VtZW50LnhtbC5yZWxzUEsBAi0AFAAGAAgAAAAhAKsR9BCDCgAA yx0AABEAAAAAAAAAAAAAAAAA/wgAAHdvcmQvZG9jdW1lbnQueG1sUEsBAi0AFAAGAAgAAAAhAJa1 reKWBgAAUBsAABUAAAAAAAAAAAAAAAAAsRMAAHdvcmQvdGhlbWUvdGhlbWUxLnhtbFBLAQItABQA BgAIAAAAIQChCYNYJQMAAB8HAAARAAAAAAAAAAAAAAAAAHoaAAB3b3JkL3NldHRpbmdzLnhtbFBL AQItABQABgAIAAAAIQBvY1CbiwEAAAcEAAASAAAAAAAAAAAAAAAAAM4dAAB3b3JkL2ZvbnRUYWJs ZS54bWxQSwECLQAUAAYACAAAACEAKIdxpc8AAAAfAQAAFAAAAAAAAAAAAAAAAACJHwAAd29yZC93 ZWJTZXR0aW5ncy54bWxQSwECLQAUAAYACAAAACEA94x95uABAADbAwAAEAAAAAAAAAAAAAAAAACK IAAAZG9jUHJvcHMvYXBwLnhtbFBLAQItABQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAAAAAAAAAAA AAAAAKAjAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQDmJsFqGgcAAPs5AAAPAAAA AAAAAAAAAAAAAC0mAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAAdC0AAAAA --===============0921340515==-- From MAILER-DAEMON Mon Jan 4 14:59:22 2016 Return-Path: <> X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: * X-Spam-Status: No, score=1.9 required=5.0 tests=FREEMAIL_FROM,FREEMAIL_REPLYTO, TO_MALFORMED,T_FREEMAIL_DOC_PDF autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 6ED7F7F37 for ; Mon, 4 Jan 2016 14:59:22 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 241338F8033 for ; Mon, 4 Jan 2016 12:59:19 -0800 (PST) X-ASG-Debug-ID: 1451941155-04bdf0264e454270001-NocioJ Received: from mail.gaf.ni.ac.rs (mail.gaf.ni.ac.rs [160.99.31.9]) by cuda.sgi.com with ESMTP id vXOw5gynBeZoB4qv (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 12:59:16 -0800 (PST) X-Barracuda-Envelope-From: X-Barracuda-Apparent-Source-IP: 160.99.31.9 Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 837CB810E48C4; Mon, 4 Jan 2016 21:59:14 +0100 (CET) Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id M8fTBcaomELQ; Mon, 4 Jan 2016 21:59:14 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.gaf.ni.ac.rs (Postfix) with ESMTP id 50726810E48C0; Mon, 4 Jan 2016 21:59:14 +0100 (CET) X-Virus-Scanned: amavisd-new at gaf.ni.ac.rs Received: from mail.gaf.ni.ac.rs ([127.0.0.1]) by localhost (mail.gaf.ni.ac.rs [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4fDr0toLp2E1; Mon, 4 Jan 2016 21:59:14 +0100 (CET) Received: from [100.67.150.48] (unknown [116.202.38.88]) by mail.gaf.ni.ac.rs (Postfix) with ESMTPSA id DF9AF810E48C9; Mon, 4 Jan 2016 21:59:05 +0100 (CET) Content-Type: multipart/mixed; boundary="===============0781748591==" MIME-Version: 1.0 Subject: Re To: Recipients <> X-ASG-Orig-Subj: Re From: "<"@LIVE.COM Date: Tue, 05 Jan 2016 02:29:04 +0530 Reply-To: cruzcelia2011@gmail.com Message-Id: <20160104205905.DF9AF810E48C9@mail.gaf.ni.ac.rs> X-Barracuda-Connect: mail.gaf.ni.ac.rs[160.99.31.9] X-Barracuda-Start-Time: 1451941156 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 2.00 X-Barracuda-Spam-Status: No, SCORE=2.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0646, BSF_SC0_MV0646_4, EMPTY_ENV_FROM, NO_REAL_NAME, TO_MALFORMED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25830 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 TO_MALFORMED To: has a malformed address 0.00 EMPTY_ENV_FROM Empty Envelope From Address 0.00 NO_REAL_NAME From: does not include a real name 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.50 BSF_SC0_MV0646 Custom rule MV0646 1.50 BSF_SC0_MV0646_4 BSF_SC0_MV0646_4 You will not see this in a MIME-aware mail reader. --===============0781748591== Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body please kindly open the fill --===============0781748591== Content-Type: application/octet-stream MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="BUSINESS PROPOSAL.docx" UEsDBBQABgAIAAAAIQDd/JU3ZgEAACAFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0 VMtuwjAQvFfqP0S+Vomhh6qqCBz6OLZIpR9g7A1Y9Uv28vr7bgJEVQtBKuUSKVnvzOzsxIPR2pps CTFp70rWL3osAye90m5Wso/JS37PsoTCKWG8g5JtILHR8PpqMNkESBl1u1SyOWJ44DzJOViRCh/A UaXy0Qqk1zjjQchPMQN+2+vdcekdgsMcaww2HDxBJRYGs+c1fd4qiWASyx63B2uukokQjJYCSSlf OvWDJd8xFNTZnElzHdINyWD8IENdOU6w63sja6JWkI1FxFdhSQZf+ai48nJhaYaiG+aATl9VWkLb X6OF6CWkRJ5bU7QVK7Tb6z+qI+HGQPp/FVvcLnrSOY4+JE57OZsf6s0rUDlZESCihnZ1x0cHRLLs EsPvkLvGb1KAlHfgzbN/tgcNzEnKin6JiZgaOJvvV/Ja6JMiVjB9v5j738C7hLT5kz7+wYz9dVF3 H0gdb+634RcAAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJlbHMg ogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfLT9ab g5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+ImYz sKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kVyAdh b9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ5x687 HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAgBd29yZC9f cmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AKySzWrDMBCE74W+g9h7LTv9oYTIuZRArq37AIq9/qGyJLSbtn77CkNShwb34otgRmjmk7Sb7Xdv xCcG6pxVkCUpCLSlqzrbKHgvdnfPIIi1rbRxFhUMSLDNb282r2g0x0PUdp5ETLGkoGX2aympbLHX lDiPNu7ULvSaowyN9Lr80A3KVZo+yTDNgPwiU+wrBWFf3YMoBh+b/892dd2V+OLKY4+Wr1TILzy8 IXO8HMVYHRpkBRMzibQgr4OslgShPxQnZw4hWxSBBxM/8/wMNOq5+scl6zmOCP62j1KOazbH8LAk Q+0sF/pgJhxn6wQhLwY9/wEAAP//AwBQSwMEFAAGAAgAAAAhAKsR9BCDCgAAyx0AABEAAAB3b3Jk L2RvY3VtZW50LnhtbNRYzW7bRhC+F+g7DIQcWkCWKAcuEjVW4FpO40NTI7ZzX5FLcWFyl91dSuGt r9FbniWP0ifpN0tSliXFFgSjRU+0l9z5n++b0Zu3n4ucFtI6ZfRpbzSIeiR1bBKl56e925t3R696 5LzQiciNlqe9Wrre28n3371ZjhMTV4XUniBCu/ECbzPvy/Fw6OJMFsINTCk1XqbGFsLjXzsfFsLe VeVRbIpSeDVTufL18DiKfuq1Ysxpr7J63Io4KlRsjTOp5ytjk6Yqlu2ju2H30dvcnLYmB41DK3PY YLTLVOk6acWh0uBi1glZPObEosi775blPtoSK5bIR5E3Zi+NTUprYukcTqfNy5XEUfSY7jaALGJ1 Yx8THursLCmE0isxXB0b+V8lb4DkDRvdQxZ17whiMUEtzUxS87Ok5Ri1mHw87UXR+Un07mLa646m MhVV7vnNL+fHr6Yvw83yyvJF2zxmwyDt3OE5bE/x7D564svl2E9+tVJ6xNUN6CqXwkkqpS2Up0KS N6S0tyapYklF7WSe8lltKjKaj4yVA7okUZDA94mKRU4xCgxmCzTKrCYkLuXEGU1oKkIw7thQ35gb 7Ov8ZUePR6PXoyg4+oiHuM/dM3aliJGF0kon7UL2Jo6EY5VxZWVo1UJoMZeWlspn9Jjily9P3p1M 91A8uS6FVi6jq0ygzWNZeXZ7U/bTkd/tAZ0zUuiaw7qEM0rTpU6U4D+Oo9GIAC4INnssbJxRWdnS IGlJZZFEWmYKh5eUKBcbwJxMyGfC0w+b9h0U9MkNlAj6JK34MVgnFkLlYpbLlZ2D51A0s205H1wf QQKX2c2mPYcmJni+KeygKO5OPAd1q0KfNnZyLZFj5ahyeHJxhD5EmvKuNlx/w2w0iEkvLLpi7OsS DTS3orj2wvpegJOnAIbj+vULUr6P3Aud7C11d2TI+IxbGDiKfvDsrNDQL2KvFpLwbsbecwmi3asU 500zmJTOtFf0/vIT3eAzUdZ9eg+PrZc6QNIV2FlpB3C6mJlc0FShrZx0dOHjAe5AlWvj60oZK7Cx yPMaca4AZlAIyyhTfCSsFASN16YC1FwI5+nMKdEPsCfiGNZzgwI+0TDGCm9sTV7is1UvszAMHZRa U7RdnwECZlJqBrUFHrg+Ay7jwxmuMkhkAjFYKKc88t9ABdzxGWCdTBwL9tPxPSBFDHF8u3OLEZll JZhe8CcrA3qDDSrGDUMW4SA4jFBjKEFUKosRJ3h0q4NCVA18YL/PCmkRnYBbCEgwWCWBQ5gvWA3L RnKgzgM2+BKf5oZZI5EiR5JF6jnVEkERADIxhxsByGENcwesmkvcsls482SrPASV/T7nSr9EcVUh 6zsimG60APfUis/3JrMJ2vYjCPhl1KcoioZHzMEEcruTnsM02tTypPXf4MdoXjTVtRn4Ps0qJKYK +VLekRUWk2pINX+bqXnWlkkfRYdJIN8Cq8Ncv69GpPi3uuO/AGUvthxf17E/ZffpJAqhfa440u01 TU2eC+s2RR5m4dcvaMBNUYdmme6kLENv8fQFxAy0EMa3tbECYGDnwIj0HhkYBFxVloAzPg6DH2qh AiTk7TyCw8Iw1CBbgrRcoipKMEdoUgDiVMaymKF6c0bAGkMK46hcpZUxRpslibI0mC0hqJECudtD W4amZywJ8HY/By2lvHN///kXedUMqduN+e/BAzunnEPnWPSID84BX8PUhTnM4MBjVsI0zH3EAdvl KqIJPMckxZNloqwETGKHKCoNTOWdrZlgWcQ6XqJtZSyY/B6qylTBCUoVEgo+CqqZFnkoAEqHrLMs JrcmP82gHLKI1Bf3fQhfPvx+Q2IJgusge1vUOqOEfQBgbRWXCU+fnefNRgSM5KTKz2FwlZhSa4AA uFGlIHOEDITywHJwQWt3y8nQxlEFj8I3pojk524wCEfe1i3XItaeYwmHuNrmDGor1rxDITJVsjjk pdsVuFtSniy4ennWAK/yivNHBcpjElsoLD+ta6CqWGi2JmY8CMweRpEgdlZhZcX687AceUHj3v6v l79uSN49dgGL0aQcelQSUzjCxKHBn+BprHT8MwlKCW/X+q+pqd0k07GJBrsfBQoBNuE6C2zXnm82 yDLD9IPqECirBQoEUBOGnaZ613uiqa4gvivxzoNNgF3H6v3JmoPQsvUzkfUEvNwwMFu8Cx/QMc9j fNgKmjmQiYEnTc4tfMJE/WLUP9lU8zQJTcCt0ea1w0Ib0e1gxawwqFlnUXhNq7/eUrOHdX0kiS4/ fAyVAgwL4NHCoJZz41UDsMjqDIwolGbsAAhiCydsB0k/4Nc3ssK/d+Q5Q0ybOvxWshZXBo9S1CHG mMWz/yMQnIWZ/B8AAAD//6xWXW7bOBC+ysBA31xXTtLYzTYG6iQt+rCA0e4FKGkkEZZIlaSiuk+9 Rt/2LD3KnmS/oawkNookBQokkvgzw5lvvm/oUDG1zmad44ZNoEYZVbKjSnlqO5fhzTnJLs/4KJxt aGe7qTyo13VNOeddFuKWe4PM+kDK5LLN0dtX/UVYydOt3uJB+Pc6/3Q5SZLzk/nN4noS5zdxOX0l g/TK4y0mw2xYBaduua61KY/9PW1MX5v6wrcq48tJ69izu+XJivhryybXAdlPY7hI1AypOFvoMGSY IvlKOaSfcugZOzpPKtBp8oJe0iJ5MaW+0llFDSvj75EJasuyLA7Nz3/p4+AuTsN2Fqccq7reUW7J WJynUINgqbaeyXaBrISjPaWd14Y9PjhTHRYRHKaVIdu21oXO6LCjwjoEIQ4aObuxhjEpNZMyZLZp ldnN6J1HLF7pnFi5WrObxpxzRvVzssVwJGgRK4tj5kmSENDbcvB/jUenEl9le+HMblyN2WcIa+TC cLwQqLaZErqoejzQdE0KruHAve9ofWe5p1dg1yD3GKJqbAeExGJfIB5ALTnMDmgB5rTCi/Yh2c7W i+Wbm8k4dc2F6uogNFxfnSyvTwcaPkmn1EWKhl+z6iOpBlibQueiqFCBKr0O1aCFCjXxKJUgjZJm 7KRwklDnWql6v88oVrC0FqLTRplMA7xSaUP4O6DEbCRW2eHECCFYU3QOoLsobigXqgFbPHmLNbxF vjFGF/XsCDzEJmEY2HOA5Kjax9RZKx8+QUoMlWzQQNZwt30MpN667YBKwzNag0kqtbdMEMMU+Wy1 yaEKx186oBWjRVhb5nbIHVRuQPlMBY187sAGRjPa1IymJYryiEj0EJsQNgWFRrUnneB/nObIi9/r SyrP0VE86eIxfyfz+Zt58ow+92tS/aFYgcR/3384tA8DWQFazo+DfpL9YTWjfyplttJ+cvoAhqY1 ADjU33NYM+potT4O4mElno3cSqjiuFQu99M/4fEuvncQ6TdFV7Vy6tjxc/A6sBk7k+csbA7uwsP2 9BnrwsTXN4vk9fvInLb8/A3Q9JcTsCk5l0ZW4ft8ebqcRLW15d9KXAbbYv7sLIm9TpcVPI3D1IZg m/txzcWD1Qp9gN3lZJEsxbawFkS5G5ZdiMNkOC6ztcem/cUqJjGK3GYfHG6X/gK3NW90yBDl6Xk0 QvZD4vFyT22+ix8w6eQHyOp/AAAA//8DAFBLAwQUAAYACAAAACEAlrWt4pYGAABQGwAAFQAAAHdv cmQvdGhlbWUvdGhlbWUxLnhtbOxZT2/bNhS/D9h3IHRvYyd2Ggd1itixmy1NG8Ruhx5piZbYUKJA 0kl9G9rjgAHDumGHFdhth2FbgRbYpfs02TpsHdCvsEdSksVYXpI22IqtPiQS+eP7/x4fqavX7scM HRIhKU/aXv1yzUMk8XlAk7Dt3R72L615SCqcBJjxhLS9KZHetY3337uK11VEYoJgfSLXcduLlErX l5akD8NYXuYpSWBuzEWMFbyKcCkQ+AjoxmxpuVZbXYoxTTyU4BjI3hqPqU/QUJP0NnLiPQaviZJ6 wGdioEkTZ4XBBgd1jZBT2WUCHWLW9oBPwI+G5L7yEMNSwUTbq5mft7RxdQmvZ4uYWrC2tK5vftm6 bEFwsGx4inBUMK33G60rWwV9A2BqHtfr9bq9ekHPALDvg6ZWljLNRn+t3slplkD2cZ52t9asNVx8 if7KnMytTqfTbGWyWKIGZB8bc/i12mpjc9nBG5DFN+fwjc5mt7vq4A3I4lfn8P0rrdWGizegiNHk YA6tHdrvZ9QLyJiz7Ur4GsDXahl8hoJoKKJLsxjzRC2KtRjf46IPAA1kWNEEqWlKxtiHKO7ieCQo 1gzwOsGlGTvky7khzQtJX9BUtb0PUwwZMaP36vn3r54/RccPnh0/+On44cPjBz9aQs6qbZyE5VUv v/3sz8cfoz+efvPy0RfVeFnG//rDJ7/8/Hk1ENJnJs6LL5/89uzJi68+/f27RxXwTYFHZfiQxkSi m+QI7fMYFDNWcSUnI3G+FcMI0/KKzSSUOMGaSwX9nooc9M0pZpl3HDk6xLXgHQHlowp4fXLPEXgQ iYmiFZx3otgB7nLOOlxUWmFH8yqZeThJwmrmYlLG7WN8WMW7ixPHv71JCnUzD0tH8W5EHDH3GE4U DklCFNJz/ICQCu3uUurYdZf6gks+VuguRR1MK00ypCMnmmaLtmkMfplW6Qz+dmyzewd1OKvSeosc ukjICswqhB8S5pjxOp4oHFeRHOKYlQ1+A6uoSsjBVPhlXE8q8HRIGEe9gEhZteaWAH1LTt/BULEq 3b7LprGLFIoeVNG8gTkvI7f4QTfCcVqFHdAkKmM/kAcQohjtcVUF3+Vuhuh38ANOFrr7DiWOu0+v Brdp6Ig0CxA9MxHal1CqnQoc0+TvyjGjUI9tDFxcOYYC+OLrxxWR9bYW4k3Yk6oyYftE+V2EO1l0 u1wE9O2vuVt4kuwRCPP5jeddyX1Xcr3/fMldlM9nLbSz2gplV/cNtik2LXK8sEMeU8YGasrIDWma ZAn7RNCHQb3OnA5JcWJKI3jM6rqDCwU2a5Dg6iOqokGEU2iw654mEsqMdChRyiUc7MxwJW2NhyZd 2WNhUx8YbD2QWO3ywA6v6OH8XFCQMbtNaA6fOaMVTeCszFauZERB7ddhVtdCnZlb3YhmSp3DrVAZ fDivGgwW1oQGBEHbAlZehfO5Zg0HE8xIoO1u997cLcYLF+kiGeGAZD7Ses/7qG6clMeKuQmA2Knw kT7knWK1EreWJvsG3M7ipDK7xgJ2uffexEt5BM+8pPP2RDqypJycLEFHba/VXG56yMdp2xvDmRYe 4xS8LnXPh1kIF0O+EjbsT01mk+Uzb7ZyxdwkqMM1hbX7nMJOHUiFVFtYRjY0zFQWAizRnKz8y00w 60UpYCP9NaRYWYNg+NekADu6riXjMfFV2dmlEW07+5qVUj5RRAyi4AiN2ETsY3C/DlXQJ6ASriZM RdAvcI+mrW2m3OKcJV359srg7DhmaYSzcqtTNM9kCzd5XMhg3krigW6Vshvlzq+KSfkLUqUcxv8z VfR+AjcFK4H2gA/XuAIjna9tjwsVcahCaUT9voDGwdQOiBa4i4VpCCq4TDb/BTnU/23OWRomreHA p/ZpiASF/UhFgpA9KEsm+k4hVs/2LkuSZYRMRJXElakVe0QOCRvqGriq93YPRRDqpppkZcDgTsaf +55l0CjUTU4535waUuy9Ngf+6c7HJjMo5dZh09Dk9i9ErNhV7XqzPN97y4roiVmb1cizApiVtoJW lvavKcI5t1pbseY0Xm7mwoEX5zWGwaIhSuG+B+k/sP9R4TP7ZUJvqEO+D7UVwYcGTQzCBqL6km08 kC6QdnAEjZMdtMGkSVnTZq2Ttlq+WV9wp1vwPWFsLdlZ/H1OYxfNmcvOycWLNHZmYcfWdmyhqcGz J1MUhsb5QcY4xnzSKn914qN74OgtuN+fMCVNMME3JYGh9RyYPIDktxzN0o2/AAAA//8DAFBLAwQU AAYACAAAACEAoQmDWCUDAAAfBwAAEQAAAHdvcmQvc2V0dGluZ3MueG1snFXbbts4EH1fYP9B0PM6 lmQ7ToQ4RWNb3RZJW6zSD6CkkUyEN5CUFffrO5TEKul6g2KfTJ0zczicm2/ePXMWHEEbKsUmjC+i MABRyoqKZhN+e8xmV2FgLBEVYVLAJjyBCd/d/vnHTZcasBbNTIASwqRyE7ZapKY8ACdmxmmppZG1 nZWSp7KuaQnjTzh66E14sFal8/nodCEVCFSrpebEmgupm/nguZNly0HYeRJFl3MNjFgM2ByoMl6N /181vOrgRY5vPeLImbfr4ugty/G5ndTVT4/fCc85KC1LMAYzy9nwXE6o8DKG/Y7OkM97WmiiTy9E brFs36XkQZcq0CUmFGseReHcEZX8LO2OGsXI6Stp4E62WHZNwfQ0xiXr3BIL6G0UMNb3SMmAYHRd 2mjCOcGaDsggCTVpmX0kRW6lQqMjwfjXyXhjeSCalBZ0rkiJalsprJbM2/UBbSVXGvMxxIi9pIh1 0bYGsv09OcnW4lXzLp0obObKOBt3+EdK6wWj6G6bXO0Wg5ZjJyaJ4+t4jOs1s1isstXunM/ybn11 vT/HrPbraJWdYy6TeL8+q/Y+W27jsz7/HfV2FWX7s2rZ1fI6XrsIMDdjRnjqev2rvr0ZThnmO+BD UbaEF5qS4MFNA3rxtNBPd1R4vgCcSnjJ5G3hydlsIAwnjGVYU0/gIAxMhX21g7oXZg9EN5Nyn3Se 6rNoBfWnn2quYUF/0LJVg2qnifooKoT9hfFyOepRYe8p97hpi9x7CRyKFxR2+ZejdoLzKUFdanGP gcvQPRGN7xMQs2+5M8V+Yzp3uw4eiFLYvGhSNPEmZLQ52NhNhMUvHKCn/qNokpFLeg6/HNd/kNK9 DK3HgzMYjmg1HiZs4bHFhC09tpywlcdWE3bpsUuHHU64BXCMn3Cn+KPDa8mY7KD624Ob8F/QkARz IAqwrm7KceRk2gNYtB4Ijik844qBilr8G1G04uR5E67iJHHuozWuGxziV7aOc8bqFRpUxBJcWH2p Xjn3Tf5LLLjQoKTYkPmJF9NSuRgCZ9TYHBTuHys1PrlfTH/1ytM/2+0PAAAA//8DAFBLAwQUAAYA CAAAACEAb2NQm4sBAAAHBAAAEgAAAHdvcmQvZm9udFRhYmxlLnhtbKSS326CMBTG75fsHUjvJwX/ TI1onNPLXSzuAY5YpAltSU+V+fY7UORiZolkkDThO+3H6e98i9W3KoKLsCiNTlg04CwQOjVHqU8J +9rvXqYsQAf6CIXRImFXgWy1fH5aVPPMaIcBndc4twnLnSvnYYhpLhTgwJRCUy0zVoGjT3sKTZbJ VLyb9KyEdmHM+SS0ogBH/8Zclshat+oRt8rYY2lNKhCpWVV4PwVSs2XbXVDNNSjqegOFPFjZFErQ BkVEtQsUCeMx3/ExrfU74sN6ZWHtkOZgUbhuI/dyBkoW15uKlUT0hVK6NL/pF7ASDoXwJZQnKpzx wBO2jTjn8W7HvBIlbETCetMpMTXln1m7Z9gpNB5qrPFptkSzxocU8mlPNX2Gfj53JPZSCQw+RBV8 GgUe1T2RmE+IxJh41GSGvYjYxrch+CgRajxed/enm2xIeZ2Oovb+vYh4n8eJbEBRNOCPbNQEPIma SL9s9CexrkcYb39lg/PR2x2JJgmUqP9kow0JLn8AAAD//wMAUEsDBBQABgAIAAAAIQAoh3GlzwAA AB8BAAAUAAAAd29yZC93ZWJTZXR0aW5ncy54bWyMj8tOAzEMRfdI/MMoe5qBRYVGnamEUNlQqMRj n2Y8nUiJHdmB0H495rFhx/LaV8fHq/VHis07sATC3lwuWtMAehoDHnrz8ry5uDaNFIeji4TQmyOI WQ/nZ6vaVdg/QSnalEYpKB33Zi4ld9aKnyE5WVAG1N1EnFzRyAdL0xQ83JJ/S4DFXrXt0jJEV9RA 5pDF/NLqf2iVeMxMHkRUJMUfXnIBzaCOlEtI4QQb4humKsD2a6z3jo/4ur3/Ti5GqruHOw32z1vD JwAAAP//AwBQSwMEFAAGAAgAAAAhAPeMfebgAQAA2wMAABAACAFkb2NQcm9wcy9hcHAueG1sIKIE ASigAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAnFPBbtswDL0P2D8Yvjd2nCUoAkXFkGLo YVsDxG3PnEzHwmRJkNSg2dePshtX2XqqT+Qj/fT0SLGbl15lR3ReGr3J57Myz1AL00h92OQP9ber 6zzzAXQDymjc5Cf0+Q3//IntnLHogkSfEYX2m7wLwa6LwosOe/AzKmuqtMb1ECh1h8K0rRR4a8Rz jzoUVVmuCnwJqBtsruxEmI+M62P4KGljRNTnH+uTJcGc1dhbBQH5zyhHsWICWG0CqFr2yBcl4VPG dnBAz+esGAP2ZFzj+bIkZAzZtgMHIpB7vLpe0d8JwL5aq6SAQMbyH1I4400bsvvBgiwSsCJtYWTL HsWzk+HEiSpN2XepSUq1YMUYkTYHBwe283wVBU4Z2wtQuKXL8xaUR1a8AewOIQ52B5IUs2NYH1EE 4zIv/9Boqzz7BR6jZZv8CE6CDmRdbBuTIVbWB8drGRRxU23MhzBtS2P5JbpIvRRcNkZw1ECFS3XD Cf6+pbuFd8TOU7GDhlFqIicJpzP+Yd2a3oI+0eFTRAb/9g+2NrdxXV49vASTuT/J0O0tCJrOYrFc phuQlNieFgUbGumZ8A1gd+S3U/FU2h59wObc838h7tTj+Fb5vJqV9A1LdMZoE6ZHxP8CAAD//wMA UEsDBBQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAgBZG9jUHJvcHMvY29yZS54bWwgogQBKKAAAQAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACMkstuwjAQRfeV+g+R94mTAC2ykiD1wQraSk0f6s6y B4gaP2S7BP6+TgIpqF10Ob7XZ+6Mnc12og62YGylZI6SKEYBSKZ4Jdc5einn4RQF1lHJaa0k5GgP Fs2Ky4uMacKUgSejNBhXgQ08SVrCdI42zmmCsWUbENRG3iG9uFJGUOdLs8aask+6BpzG8RUW4Cin juIWGOqBiA5Izgak/jJ1B+AMQw0CpLM4iRL843VghP3zQqecOEXl9trPdIh7yuasFwf3zlaDsWma qBl1MXz+BL8vF8/dqGEl210xQEXGGWEGqFOmqEGqrcrwyVG7vppat/SbXlXAb/bF4v7h8fUxw7+V 1mxgW7VvVEw6x1D6Pt1YfTPggQ9K+rGOytvo9q6coyKNk0mYpGEyLuNrMhmROP5oQ53db4P3B+IQ 7R/EdFqmMUnG58QjoOgSn3+W4hsAAP//AwBQSwMEFAAGAAgAAAAhAOYmwWoaBwAA+zkAAA8AAAB3 b3JkL3N0eWxlcy54bWy0m1FT2zgQx99v5r6Dx+80ISnJlWnaoRSuzLQcbWDuWbEVoqlj5SylQD/9 SStbGBvbu9g8EcvW/lba1X+dIL3/eL9Ngl88U0Kmi/DwzTgMeBrJWKS3i/Dm+vzgrzBQmqUxS2TK F+EDV+HHD3/+8f7uWOmHhKvAGEjVcbYIN1rvjkcjFW34lqk3csdTc28tsy3T5jK7Hcn1WkT8s4z2 W57q0WQ8no0ynjBt4GojdirMrd1hrN3JLN5lMuJKGW+3ibO3ZSINPxj3Yhl95mu2T7Syl9lVll/m V/DnXKZaBXfHTEVCXBvHzRC3IpXZl5NUidDc4UzpEyVY+eZZ3mbvb+yD5Zu+Z6R0yeAnEYtwZKHq t+n2iyWLcDIpWk6tE0/aEpbeFm08Pbi4LDuzCH3TythdhCw7WJ5YYyMYafG3NOLdk/GbK3BlxyIz d4bD1pqbGJqQWE4ibKwn81lx8WOfmAa21zKHgAEDK5s1l5VJN6E1gV66RDF3+fqrjH7yeKnNjUUI LNN4c3GVCZkJ/bAI372zTNO45FvxRcQxt3mZt92kGxHzfzc8vVE8fmz/fg5ZlluM5D7Vxv3ZHBIh UfHZfcR3NsuM6ZTZIF/aDok1q0occGgvHr1xDRUqNP5XIA9dDJ+lbDizKykA/1tBMOp9b9DEjqg8 ALBL8nXa38Tb/iaO+puA5O03F/P+Xhj97BsRlxulrMQHVcvIJV95HqbvWlLW9qhlUWePWtJ09qjl SGePWkp09qhlQGePWsA7e9Ti29mjFs7WHhED4apm0RRmA7Wwr4VOuO3fKkCHPaUuLzXBFcvYbcZ2 m8DW1qrbbWK53K80zlWQ05eL5VJnMr3tnBFTne3SfbEmn213G6aEeanpmPpJz6m/ZquEB39nIu5E Hbnkq40JXkyeLWFXCYv4RiYxz4Jrfu8iSuh/KYOle8vodK5nWL+K240OlhsouZ2wWcOkN8+Es/9V KJiD1sU0axhKl3FUDGcNedls/BuPxX5bTA3ibWTm9JwQ5goCXGyforc2RPXV1TkKGwDMEFy5oA8B 7CP8d8WFbt/GGOO/K0UvtI/w3xWuF9qH/GiPL1lpPrPsZ4BaXnPy2j2ViczW+6RYA53yMCevYI/A DYG8iL19lEjMySv4iXwGJ1Fkvrlh8pQci0cdJVDI4XAUWGz4sZCDUpG9Q8KIyAGqsCYEVj+tJYDI ovuD/xL2tydqMQCV9u+anct52jADpgSh3qG/76XufoeeNGgelnKRmp9LFA9wtGnDysPS8nxy9Y4Q 436FjwDqVwEJoH6lkABqyI/mdx5fE/GQ/sWRwCLLsq9ikHZoZZ6TldmDaCVgoLqJeP9qWL3NuVCv mwgKOUD1uomgkKNTqWW+biJYg9VNBKuhajTHqKyplEGR62YZ5N8EECMaRrwRoGHEGwEaRrwRoP7i 3Q0ZTrwRLLI2eE0tizcCBI9Qvup7UFm8ESCyNji1y38zKuoeWGn/cjuAeCMo5ADVxRtBIUenSbwR LHiEkgkVlpc6BGsY8UaAhhFvBGgY8UaAhhFvBGgY8UaA+ot3N2Q48UawyNrgNbUs3ggQWR48qCze CBA8QtGGZ8UbVv2rizeCQg5QXbwRFHJ0KoLqX1IRLHKAKiwv3ggWPEJJhpwFyU0Z1DDijRjRMOKN AA0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwRILI8eFBZvBEgsjY8K96wGF9dvBEUcoDq4o2gkKNT EVSvcwgWOUAVlhdvBAvypbd4I0DwyEtBlBENI96IEQ0j3gjQMOKNAPUX727IcOKNYJG1wWtqWbwR ILI8eFBZvBEgsjY8K96wRl5dvBEUcoDq4o2gkKNTEVQv3ggWOUAVlpc6BGsY8UaAIDF7izcCBI+8 AASriBKmYcQbMaJhxBsB6i/e3ZDhxBvBImuD19SyeCNAZHnwoLJ4I0BkbbD7bM1+UfT21MOGJMDu Myh2NaCBk4YgYYH5AH/wNc/MYSbevTukJ7AYIYHYkB7YIX6S8meA29g9bUgQNEqsEiFhS/cD7NIp HUSYzltOElz/cxp8cQdgav0gpZ7uvDGnh8rHheB4kj04ZPzUDztzZGdX7Cy31swBIXu0Kz8CBEfR LsyBoPxYj+1sz/mYB+FQVd4M/7fNqfDZHHuLi2fG46Oz+fjoPD/gBCbrTkQb40Vkzkq1OJFvhfe7 k2AjfNWlhv3y4NbjYY3CuXzf/OPblXvuye5N02TmsMFvbfeIt/gMe8hbZy+AR1y86w6aY1vgUpeH fr8VPK1XiTuIZj5cpDYU5uQf/G/NhTy+Z86suX/Kk+Qbg2NrWu6aH034Wru7h2OokxVTK6m13Db3 z2AbOXjynAEzxWVn3KUdRPPcp/vtimfmHFjL/F9KW1/gvNrTxHU7Yl24/coz3kNeY2f90bfik/rw PwAAAP//AwBQSwECLQAUAAYACAAAACEA3fyVN2YBAAAgBQAAEwAAAAAAAAAAAAAAAAAAAAAAW0Nv bnRlbnRfVHlwZXNdLnhtbFBLAQItABQABgAIAAAAIQAekRq38wAAAE4CAAALAAAAAAAAAAAAAAAA AJ8DAABfcmVscy8ucmVsc1BLAQItABQABgAIAAAAIQDWZLNR+gAAADEDAAAcAAAAAAAAAAAAAAAA AMMGAAB3b3JkL19yZWxzL2RvY3VtZW50LnhtbC5yZWxzUEsBAi0AFAAGAAgAAAAhAKsR9BCDCgAA yx0AABEAAAAAAAAAAAAAAAAA/wgAAHdvcmQvZG9jdW1lbnQueG1sUEsBAi0AFAAGAAgAAAAhAJa1 reKWBgAAUBsAABUAAAAAAAAAAAAAAAAAsRMAAHdvcmQvdGhlbWUvdGhlbWUxLnhtbFBLAQItABQA BgAIAAAAIQChCYNYJQMAAB8HAAARAAAAAAAAAAAAAAAAAHoaAAB3b3JkL3NldHRpbmdzLnhtbFBL AQItABQABgAIAAAAIQBvY1CbiwEAAAcEAAASAAAAAAAAAAAAAAAAAM4dAAB3b3JkL2ZvbnRUYWJs ZS54bWxQSwECLQAUAAYACAAAACEAKIdxpc8AAAAfAQAAFAAAAAAAAAAAAAAAAACJHwAAd29yZC93 ZWJTZXR0aW5ncy54bWxQSwECLQAUAAYACAAAACEA94x95uABAADbAwAAEAAAAAAAAAAAAAAAAACK IAAAZG9jUHJvcHMvYXBwLnhtbFBLAQItABQABgAIAAAAIQBTfUAmVgEAAHkCAAARAAAAAAAAAAAA AAAAAKAjAABkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQDmJsFqGgcAAPs5AAAPAAAA AAAAAAAAAAAAAC0mAAB3b3JkL3N0eWxlcy54bWxQSwUGAAAAAAsACwDBAgAAdC0AAAAA --===============0781748591==-- From david@fromorbit.com Mon Jan 4 15:37:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 35A407F37 for ; Mon, 4 Jan 2016 15:37:18 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id CBFDCAC002 for ; Mon, 4 Jan 2016 13:37:14 -0800 (PST) X-ASG-Debug-ID: 1451943428-04bdf06f7100610001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id 9v8HZ37XyFcbesP8 for ; Mon, 04 Jan 2016 13:37:08 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DyCACX5YpWPGu7LXleKAECgw8iMG2GYIF5oDICCgwBAQEBAQEGgQyKSYk+GAKFbwSBIE0BAQEBAQEHAQEBAUE/hGIvOyQ0BSUDBy2ILqBpoTMJGYV1igIJgnUMQRuBGwWNOYlNhUCICIFlh26FLwKOOoI6gkgqNINGgUoBAQE Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 08:06:44 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGCnz-0002nq-QY for xfs@oss.sgi.com; Tue, 05 Jan 2016 08:36:43 +1100 Date: Tue, 5 Jan 2016 08:36:43 +1100 From: Dave Chinner To: xfs@oss.sgi.com Subject: [ANNOUNCE] xfs: for-next branch rebased to 4922be5 Message-ID: <20160104213643.GE19802@dastard> X-ASG-Orig-Subj: [ANNOUNCE] xfs: for-next branch rebased to 4922be5 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="kVXhAStRUZ/+rrGn" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451943428 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25831 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- --kVXhAStRUZ/+rrGn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi folks, The for-next branch of the xfs kernel repository at git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git has just been updated. I rebased the xfs-log-torn-write branch to include the 32bit arch build fix that the kbuild robot noticed and Brain fixed. It's the base branch in the for-next tree so while only one commit changed, I had to restart the for-next branch build from scratch... -Dave The new head of the for-next branch is commit: 4922be5 Merge branch 'xfs-dax-fixes-for-4.5' into for-next New Commits: Alexander Kuleshov (1): [211fe1a] xfs: make xfs_buf_ioend_async() static Brian Foster (8): [a70f9fe] xfs: detect and handle invalid iclog size set by mkfs [9d94901] xfs: refactor log record unpack and data processing [b94fb2d] xfs: refactor and open code log record crc check [d7f3769] xfs: return start block of first bad log record during reco= very [6528250] xfs: support a crc verification only log record pass [eed6b46] xfs: refactor log record start detection into a new helper [7088c41] xfs: detect and trim torn writes during log recovery [609adfc] xfs: debug mode log record crc error injection Darrick J. Wong (5): [2e9101d] libxfs: make xfs_alloc_fix_freelist non-static [9b434a3] xfs: fix log ticket type printing [6d3eb1e] libxfs: use a convenience variable instead of open-coding t= he fork [96f859d] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is = correct [c5ab131] libxfs: refactor short btree block verification Dave Chinner (4): [3b0fe47] xfs: Don't use reserved blocks for data blocks with DAX [a6d7636] xfs: fix recursive splice read locking with DAX [7eeabbd] Merge branch 'xfs-misc-fixes-for-4.5' into for-next [4922be5] Merge branch 'xfs-dax-fixes-for-4.5' into for-next Eric Sandeen (2): [233135b] xfs: print name of verifier if it fails [f1f96c4] xfs: get mp from bma->ip in xfs_bmap code Jia He (1): [1d4292b] libxfs: Optimize the loop for xfs_bitmap_empty Markus Elfring (1): [a841b64] XFS: Use a signed return type for suffix_kstrtoint() Masatake YAMATO (1): [ffc671f] xfs: send warning of project quota to userspace via netlink Code Diffstat: fs/xfs/libxfs/xfs_alloc.c | 4 +- fs/xfs/libxfs/xfs_alloc.h | 1 + fs/xfs/libxfs/xfs_alloc_btree.c | 35 +-- fs/xfs/libxfs/xfs_attr_leaf.c | 1 + fs/xfs/libxfs/xfs_attr_remote.c | 1 + fs/xfs/libxfs/xfs_bit.c | 6 +- fs/xfs/libxfs/xfs_bmap.c | 27 +- fs/xfs/libxfs/xfs_bmap_btree.c | 1 + fs/xfs/libxfs/xfs_btree.c | 58 ++++ fs/xfs/libxfs/xfs_btree.h | 3 + fs/xfs/libxfs/xfs_da_btree.c | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 1 + fs/xfs/libxfs/xfs_dir2_data.c | 2 + fs/xfs/libxfs/xfs_dir2_leaf.c | 2 + fs/xfs/libxfs/xfs_dir2_node.c | 1 + fs/xfs/libxfs/xfs_dquot_buf.c | 1 + fs/xfs/libxfs/xfs_format.h | 2 +- fs/xfs/libxfs/xfs_ialloc.c | 1 + fs/xfs/libxfs/xfs_ialloc_btree.c | 27 +- fs/xfs/libxfs/xfs_inode_buf.c | 2 + fs/xfs/libxfs/xfs_log_recover.h | 1 + fs/xfs/libxfs/xfs_sb.c | 2 + fs/xfs/libxfs/xfs_symlink_remote.c | 1 + fs/xfs/xfs_buf.c | 2 +- fs/xfs/xfs_buf.h | 1 + fs/xfs/xfs_error.c | 4 +- fs/xfs/xfs_file.c | 25 +- fs/xfs/xfs_iomap.c | 11 +- fs/xfs/xfs_log.c | 51 +++- fs/xfs/xfs_log_priv.h | 3 + fs/xfs/xfs_log_recover.c | 572 ++++++++++++++++++++++++++++++---= ---- fs/xfs/xfs_super.c | 2 +- fs/xfs/xfs_sysfs.c | 36 +++ fs/xfs/xfs_trans_dquot.c | 14 +- 34 files changed, 693 insertions(+), 209 deletions(-) --=20 Dave Chinner david@fromorbit.com --kVXhAStRUZ/+rrGn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJWiuXrAAoJEK3oKUf0dfodsOcP+gLWHk4z3AYYaKUwWLpfuCq9 HWPm68c+2WE3MvoX90UMUAxKhVPzZhWas4zbLEiKKzhWj+oANm28jnUqX+cRbvrQ EihIp1cuJVvtm5ZYv8FQNEup0PlfhzhNro9DrCU5k7X8qHfCNxUWnRSk97gbIUiQ cYymhYudBc4CUkyroShqX9obCYtJAlITxW3zZwGktXDFO+6mlRBIeZ38ok4A5+zb QszqglLqHUFx8XrKYB8CG03FFp2ShNZ7J0MgeYsCnT7yooD7taUPi2VrwdcR78d5 RDrzgw7f/eXypaxk2lUSjRR9Vwk6xEkBw8RoPHuWOMCzayMP235CduD5kFY7Fshd PZ5USCqYOu14vGjBtpZbDAYCjsUBEa7IPUxNMGScl8wfnYYCtwKDSSv6vvEEmcn0 +sl4lwv+Sd2+63f94quCxKDyyfCIlrBo+hkPtns2Q7cCRz3uBSfDyrc+wJ8DFtDO Sb+31kvrO1Rmy3Kd+5JKQyRZqZbbyuWWNPUl5QEoYhXw+Qxe619/oeZgk11JY0hJ nyp74AEpVAUvyzRIUTI93KKlgp4jANnRB+y36kR9yxsgfCEhpP6Few4nvVuGqASy oPCqVOXxqyb99pyJKDw7F+s0OHyvxxSMA9g807R1tMFhW8ZeF2KuTi1ntBEF8O5p KnKKjeI+FRl2DxMeAT7A =MYxv -----END PGP SIGNATURE----- --kVXhAStRUZ/+rrGn-- From darrick.wong@oracle.com Mon Jan 4 16:06:24 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 657A77F37 for ; Mon, 4 Jan 2016 16:06:24 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 5522230405F for ; Mon, 4 Jan 2016 14:06:21 -0800 (PST) X-ASG-Debug-ID: 1451945176-04cb6c20fe01100001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id zL0wQfbw5k3ObEpe (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 14:06:16 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04M6FWi022182 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 22:06:15 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u04M6FZK005879 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 22:06:15 GMT Received: from abhmp0013.oracle.com (abhmp0013.oracle.com [141.146.116.19]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u04M6EGu030154; Mon, 4 Jan 2016 22:06:15 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 14:06:14 -0800 Date: Mon, 4 Jan 2016 14:06:13 -0800 From: "Darrick J. Wong" To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 03/76] libxfs: refactor the btree size calculator code Message-ID: <20160104220613.GA28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 03/76] libxfs: refactor the btree size calculator code References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085642.12713.80467.stgit@birch.djwong.org> <20151220203928.GS26718@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151220203928.GS26718@dastard> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451945176 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25832 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Mon, Dec 21, 2015 at 07:39:28AM +1100, Dave Chinner wrote: > On Sat, Dec 19, 2015 at 12:56:42AM -0800, Darrick J. Wong wrote: > > Create a macro to generate btree height calculator functions. > > This will be used (much) later when we get to the refcount > > btree. > > > > Signed-off-by: Darrick J. Wong > .... > > +/* btree size calculator templates */ > > +#define DECLARE_BTREE_SIZE_FN(btree) \ > > +xfs_filblks_t xfs_##btree##_calc_btree_size(struct xfs_mount *mp, \ > > + unsigned long len); > > + > > +#define DEFINE_BTREE_SIZE_FN(btree, limitfield, maxlevels) \ > > +xfs_filblks_t \ > > +xfs_##btree##_calc_btree_size( \ > > + struct xfs_mount *mp, \ > > + unsigned long len) \ > > +{ \ > > + int level; \ > > + int maxrecs; \ > > + xfs_filblks_t rval; \ > > +\ > > + maxrecs = mp->limitfield[0]; \ > > + for (level = 0, rval = 0; level < maxlevels; level++) { \ > > + len += maxrecs - 1; \ > > + do_div(len, maxrecs); \ > > + rval += len; \ > > + if (len == 1) \ > > + return rval + maxlevels - \ > > + level - 1; \ > > + if (level == 0) \ > > + maxrecs = mp->limitfield[1]; \ > > + } \ > > + return rval; \ > > +} > > I really don't like using macros like this. The code becomes hard to > debug, hard to edit, the functions don't show up in grep/cscope, > etc. > > A helper function like this: > > xfs_filblks_t > xfs_btree_calc_size( > struct xfs_mount *mp, > int *limits, > int maxlevels, > unsigned long len) > { > int level; > int maxrecs; > xfs_filblks_t rval; > > maxrecs = limits[0]; > for (level = 0, rval = 0; level < maxlevels; level++) { > len += maxrecs - 1; > do_div(len, maxrecs); > rval += len; > if (len == 1) > return rval + maxlevels - level - 1; > if (level == 0) > maxrecs = limits[1]; > } > return rval; > } > > will work just as well when wrapped with the btree specific calling > function and that will have none of the problems using a macro to > build the functions has... Will change to use a helper function. --D > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 16:12:48 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 781D27F37 for ; Mon, 4 Jan 2016 16:12:48 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 57CD08F8040 for ; Mon, 4 Jan 2016 14:12:48 -0800 (PST) X-ASG-Debug-ID: 1451945564-04cbb0451501520001-NocioJ Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) by cuda.sgi.com with ESMTP id oLLsXIuDddf98yBy (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 14:12:45 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 156.151.31.81 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04MCLBb016666 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 22:12:21 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u04MCKLm017638 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 22:12:21 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by userv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04MCJXI005741; Mon, 4 Jan 2016 22:12:19 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 14:12:19 -0800 Date: Mon, 4 Jan 2016 14:12:18 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct Message-ID: <20160104221218.GB28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085655.12713.60317.stgit@birch.djwong.org> <20160103121525.GB28429@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103121525.GB28429@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Barracuda-Connect: userp1040.oracle.com[156.151.31.81] X-Barracuda-Start-Time: 1451945565 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25832 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 04:15:25AM -0800, Christoph Hellwig wrote: > On Sat, Dec 19, 2015 at 12:56:55AM -0800, Darrick J. Wong wrote: > > Because struct xfs_agfl is 36 bytes long and has a 64-bit integer > > inside it, gcc will quietly round the structure size up to the nearest > > 64 bits -- in this case, 40 bytes. This results in the XFS_AGFL_SIZE > > macro returning incorrect results for v5 filesystems on 64-bit > > machines (118 items instead of 119). As a result, a 32-bit xfs_repair > > will see garbage in AGFL item 119 and complain. > > > > Therefore, tell gcc not to pad the structure so that the AGFL size > > calculation is correct. > > Do you have a testcase for this? Not much aside from: 0. Build kernel/xfsprogs with RFCv4 patches on a 64bit machine. 1. Build kernel/xfsprogs with RFCv4 patches on a 32bit machine. 2. Format a XFS with reflink and rmap on a 64-bit machine, so that the AGFL size is maximized. 3. Mount FS and create a reflinked file. 4. Unmount and xfs_repair with the 32-bit build. I guess we could create a program that compares all the known sizeof(struct xfs_disk_object) values against known good values and stuff that into the xfsprogs build process. --D > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 17:24:01 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 799C47F37 for ; Mon, 4 Jan 2016 17:24:01 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 5BCC2304062 for ; Mon, 4 Jan 2016 15:23:58 -0800 (PST) X-ASG-Debug-ID: 1451949836-04cb6c20ff02670001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id JK5BCkHISYjlLfEp (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 15:23:56 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04NNW9m005206 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:23:33 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u04NNWC2019928 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:23:32 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04NNRa6005479; Mon, 4 Jan 2016 23:23:29 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 15:23:24 -0800 Date: Mon, 4 Jan 2016 15:23:18 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct Message-ID: <20160104232318.GC28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085655.12713.60317.stgit@birch.djwong.org> <20160103121525.GB28429@infradead.org> <20160104221218.GB28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104221218.GB28330@birch.djwong.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451949836 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25833 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Mon, Jan 04, 2016 at 02:12:18PM -0800, Darrick J. Wong wrote: > On Sun, Jan 03, 2016 at 04:15:25AM -0800, Christoph Hellwig wrote: > > On Sat, Dec 19, 2015 at 12:56:55AM -0800, Darrick J. Wong wrote: > > > Because struct xfs_agfl is 36 bytes long and has a 64-bit integer > > > inside it, gcc will quietly round the structure size up to the nearest > > > 64 bits -- in this case, 40 bytes. This results in the XFS_AGFL_SIZE > > > macro returning incorrect results for v5 filesystems on 64-bit > > > machines (118 items instead of 119). As a result, a 32-bit xfs_repair > > > will see garbage in AGFL item 119 and complain. > > > > > > Therefore, tell gcc not to pad the structure so that the AGFL size > > > calculation is correct. > > > > Do you have a testcase for this? > > Not much aside from: > > 0. Build kernel/xfsprogs with RFCv4 patches on a 64bit machine. > 1. Build kernel/xfsprogs with RFCv4 patches on a 32bit machine. > 2. Format a XFS with reflink and rmap on a 64-bit machine, so that the AGFL > size is maximized. > 3. Mount FS and create a reflinked file. > 4. Unmount and xfs_repair with the 32-bit build. > > I guess we could create a program that compares all the known sizeof(struct > xfs_disk_object) values against known good values and stuff that into the > xfsprogs build process. I created a patch that uses BUILD_BUG_ON to stop kbuild if the sizes of the on-disk data structures don't match known values. I've also ported it to xfsprogs, so we can check both. --D > > --D > > > > > _______________________________________________ > > xfs mailing list > > xfs@oss.sgi.com > > http://oss.sgi.com/mailman/listinfo/xfs > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 17:41:04 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id B822C7F37 for ; Mon, 4 Jan 2016 17:41:04 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 47EB8AC002 for ; Mon, 4 Jan 2016 15:41:01 -0800 (PST) X-ASG-Debug-ID: 1451950858-04cbb04515032f0001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id U1E0MFanjm06cVVI (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 15:40:59 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04NeZHV021105 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:40:35 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u04NeZS9003628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:40:35 GMT Received: from abhmp0001.oracle.com (abhmp0001.oracle.com [141.146.116.7]) by userv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04NeYXn012553; Mon, 4 Jan 2016 23:40:34 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 15:40:34 -0800 Date: Mon, 4 Jan 2016 15:40:33 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled Message-ID: <20160104234033.GD28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090008.12713.25755.stgit@birch.djwong.org> <20160103121728.GC28429@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103121728.GC28429@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451950859 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25834 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 04:17:28AM -0800, Christoph Hellwig wrote: > On Sat, Dec 19, 2015 at 01:00:08AM -0800, Darrick J. Wong wrote: > > Implement extent swapping when reverse-mapping is enabled. > > Can you just fold this into the previous patch? There seem to be a lot > of patches in the series that would benefit from merging, but this > is the one that really screams for it :) The only reason I'm keeping Dave's original rmap patches separate is because the last time I checked with Dave, he said that he'd made a few bugfixes to his patchset that hadn't yet been sent out, so I'd like to merge those fixes into the original patches before starting any combining work. It also might be more convenient (for him, anyway) to keep my rmapbt changes separate for review purposes. At this point, though, I've made so many changes that I'd like to pull in Dave's changes and combine them with mine into fewer patches. (AFAIK, Dave hasn't sent any further reverse-mapping patches since 3 June 2015.) --D > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From david@fromorbit.com Mon Jan 4 17:45:44 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id AADEA7F37 for ; Mon, 4 Jan 2016 17:45:44 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 988688F8033 for ; Mon, 4 Jan 2016 15:45:41 -0800 (PST) X-ASG-Debug-ID: 1451951137-04cb6c20fe02b70001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id JVBA7WEmGKDxF7PL for ; Mon, 04 Jan 2016 15:45:37 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2D1CQDmAotWPGu7LXleKAECgw8iMG2CYoN+gXmgMgIKDAEBAQEBAQaBDIpJhTmEBRgChW8EgSFNAQEBAQEBBwEBAQFBP4RiLzskNAUlAwctiC6gZ6ETCRmFdY0ADEEbgRsFlwaFQIgIgWWNHQKFWIhigjo4H4FxKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 10:15:36 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGEoh-000349-DL for xfs@oss.sgi.com; Tue, 05 Jan 2016 10:45:35 +1100 Date: Tue, 5 Jan 2016 10:45:35 +1100 From: Dave Chinner To: xfs@oss.sgi.com Subject: [ANNOUNCE] xfs-documentation: master branch updated to a04146a Message-ID: <20160104234535.GF19802@dastard> X-ASG-Orig-Subj: [ANNOUNCE] xfs-documentation: master branch updated to a04146a MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EY/WZ/HvNxOox07X" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451951137 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25833 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- --EY/WZ/HvNxOox07X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi folks, The xfs documentation repository at git://git.kernel.org/pub/scm/fs/xfs/xfs-documentation.git has just been updated. This is a major update to the XFS Filesystem Structure document which brings it up to date with the current on disk format. A big thank you must go to Darrick for doing this work, and to Christoph for reviewing it. -Dave. The new head of the guilt/working branch is commit: a04146a move directory chapter before xattr chapter New Commits: Darrick J. Wong (20): [38d3e30] fix asciidoc confusion w.r.t. include file newline at EOF [bda4165] update book subtitles and headings [728d229] restore the old rendering of structure field definitions [042c084] convert html entities to regular characters [f14c343] convert images to text [c93a293] delete images of hex dumps [0b39c06] update types [d77fa4e] make example section titles consistent [983e8a1] fix various errors and missing bits in the text [38dcd43] add missing structure field definitions [410e764] add missing xfs_db examples [36ce418] fix quoted text markings [8b4a922] introduce XFS at a high level [02be33c] document the xfs log format [13be785] document the operation of the realtime device and inodes [372d53f] add documentation of v5 fields [e95c161] add a chapter discussing v5 disk format [70cbe0d] document magic numbers and theoretical limits [83abac0] document the sparse inodes feature [a04146a] move directory chapter before xattr chapter Code Diffstat: .../allocation_groups.asciidoc | 808 ++++++++++++++++-= --- .../XFS_Filesystem_Structure/common_types.asciidoc | 41 +- .../XFS_Filesystem_Structure/data_extents.asciidoc | 210 ++++-- .../XFS_Filesystem_Structure/directories.asciidoc | 818 +++++++++++++++++= --- design/XFS_Filesystem_Structure/docinfo.xml | 43 +- .../extended_attributes.asciidoc | 466 ++++++++++-- .../XFS_Filesystem_Structure/images/code/33a.png | Bin 106484 -> 0 bytes .../XFS_Filesystem_Structure/images/code/33b.png | Bin 40229 -> 0 bytes design/XFS_Filesystem_Structure/images/code/40.png | Bin 94094 -> 0 bytes design/XFS_Filesystem_Structure/images/code/46.png | Bin 78484 -> 0 bytes design/XFS_Filesystem_Structure/images/code/57.png | Bin 17987 -> 0 bytes design/XFS_Filesystem_Structure/images/code/60.png | Bin 32907 -> 0 bytes design/XFS_Filesystem_Structure/images/code/61.png | Bin 54687 -> 0 bytes design/XFS_Filesystem_Structure/images/code/65.png | Bin 107515 -> 0 bytes design/XFS_Filesystem_Structure/images/code/66.png | Bin 69316 -> 0 bytes design/XFS_Filesystem_Structure/images/code/67.png | Bin 109738 -> 0 bytes design/XFS_Filesystem_Structure/images/code/71.png | Bin 62675 -> 0 bytes .../XFS_Filesystem_Structure/images/code/73-74.png | Bin 153732 -> 0 bytes design/XFS_Filesystem_Structure/images/code/74.png | Bin 62210 -> 0 bytes .../images/code/SVG/33a.svg | 185 ----- .../images/code/SVG/33b.svg | 158 ---- .../images/code/SVG/40.svg | 177 ----- .../images/code/SVG/46.svg | 197 ----- .../images/code/SVG/57.svg | 129 ---- .../images/code/SVG/60.svg | 182 ----- .../images/code/SVG/61.svg | 157 ---- .../images/code/SVG/65.svg | 233 ------ .../images/code/SVG/66.svg | 260 ------- .../images/code/SVG/67.svg | 232 ------ .../images/code/SVG/71.svg | 150 ---- .../images/code/SVG/73-74.svg | 364 --------- .../images/code/SVG/74.svg | 201 ----- .../internal_inodes.asciidoc | 123 +-- .../journaling_log.asciidoc | 836 +++++++++++++++++= +++- design/XFS_Filesystem_Structure/magic.asciidoc | 86 +++ .../metadata_integrity.asciidoc | 36 + .../XFS_Filesystem_Structure/ondisk_inode.asciidoc | 209 ++++-- design/XFS_Filesystem_Structure/overview.asciidoc | 38 + .../symbolic_links.asciidoc | 76 +- .../xfs_filesystem_structure.asciidoc | 51 +- 40 files changed, 3281 insertions(+), 3185 deletions(-) delete mode 100644 design/XFS_Filesystem_Structure/images/code/33a.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/33b.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/40.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/46.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/57.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/60.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/61.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/65.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/66.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/67.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/71.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/73-74.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/74.png delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/33a.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/33b.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/40.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/46.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/57.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/60.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/61.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/65.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/66.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/67.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/71.svg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/73-74.s= vg delete mode 100644 design/XFS_Filesystem_Structure/images/code/SVG/74.svg create mode 100644 design/XFS_Filesystem_Structure/magic.asciidoc create mode 100644 design/XFS_Filesystem_Structure/metadata_integrity.asci= idoc create mode 100644 design/XFS_Filesystem_Structure/overview.asciidoc --=20 Dave Chinner david@fromorbit.com --EY/WZ/HvNxOox07X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQIcBAEBAgAGBQJWiwQeAAoJEK3oKUf0dfodB/QQANIAE0fJ1FoSuRGXOInJDtqw kv/MYlzd5qKe2RJdaHgMGmOWY0AHKtEhdRbhLyf0jYpaWqWxxXL7pVRAB5nrsSk3 8jFET0iyrTr+u4CWiYYzv98ybMl/DACggeC7RqirOwLXM25vR4trywST+lSov+PL 9cSE15nzPt+hD/HB70BSDlFdT6++DQ7Mnbi7LOwXivM3mKrhjBUqj6LV3L15eZbU IhNr6iU6Cggrr8eZ6p0XnEMNV0zhMtOsd8qjY8AgEq0a6KEYMCClCwscZfrFXi+8 1ddfgIVyjzL2CUk7FHYBju4A1BelRPi6itztRm6HiSOdQ/FDL5c8Q89AoH6rLqTx sNWQqHdzIhECjZOZy9z6VjRppL2GbT2eDu0Xj2DsADbYRP6qV/I6cdB79hn9yAxj L4LAHjWh4NrKbMyfquJ+t+b2R7SDkBn8Bkjx/5cu1rOyHSaI6wuUVs7RwrusngLX kZ0wr8ewD4Gdlat1KSjC9UJTMD/U1TO3s8K8xItzxCVAWA9x44u/ZDNVRf0pxivo ax0iX62A5ulnyQAssB2v/V/eOCQYaSY+1qRob/oAyKLj+EdAxw+MGr6uKhEOfDlG tW5BOZJCeMvMkuWQjVXMjcKMHdG05eCKpUvWz2+qGAb3HNPYICJyEMx6WStSPVS0 AkRtjJDjd1sQgElsEsAP =JQrY -----END PGP SIGNATURE----- --EY/WZ/HvNxOox07X-- From david@fromorbit.com Mon Jan 4 17:52:46 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 685817F37 for ; Mon, 4 Jan 2016 17:52:46 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 24A26304066 for ; Mon, 4 Jan 2016 15:52:46 -0800 (PST) X-ASG-Debug-ID: 1451951556-04bdf06f71031e0001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id lykMh2Zm4qHUnwZD for ; Mon, 04 Jan 2016 15:52:37 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CpCQBuBYtWPGu7LXleKAECgw+BP4Jig36BeaBKAQEBAQEBBotVhTmEBYYJBAICgR1NAQEBAQEBBwEBAQFBP4Q1AQEEOhwjEAgDGAklDwUlAwcaE4guwVgBAQgCIRmFdYVHiTwFlwaNSI8ERI12gnIfHYFUKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 10:21:59 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGEuL-00034r-Oo; Tue, 05 Jan 2016 10:51:25 +1100 Date: Tue, 5 Jan 2016 10:51:25 +1100 From: Dave Chinner To: "Darrick J. Wong" Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct Message-ID: <20160104235125.GG19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 05/76] libxfs: pack the agfl header structure so XFS_AGFL_SIZE is correct References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219085655.12713.60317.stgit@birch.djwong.org> <20160103121525.GB28429@infradead.org> <20160104221218.GB28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104221218.GB28330@birch.djwong.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451951556 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25835 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 02:12:18PM -0800, Darrick J. Wong wrote: > On Sun, Jan 03, 2016 at 04:15:25AM -0800, Christoph Hellwig wrote: > > On Sat, Dec 19, 2015 at 12:56:55AM -0800, Darrick J. Wong wrote: > > > Because struct xfs_agfl is 36 bytes long and has a 64-bit integer > > > inside it, gcc will quietly round the structure size up to the nearest > > > 64 bits -- in this case, 40 bytes. This results in the XFS_AGFL_SIZE > > > macro returning incorrect results for v5 filesystems on 64-bit > > > machines (118 items instead of 119). As a result, a 32-bit xfs_repair > > > will see garbage in AGFL item 119 and complain. > > > > > > Therefore, tell gcc not to pad the structure so that the AGFL size > > > calculation is correct. > > > > Do you have a testcase for this? > > Not much aside from: > > 0. Build kernel/xfsprogs with RFCv4 patches on a 64bit machine. > 1. Build kernel/xfsprogs with RFCv4 patches on a 32bit machine. > 2. Format a XFS with reflink and rmap on a 64-bit machine, so that the AGFL > size is maximized. > 3. Mount FS and create a reflinked file. > 4. Unmount and xfs_repair with the 32-bit build. > > I guess we could create a program that compares all the known sizeof(struct > xfs_disk_object) values against known good values and stuff that into the > xfsprogs build process. There's an xfstest for that: xfs/122. It notruns on my systems since the big xfsprogs build/header rework, and I haven't found the time to work out what it needs to run again. Also, I don't think it covers the AGFL structure, because that it relatively new and the test doesn't check any of the v5 specific structures.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From darrick.wong@oracle.com Mon Jan 4 17:59:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id BFBAA7F37 for ; Mon, 4 Jan 2016 17:59:56 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id 9C1738F8037 for ; Mon, 4 Jan 2016 15:59:56 -0800 (PST) X-ASG-Debug-ID: 1451951994-04cbb04514037d0001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id aAhKF4HBFjPIcBZ6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 15:59:54 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u04NxrWN005521 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:59:54 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u04Nxr4p014617 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Mon, 4 Jan 2016 23:59:53 GMT Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u04NxqTI020122; Mon, 4 Jan 2016 23:59:53 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 15:59:52 -0800 Date: Mon, 4 Jan 2016 15:59:51 -0800 From: "Darrick J. Wong" To: Brian Foster Cc: xfs@oss.sgi.com Subject: Re: [RFCv4 00/76] xfs: add reverse-mapping, reflink, and dedupe support Message-ID: <20160104235951.GE28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [RFCv4 00/76] xfs: add reverse-mapping, reflink, and dedupe support References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151220140254.GA3618@laptop.bfoster> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151220140254.GA3618@laptop.bfoster> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451951994 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25835 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Dec 20, 2015 at 09:02:54AM -0500, Brian Foster wrote: > On Sat, Dec 19, 2015 at 12:56:23AM -0800, Darrick J. Wong wrote: > > Hi all, > > > ... > > Fixed since RFCv3: > > > > * The reflink and dedupe ioctls are being hoisted to the VFS, as > > provided in the first few patches. Patch 81 connects to this > > functionality. > > > > * Copy on write has been rewritten for v4. We now use the existing > > delayed allocation mechanism to coalesce writes together, deferring > > allocation until writeout time. This enables CoW to make better > > block placement decisions and significantly reduces overhead. > > CoW is still pretty slow, but not as slow as before. > > > > * Direct IO CoW has been implemented using the same mechanism as > > above, but modified to perform the allocation and remapping right > > then and there. Throughput is much higher than pushing data > > through the page cache CoW. (It's the same mechanism, but we're > > playing with chunks bigger than a single memory page.) > > > > * CoW ENOSPC works correctly now, except in the pathological case > > that the AG fills up and the rmap btree cannot expand. That will > > be addressed for v5. > > > > * fallocate will now unshare blocks to prevent future ENOSPC, as > > you'd expect. > > > > * refcount btree blocks are preallocated at mount time to prevent > > ENOSPC while trying to expand the tree. This also has the effect > > of grouping the btree blocks together, which can speed up CoW > > remapping. > > > > Can you elaborate on how these blocks are preallocated? E.g., is the > tree "preconstructed" in some sense? However that is done, is this the > anticipated solution or a temporary workaround..? > > Also, shouldn't the enospc condition be handled by the agfl? I take it > there is something going on here that renders that solution flawed, so > I'm just curious what it is. > > (Sorry if this is all explained elsewhere, but I haven't yet had a > chance to take a close enough look at this feature..). Reference count btree blocks aren't allocated from the AGFL; they're allocated from the free space in the same manner as the inobt, per a review comment from Dave a looong time ago. :) As such, we can get ourselves into the nasty situation where every block in the AG has been allocated to file data. If we then see a bunch of reference count changes that are scattered around the AG, the reference count btree has to expand to hold all the new records... but there isn't space, and the operation fails. Given that we know the maximum possible size of the refcount btree (it's 0.3% of the AG size with 4k blocks), I figured it was easy enough to avoid ENOSPC for reflink operations. I've temporarily fixed this by adding code that figures out how many blocks we need if the reference count btree has to have a unique record for every block in the AG and holding that many blocks until either they're allocated to the refcount btree or freed at umount time. Right now it's a temporary fix (if the FS crashes, the reserved blocks are lost) but it wouldn't be difficult for the FS to make a permanent reservation that's recorded on disk somehow. But that's involves writing things to disk + making xfsprogs understand the reservation; let's see what people say about the reserved pool idea at all. Does that make sense? :) --D > > Brian > > > Issues: > > > > * The extent swapping ioctl still allocates a bigger fixed-size > > transaction. That's most likely a stupid thing to do, so getting a > > better grip on how the journalling code works and auditing all the > > new transaction users will have to happen. Right now it mostly > > gets lucky. > > > > * EFI tracking for the allocated-but-not-yet-mapped blocks is > > nonexistant. A crash will leak them. > > > > * ENOSPC while expanding the rmap btree can crash the FS. For now we > > work around this problem by making the AGFL as big as possible, > > failing CoW attempts with ENOSPC if there aren't enough AGFL blocks > > available, and hoping that doesn't actually happen. > > > > If you're going to start using this mess, you probably ought to just > > pull from my github trees for kernel[1], xfsprogs[2], and xfstests[3]. > > There are also updates for xfs-docs[4] and man-pages[5]. > > > > The patches have been xfstested with x64, i386, and ppc64; while in > > general the tests run to completion, there are still periodic bugs > > that will be addressed by the next RFC. There's a persistent crash on > > arm64 and ppc64el that I haven't been able to triage. > > > > This is an extraordinary way to eat your data. Enjoy! > > Comments and questions are, as always, welcome. > > > > --D > > > > [1] https://github.com/djwong/linux/tree/for-dave > > [2] https://github.com/djwong/xfsprogs/tree/for-dave > > [3] https://github.com/djwong/xfstests/tree/for-dave > > [4] https://github.com/djwong/xfs-documentation/tree/for-dave > > [5] https://github.com/djwong/man-pages/commits/for-mtk > > > > _______________________________________________ > > xfs mailing list > > xfs@oss.sgi.com > > http://oss.sgi.com/mailman/listinfo/xfs > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 18:15:28 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 80B407F37 for ; Mon, 4 Jan 2016 18:15:28 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 5920A304067 for ; Mon, 4 Jan 2016 16:15:25 -0800 (PST) X-ASG-Debug-ID: 1451952917-04bdf06f7303bb0001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id Rulq6ulzqslJdJ2c (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 16:15:18 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u050Eq9f019242 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 00:14:52 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u050EpKP031232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 00:14:52 GMT Received: from abhmp0007.oracle.com (abhmp0007.oracle.com [141.146.116.13]) by userv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u050EpEO008330; Tue, 5 Jan 2016 00:14:51 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 16:14:51 -0800 Date: Mon, 4 Jan 2016 16:14:50 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes Message-ID: <20160105001450.GF28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091351.15558.51417.stgit@birch.djwong.org> <20160103134151.GB9144@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103134151.GB9144@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451952918 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25835 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 05:41:51AM -0800, Christoph Hellwig wrote: > Looks generally good, although I wonder the take on the realtime device > is a bit too positive given that we generally advise people not to use > it. Last time I tried formatting with a rt device (a month ago) the kernel crashed, as soon as I created an inode, so I guess we ought to warn people that this feature is severely bitrotted. What /is/ the official status of XFS RT? Most sane distros (looking right at you, Ubuntu) disable kernel support, so I guess: ------------------------------------------------------------------------ WARNING: The realtime feature is not well maintained. Use with caution. ------------------------------------------------------------------------ --D > > Reviewed-by: Christoph Hellwig > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 18:23:52 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 3A48E7F37 for ; Mon, 4 Jan 2016 18:23:52 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 17C96304066 for ; Mon, 4 Jan 2016 16:23:51 -0800 (PST) X-ASG-Debug-ID: 1451953428-04bdf06f7103e00001-NocioJ Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) by cuda.sgi.com with ESMTP id rpjiS8hdt9pOcHmC (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 16:23:49 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 156.151.31.81 Received: from userv0021.oracle.com (userv0021.oracle.com [156.151.31.71]) by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u050NKPf010752 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 00:23:21 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u050NKKQ028828 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 00:23:20 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u050NK9n007154; Tue, 5 Jan 2016 00:23:20 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 16:23:19 -0800 Date: Mon, 4 Jan 2016 16:23:18 -0800 From: "Darrick J. Wong" To: Dave Chinner Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [RFCv4 00/22] xfs-documentation: massive cleanups and add v5 format, reverse-mapping, reflink, and dedupe support Message-ID: <20160105002318.GG28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [RFCv4 00/22] xfs-documentation: massive cleanups and add v5 format, reverse-mapping, reflink, and dedupe support References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151220152936.GA20135@infradead.org> <20151220204657.GT26718@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151220204657.GT26718@dastard> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0021.oracle.com [156.151.31.71] X-Barracuda-Connect: userp1040.oracle.com[156.151.31.81] X-Barracuda-Start-Time: 1451953429 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC5_SA210e, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25837 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines 0.00 BSF_SC5_SA210e Custom Rule SA210e On Mon, Dec 21, 2015 at 07:46:57AM +1100, Dave Chinner wrote: > On Sun, Dec 20, 2015 at 07:29:36AM -0800, Christoph Hellwig wrote: > > On Sat, Dec 19, 2015 at 01:12:08AM -0800, Darrick J. Wong wrote: > > > This patch set does the following: > > > > I'll review the actual patches once I'll get more time, but maybe it's a > > good idea to split this series and get all the cleanups and fixes in > > ASAP and only land the new reverse mapping and reflink bits once the > > code hits mainline? > > I'll be cherrying picking all the cleanups and fixes out of all the > various patch series over the next couple days, so that's in effect > what will happen... If I did it right, the rmap/reflink content should be contained within their respective patches. Let me know if you find mistakes, though. --D > > Cheers, > > Dave. > -- > Dave Chinner > david@fromorbit.com > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From st101368@stud.uni-stuttgart.de Mon Jan 4 18:23:52 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: ** X-Spam-Status: No, score=2.1 required=5.0 tests=FREEMAIL_FORGED_REPLYTO, FSL_MISSP_REPLYTO,LOTS_OF_MONEY,MONEY_FRAUD_3 autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C168D7F37 for ; Mon, 4 Jan 2016 18:23:52 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id A2AC88F8037 for ; Mon, 4 Jan 2016 16:23:52 -0800 (PST) X-ASG-Debug-ID: 1451953422-04bdf06f7403e00001-NocioJ Received: from mx4.rus.uni-stuttgart.de (mx4.rus.uni-stuttgart.de [129.69.192.4]) by cuda.sgi.com with ESMTP id uORQrStdCnzcdXu3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 16:23:43 -0800 (PST) X-Barracuda-Envelope-From: st101368@stud.uni-stuttgart.de X-Barracuda-Apparent-Source-IP: 129.69.192.4 Received: from localhost (localhost [127.0.0.1]) by mx4.rus.uni-stuttgart.de (Postfix) with ESMTP id 1B0E63C7; Tue, 5 Jan 2016 01:23:42 +0100 (CET) X-Virus-Scanned: USTUTT mailrelay AV services at mx4.rus.uni-stuttgart.de Received: from mx4.rus.uni-stuttgart.de ([127.0.0.1]) by localhost (mx4.rus.uni-stuttgart.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ek-Jnw6WeMfM; Tue, 5 Jan 2016 01:23:40 +0100 (CET) Received: from DESKTOP-R0UORM3.rus.uni-stuttgart.de (s-8d3a1191.off.site.uni-stuttgart.de [141.58.17.145]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx4.rus.uni-stuttgart.de (Postfix) with ESMTPS; Tue, 5 Jan 2016 01:23:31 +0100 (CET) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Charity/Donation To: you X-ASG-Orig-Subj: Charity/Donation From: "Jeffrey Skoll" Date: Tue, 05 Jan 2016 02:23:32 +0200 Reply-To: jeffrey_skoll@gmx.com Message-Id: <20160105002342.1B0E63C7@mx4.rus.uni-stuttgart.de> X-Barracuda-Connect: mx4.rus.uni-stuttgart.de[129.69.192.4] X-Barracuda-Start-Time: 1451953423 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25837 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) Hi My name is Jeffrey Skoll, a philanthropist and the founder of one of the la= rgest private foundations in the world. I believe strongly in =E2=80=98givi= ng while living.=E2=80=99 I had one idea that never changed in my mind =E2= =80=94 that you should use your wealth to help people and I have decided to= secretly give USD2.498 Million to a randomly selected individual. On recei= pt of this email, you should count yourself as the individual. Your email a= ddress was chosen online while searching at random. Kindly get back to me a= t your earliest convenience, so I know your email address is valid. Visit the web page to know more about me: http://www.theglobeandmail.com/ne= ws/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article42= 09888/ or you can read an article of me on Wikipedia. Regards, Jeffrey Skoll. From darrick.wong@oracle.com Mon Jan 4 19:14:28 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id CB8897F37 for ; Mon, 4 Jan 2016 19:14:28 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id AA80130405F for ; Mon, 4 Jan 2016 17:14:25 -0800 (PST) X-ASG-Debug-ID: 1451956463-04cbb0451604c30001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id MVUL3gZjLMa27jzc (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 17:14:24 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u051DxCl005283 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:14:00 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u051DxZp032488 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:13:59 GMT Received: from abhmp0003.oracle.com (abhmp0003.oracle.com [141.146.116.9]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u051DxZx004254; Tue, 5 Jan 2016 01:13:59 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 17:13:59 -0800 Date: Mon, 4 Jan 2016 17:13:58 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com, Dave Chinner Subject: Re: [PATCH 57/76] xfs: allocate delayed extents in CoW fork Message-ID: <20160105011358.GH28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 57/76] xfs: allocate delayed extents in CoW fork References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090248.12713.32309.stgit@birch.djwong.org> <20160103122058.GE28429@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160103122058.GE28429@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451956464 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25838 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 04:20:58AM -0800, Christoph Hellwig wrote: > I really don't like the separate XFS_IOEND_COW flag. This should > be an ioend type just like XFS_IO_DELALLOC, XFS_IO_UNWRITTEN and > XFS_IO_OVERWRITE. We might need some work in the direct I/O > completions for that to really work, though. The COW completion is an ioend flag (instead of an ioend type) *because* of how I thought directio completions work. :) It looked to me like you could only have one ioend for a dio, and that there wasn't a clean way to make XFS have multiple ioends for a particular dio. Since the remap operation remaps everything it finds in the COW fork for a given part of a file and leaves the rest alone, it seemed natural to encode COW as a flag in the ioend, leave the ioend type as XFS_IO_OVERWRITE, and try to have as few ioends as possible. For dio writes I can COW write almost as fast as a regular overwrite. I further observed with the RFCv3 COW (which did have a separate ioend type for COW) that random writes had a tendency to create a /lot/ of ioends. If you let all these writes accumulate and then flushed them out with a single fsync, the scatteredness of the COW blocks caused the creation of a lot of ioends, which then caused a thundering herd when all those ioends had to be pushed separately through the workqueue. Changing the ioend handling to the way it is in RFCv4 greatly reduced the number of ioends that have to be processed, and dramatically improved write speeds. (I also fixed a bug where it would create ioends for each COW block in a streaming write; both fixes make v4's COW 5x faster on a ramdisk than v3's.) Did I overlook some subtlety with ioends? So far it's worked fine as-is. --D > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 19:23:21 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 08A297F37 for ; Mon, 4 Jan 2016 19:23:21 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 96B2DAC002 for ; Mon, 4 Jan 2016 17:23:17 -0800 (PST) X-ASG-Debug-ID: 1451956995-04bdf06f7405590001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id yQFFGXvo1FXwqs8J (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 17:23:15 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u051NC1J012947 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:23:12 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u051NBou001911 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:23:11 GMT Received: from abhmp0006.oracle.com (abhmp0006.oracle.com [141.146.116.12]) by aserv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u051NA2d007513; Tue, 5 Jan 2016 01:23:11 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 17:23:10 -0800 Date: Mon, 4 Jan 2016 17:23:09 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 1/3] xfs: pass inode instead of file to xfs_reflink_dirty_range Message-ID: <20160105012309.GI28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 1/3] xfs: pass inode instead of file to xfs_reflink_dirty_range References: <1451822873-12969-1-git-send-email-hch@lst.de> <1451822873-12969-2-git-send-email-hch@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1451822873-12969-2-git-send-email-hch@lst.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0022.oracle.com [141.146.126.234] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451956995 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25839 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 01:07:51PM +0100, Christoph Hellwig wrote: > We don't actually need a file for write_begin/end, those can be passed > as NULL for disk based file systems. This is important as we won't I wasn't aware that one could /do/ that. :) Looks good, though. I think I can clean the filp arguments out of the reflink code entirely, too. --D > even have a file pointer during a truncate operation, which gives a > guaranteed NULL pointer dererference with the current code. > > Signed-off-by: Christoph Hellwig > --- > fs/xfs/xfs_reflink.c | 18 +++++++++--------- > 1 file changed, 9 insertions(+), 9 deletions(-) > > diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c > index da4a715..4a3f0ee 100644 > --- a/fs/xfs/xfs_reflink.c > +++ b/fs/xfs/xfs_reflink.c > @@ -1316,7 +1316,7 @@ out_error: > */ > STATIC int > xfs_reflink_dirty_range( > - struct file *filp, > + struct inode *inode, > xfs_off_t pos, > xfs_off_t len) > { > @@ -1330,14 +1330,14 @@ xfs_reflink_dirty_range( > unsigned long bytes; /* Bytes to write to page */ > void *fsdata; > > - mapping = filp->f_mapping; > + mapping = inode->i_mapping; > a_ops = mapping->a_ops; > flags = AOP_FLAG_UNINTERRUPTIBLE; > do { > > offset = (pos & (PAGE_CACHE_SIZE - 1)); > bytes = min_t(unsigned long, len, PAGE_CACHE_SIZE) - offset; > - rpage = xfs_get_page(file_inode(filp), pos); > + rpage = xfs_get_page(inode, pos); > if (IS_ERR(rpage)) { > error = PTR_ERR(rpage); > break; > @@ -1346,24 +1346,24 @@ xfs_reflink_dirty_range( > break; > } > > - error = a_ops->write_begin(filp, mapping, pos, bytes, flags, > + error = a_ops->write_begin(NULL, mapping, pos, bytes, flags, > &page, &fsdata); > page_cache_release(rpage); > if (error < 0) > break; > > - trace_xfs_reflink_unshare_page(file_inode(filp), page, > + trace_xfs_reflink_unshare_page(inode, page, > pos, bytes); > > if (!PageUptodate(page)) { > - pr_err("%s: STALE? ino=%lu pos=%llu\n", > - __func__, filp->f_inode->i_ino, pos); > + pr_err("%s: STALE? ino=%llu pos=%llu\n", > + __func__, XFS_I(inode)->i_ino, pos); > WARN_ON(1); > } > if (mapping_writably_mapped(mapping)) > flush_dcache_page(page); > > - error = a_ops->write_end(filp, mapping, pos, bytes, bytes, > + error = a_ops->write_end(NULL, mapping, pos, bytes, bytes, > page, fsdata); > if (error < 0) > break; > @@ -1454,7 +1454,7 @@ xfs_reflink_dirty_extents( > flen = XFS_FSB_TO_B(mp, rlen); > if (fpos + flen > isize) > flen = isize - fpos; > - error = xfs_reflink_dirty_range(filp, fpos, flen); > + error = xfs_reflink_dirty_range(VFS_I(ip), fpos, flen); > xfs_ilock(ip, XFS_ILOCK_EXCL); > if (error) > goto out; > -- > 1.9.1 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 19:28:41 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2D3BB7F37 for ; Mon, 4 Jan 2016 19:28:41 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 0A020304062 for ; Mon, 4 Jan 2016 17:28:40 -0800 (PST) X-ASG-Debug-ID: 1451957318-04cb6c20ff047d0001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id DV3Kh6StlkHTxCT6 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 17:28:38 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u051SaAI017378 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:28:36 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u051SZ1f001174 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:28:36 GMT Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u051SZ8Y005129; Tue, 5 Jan 2016 01:28:35 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 17:28:33 -0800 Date: Mon, 4 Jan 2016 17:28:32 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 2/3] xfs: only end a COW operation in xfs_zero_remaining_bytes if we started one Message-ID: <20160105012832.GJ28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 2/3] xfs: only end a COW operation in xfs_zero_remaining_bytes if we started one References: <1451822873-12969-1-git-send-email-hch@lst.de> <1451822873-12969-3-git-send-email-hch@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1451822873-12969-3-git-send-email-hch@lst.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: userv0022.oracle.com [156.151.31.74] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451957318 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25838 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 01:07:52PM +0100, Christoph Hellwig wrote: > Without these we can see NULL pointer dereferences due to a non-existing > COW fork during xfstests runs. > > Signed-off-by: Christoph Hellwig > --- > fs/xfs/xfs_bmap_util.c | 12 +++++++----- > 1 file changed, 7 insertions(+), 5 deletions(-) > > diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c > index 7bee3c7..e777095 100644 > --- a/fs/xfs/xfs_bmap_util.c > +++ b/fs/xfs/xfs_bmap_util.c > @@ -1226,13 +1226,15 @@ xfs_zero_remaining_bytes( > > error = xfs_bwrite(bp); > xfs_buf_relse(bp); > - if (error) { > - err2 = xfs_reflink_end_cow_failed(ip, offset, > + if (should_fork) { > + if (error) { > + err2 = xfs_reflink_end_cow_failed(ip, offset, > + lastoffset - offset + 1); > + return error; > + } > + error = xfs_reflink_end_cow(ip, offset, > lastoffset - offset + 1); > - return error; > } Good catch! There's a buffer leak in the error case for xfs_map_cow_blocks() so I'll fold them both into the original patch. --D > - error = xfs_reflink_end_cow(ip, offset, > - lastoffset - offset + 1); > if (error) > return error; > } > -- > 1.9.1 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From darrick.wong@oracle.com Mon Jan 4 19:43:19 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C5B3B7F37 for ; Mon, 4 Jan 2016 19:43:19 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id A55638F8033 for ; Mon, 4 Jan 2016 17:43:16 -0800 (PST) X-ASG-Debug-ID: 1451958194-04cbb0451605350001-NocioJ Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by cuda.sgi.com with ESMTP id CqOpH4l23fcFHCL4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 17:43:14 -0800 (PST) X-Barracuda-Envelope-From: darrick.wong@oracle.com X-Barracuda-Apparent-Source-IP: 141.146.126.69 Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u051hCtu029107 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:43:12 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.13.8/8.13.8) with ESMTP id u051hBQh005813 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 5 Jan 2016 01:43:12 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0122.oracle.com (8.13.8/8.13.8) with ESMTP id u051hBd9012226; Tue, 5 Jan 2016 01:43:11 GMT Received: from localhost (/24.21.154.84) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 04 Jan 2016 17:43:11 -0800 Date: Mon, 4 Jan 2016 17:43:10 -0800 From: "Darrick J. Wong" To: Christoph Hellwig Cc: xfs@oss.sgi.com Subject: Re: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend Message-ID: <20160105014310.GK28330@birch.djwong.org> X-ASG-Orig-Subj: Re: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend References: <1451822873-12969-1-git-send-email-hch@lst.de> <1451822873-12969-4-git-send-email-hch@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1451822873-12969-4-git-send-email-hch@lst.de> User-Agent: Mutt/1.5.21 (2010-09-15) X-Source-IP: aserv0021.oracle.com [141.146.126.233] X-Barracuda-Connect: aserp1040.oracle.com[141.146.126.69] X-Barracuda-Start-Time: 1451958194 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, UNPARSEABLE_RELAY X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25839 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 UNPARSEABLE_RELAY Informational: message has unparseable relay lines On Sun, Jan 03, 2016 at 01:07:53PM +0100, Christoph Hellwig wrote: > Otherwise we leak COW allocations done earlier in writepage. This > can be reproduced fairly easily when we hit the non-blocking writeback > EAGAIN case. > > Signed-off-by: Christoph Hellwig > --- > fs/xfs/xfs_aops.c | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c > index 185415a..9c69dc3 100644 > --- a/fs/xfs/xfs_aops.c > +++ b/fs/xfs/xfs_aops.c > @@ -588,6 +588,7 @@ xfs_cancel_ioend( > { > xfs_ioend_t *next; > struct buffer_head *bh, *next_bh; > + int error; > > do { > next = ioend->io_list; > @@ -605,6 +606,12 @@ xfs_cancel_ioend( > unlock_buffer(bh); > } while ((bh = next_bh) != NULL); > > + if (ioend->io_flags & XFS_IOEND_COW) { > + error = xfs_reflink_end_cow_failed( > + XFS_I(ioend->io_inode), > + ioend->io_offset, ioend->io_size); > + WARN_ON_ONCE(error); > + } Hmm. This might be the cause of the occasional complaints I've been seeing where allocated blocks remain in the COW fork when the inode is being cleared out. That said, the xfs_reflink_end_cow_failed() is apparently missing a xfs_bunmapi_cow() to actually clean out the COW fork. Good catch, in any case. Thank you for the testing and patches! :) --D > mempool_free(ioend, xfs_ioend_pool); > } while ((ioend = next) != NULL); > } > -- > 1.9.1 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From david@fromorbit.com Mon Jan 4 20:36:14 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id C50207F37 for ; Mon, 4 Jan 2016 20:36:14 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id B3C50304053 for ; Mon, 4 Jan 2016 18:36:11 -0800 (PST) X-ASG-Debug-ID: 1451961365-04cb6c20ff05710001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id HpiiT7brBTguEz9r for ; Mon, 04 Jan 2016 18:36:06 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DbCQDvKotWPGu7LXleKAECgw+BP4Jig36BeaBNAQEBBotYhTmEBYYJBAICgRxNAQEBAQEBBwEBAQFBP4Q1AQEEOhwjEAgDGAklDwUlAwcaE4guwUABAQgCIRmFdYVHiTwFh16PKI1JjwOOPoMugVQqNIUQAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 13:06:04 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGHTf-0003KF-N2; Tue, 05 Jan 2016 13:36:03 +1100 Date: Tue, 5 Jan 2016 13:36:03 +1100 From: Dave Chinner To: "Darrick J. Wong" Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes Message-ID: <20160105023603.GH19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 15/22] xfsdocs: document the operation of the realtime device and inodes References: <20151219091208.15558.65013.stgit@birch.djwong.org> <20151219091351.15558.51417.stgit@birch.djwong.org> <20160103134151.GB9144@infradead.org> <20160105001450.GF28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160105001450.GF28330@birch.djwong.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451961365 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25839 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 04:14:50PM -0800, Darrick J. Wong wrote: > On Sun, Jan 03, 2016 at 05:41:51AM -0800, Christoph Hellwig wrote: > > Looks generally good, although I wonder the take on the realtime device > > is a bit too positive given that we generally advise people not to use > > it. > > Last time I tried formatting with a rt device (a month ago) the kernel crashed, > as soon as I created an inode, so I guess we ought to warn people that this > feature is severely bitrotted. > > What /is/ the official status of XFS RT? Most sane distros (looking right at > you, Ubuntu) disable kernel support, so I guess: > > ------------------------------------------------------------------------ > WARNING: The realtime feature is not well maintained. Use with caution. > ------------------------------------------------------------------------ It gets quite a long way through xfstests here before there's an assert failure in the rt summary locking code. I'm planning on fixing that when I get a chance and then running xfstests on it regularly. I also have patches for xfstests that allow the config section code to use external devices, which makes it somewhat easier to have a "forget+test" config.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Jan 4 20:41:34 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CC9F97F37 for ; Mon, 4 Jan 2016 20:41:34 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 76B2BAC006 for ; Mon, 4 Jan 2016 18:41:34 -0800 (PST) X-ASG-Debug-ID: 1451961691-04bdf06f7307360001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id 3KRWu95HnivmK0v4 for ; Mon, 04 Jan 2016 18:41:32 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DbCQAkLItWPGu7LXleKAECgw+BP4Jig36BeaBOAQEBBotYhTmEBYYJBAICgRxNAQEBAQEBBwEBAQFBP4Q1AQEEOhwjEAgDGAklDwUlAwcaE4guwUEBLBmFdYVHiTwFlwaNSYFljR5EigeDc4J1HIFxKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 13:11:31 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGHYw-0003Kq-D1; Tue, 05 Jan 2016 13:41:30 +1100 Date: Tue, 5 Jan 2016 13:41:30 +1100 From: Dave Chinner To: "Darrick J. Wong" Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled Message-ID: <20160105024130.GI19802@dastard> X-ASG-Orig-Subj: Re: [PATCH 34/76] xfs: implement XFS_IOC_SWAPEXT when rmap btree is enabled References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151219090008.12713.25755.stgit@birch.djwong.org> <20160103121728.GC28429@infradead.org> <20160104234033.GD28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104234033.GD28330@birch.djwong.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451961691 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25840 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 03:40:33PM -0800, Darrick J. Wong wrote: > On Sun, Jan 03, 2016 at 04:17:28AM -0800, Christoph Hellwig wrote: > > On Sat, Dec 19, 2015 at 01:00:08AM -0800, Darrick J. Wong wrote: > > > Implement extent swapping when reverse-mapping is enabled. > > > > Can you just fold this into the previous patch? There seem to be a lot > > of patches in the series that would benefit from merging, but this > > is the one that really screams for it :) > > The only reason I'm keeping Dave's original rmap patches separate is because > the last time I checked with Dave, he said that he'd made a few bugfixes to his > patchset that hadn't yet been sent out, so I'd like to merge those fixes into > the original patches before starting any combining work. I haven't had a chance to sort any of those out fully, so it's probably best if you combine all your patches to minimise the separation and I'll send delta patches (if still relevant) as I work through it all in the next couple of weeks... Cheers, Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Jan 4 21:51:57 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 106A27F37 for ; Mon, 4 Jan 2016 21:51:57 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id D8A5730405F for ; Mon, 4 Jan 2016 19:51:53 -0800 (PST) X-ASG-Debug-ID: 1451965909-04bdf06f7209430001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id dva69V7iJ7FNVUbB for ; Mon, 04 Jan 2016 19:51:50 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DfCQCLPItWPGu7LXleKAECgw+BP4Jig36BeaBOAQEBBoFliXOFOYQFhgkEAgKBHU0BAQEBAQEHAQEBAUE/hDQBAQEDAScTHCMFCwgDGAklDwUlAwcaExmIDgfBRgEBCAIhGYV1hUeJPAWNOYlNjUmPA44+gnUcgXEqNIUQAQEB Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 14:21:48 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGIex-0003RM-Nb; Tue, 05 Jan 2016 14:51:47 +1100 Date: Tue, 5 Jan 2016 14:51:47 +1100 From: Dave Chinner To: Dan Williams Cc: xfs@oss.sgi.com, linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, Jens Axboe , Jan Kara , linux-fsdevel@vger.kernel.org, Matthew Wilcox , Ross Zwisler Subject: Re: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life Message-ID: <20160105035147.GJ19802@dastard> X-ASG-Orig-Subj: Re: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> <20160104182005.24118.50361.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104182005.24118.50361.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451965909 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25842 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 10:20:05AM -0800, Dan Williams wrote: > Historically we have waited for filesystem specific heuristics to > attempt to guess when a block device is gone. Sometimes this works, but > in other cases the system can hang waiting for the fs to trigger its > shutdown protocol. > > The initial motivation for this investigation was to prevent DAX > mappings (direct mmap access to persistent memory) from leaking past the > lifetime of the hosting block device. However, Dave points out that > these shutdown operations are needed in other scenarios. Quoting Dave: > > For example, if we detect a free space corruption during allocation, > it is not safe to trust *any active mapping* because we can't trust > that we having handed out the same block to multiple owners. Hence > on such a filesystem shutdown, we have to prevent any new DAX > mapping from occurring and invalidate all existing mappings as we > cannot allow userspace to modify any data or metadata until we've > resolved the corruption situation. > > The current block device shutdown sequence of del_gendisk + > blk_cleanup_queue is problematic. We want to tell the fs after > blk_cleanup_queue that there is no possibility of recovery, but by that > time we have deleted partitions and lost the ability to find all the > super-blocks on a block device. > > Introduce del_gendisk_queue to trigger ->quiesce() and ->bdi_gone() I don't see anything that introduces a ->quiesce() method. > notifications to all the filesystems hosted on the disk. Where > ->quiesce() are 'shutdown' operations while the bdev may still be alive, So you are saying "quiesce == invalidation", which is in conflict with what we typically think quiesce means. i.e. "Quiesce" is what we do when doing orderly writeback of all outstanding dirty objects in a filesystem - what we do during freeze, remount-ro, and unmount. e.g. See the functions xfs_log_quiesce() and xfs_attr_quiesce() > and ->bdi_gone() is a set of actions to take after the backing device > is known to be permanently dead. In which case, bdi_gone == shutdown. Operation methods should be named after what they do, not what their calling context is. i.e. these are "invalidate" and "shutdown" superblock operations, not "quiesce" and "bdi_gone". > generic_quiesce_super and generic_bdi_gone, are the default operations > when a filesystem does not implement ->quiesce(), ->bdi_gone(). They > invalidate inodes and unmap DAX-inodes respectively. For now only > ->bdi_gone() has an associated super operation as xfs will implement > ->bdi_gone() in a later patch. I don't quite understand what the point of factoring __invalidate_device() like this is - it's not used by anyone, so it seems completely unnecessary to me. And really, that points out that there are multiple changes in this patch set that should be done separately. The rework of del_gendisk() into del_gendisk_start/del_gendisk_end should be the first patch. The del_gendisk/blk_cleanup_queue to blk_cleanup_queue() combining should be the second part, as it builds on the factoring of del_gendisk(). Then, if we really need to keep it, the factoring to introduce generic_quiesce_super. And finally, the patch that allows the shutdown callout can be introduced. [snip] > > +static void generic_bdi_gone(struct super_block *sb) > +{ > + struct inode *inode, *_inode = NULL; > + > + spin_lock(&sb->s_inode_list_lock); > + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { > + spin_lock(&inode->i_lock); > + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) > + || !IS_DAX(inode)) { > + spin_unlock(&inode->i_lock); > + continue; > + } > + __iget(inode); > + spin_unlock(&inode->i_lock); > + spin_unlock(&sb->s_inode_list_lock); > + > + unmap_mapping_range(inode->i_mapping, 0, 0, 1); > + iput(_inode); > + _inode = inode; > + cond_resched(); > + > + spin_lock(&sb->s_inode_list_lock); > + } > + spin_unlock(&sb->s_inode_list_lock); > + iput(_inode); > +} This belongs in fs/inode.c, right next to invalidate_inodes(), and with a name that describes what it does, not the context in which it is called. e.g. unmap_dax_inodes(). > +void shutdown_partition(struct gendisk *disk, int partno) > +{ > + struct block_device *bdev = bdget_disk(disk, partno); > + struct super_block *sb = get_super(bdev); > + > + if (!bdev) > + return; Null pointer deref. Certainly a landmine waiting for someone to tread on. > + if (!sb) { > + bdput(bdev); > + return; > + } > + > + if (sb->s_op->bdi_gone) > + sb->s_op->bdi_gone(sb); > + else > + generic_bdi_gone(sb); if (sb->s_op->shutdown) sb->s_op->shutdown(sb); else unmap_dax_inodes(sb); name things correctly, and the code documents itself. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Jan 4 22:04:31 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 0349F7F37 for ; Mon, 4 Jan 2016 22:04:31 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay1.corp.sgi.com (Postfix) with ESMTP id E6AA68F8035 for ; Mon, 4 Jan 2016 20:04:27 -0800 (PST) X-ASG-Debug-ID: 1451966664-04cbb0451608a40001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id 4F4sOwIbdm0ZxWqw for ; Mon, 04 Jan 2016 20:04:25 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BDCgAEQItWPGu7LXleKAECgw+BP4Jig36BeaBOAQEBBoFliXOFOYQFhgkCAgEBAoEcTQEBAQEBAQcBAQEBQT+ENAEBAQMBJxMcIQIFCwgDGAklDwUlAwcaE4gnB8E5AQEBBwIBIBmFdYVHhCSFGAEEkwqDfI1JjwOOPoJ1HIFxKjQBg0SBSwEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 14:33:35 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGIqM-0003Sn-80; Tue, 05 Jan 2016 15:03:34 +1100 Date: Tue, 5 Jan 2016 15:03:34 +1100 From: Dave Chinner To: Dan Williams Cc: xfs@oss.sgi.com, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvdimm@lists.01.org Subject: Re: [resend PATCH 2/3] xfs: handle shutdown notifications Message-ID: <20160105040334.GK19802@dastard> X-ASG-Orig-Subj: Re: [resend PATCH 2/3] xfs: handle shutdown notifications References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> <20160104182011.24118.30446.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104182011.24118.30446.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451966664 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25842 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 10:20:11AM -0800, Dan Williams wrote: > Force a filesystem shutdown when the backing device is known to be dead. > I.e. blk_queue_enter() permanently returns -ENODEV. > > Cc: xfs@oss.sgi.com > Suggested-by: Dave Chinner > Signed-off-by: Dan Williams > --- > fs/block_dev.c | 3 ++- > fs/xfs/xfs_super.c | 9 +++++++++ > include/linux/fs.h | 2 ++ > 3 files changed, 13 insertions(+), 1 deletion(-) > > diff --git a/fs/block_dev.c b/fs/block_dev.c > index 739e43a37e64..7d6c66148948 100644 > --- a/fs/block_dev.c > +++ b/fs/block_dev.c > @@ -1806,7 +1806,7 @@ int __invalidate_device(struct block_device *bdev, bool kill_dirty) > } > EXPORT_SYMBOL(__invalidate_device); > > -static void generic_bdi_gone(struct super_block *sb) > +void generic_bdi_gone(struct super_block *sb) > { > struct inode *inode, *_inode = NULL; > > @@ -1832,6 +1832,7 @@ static void generic_bdi_gone(struct super_block *sb) > spin_unlock(&sb->s_inode_list_lock); > iput(_inode); > } > +EXPORT_SYMBOL(generic_bdi_gone); That should be in the previous patch. > void shutdown_partition(struct gendisk *disk, int partno) > { > diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c > index 36bd8825bfb0..63c36508e9db 100644 > --- a/fs/xfs/xfs_super.c > +++ b/fs/xfs/xfs_super.c > @@ -1618,6 +1618,14 @@ xfs_fs_free_cached_objects( > return xfs_reclaim_inodes_nr(XFS_M(sb), sc->nr_to_scan); > } > > +static void > +xfs_fs_bdi_gone( xfs_fs_shutdown > + struct super_block *sb) > +{ > + xfs_force_shutdown(XFS_M(sb), SHUTDOWN_DEVICE_REQ); > + generic_bdi_gone(sb); > +} > + This is wrong. we have to unmap the DAX inodes during *every* shutdown that XFS executes. Hence it needs to be done inside xfs_do_force_shutdown(), not in addition to the shutdown when the block device is pulled. i.e. something like this in xfs_do_force_shutdown(): */ if (xfs_log_force_umount(mp, logerror)) return; + + /* + * If DAX is in use, we have to unmap all direct access + * virtual mappings to ensure nothing more gets written + * directly from userspace. This will force them to refault + * and that will result in them detecting the shutdown + * condition and hence will fail appropriately. + */ + unmap_dax_inodes(mp->m_super); if (flags & SHUTDOWN_CORRUPT_INCORE) { xfs_alert_tag(mp, XFS_PTAG_SHUTDOWN_CORRUPT > static const struct super_operations xfs_super_operations = { > .alloc_inode = xfs_fs_alloc_inode, > .destroy_inode = xfs_fs_destroy_inode, > @@ -1632,6 +1640,7 @@ static const struct super_operations xfs_super_operations = { > .show_options = xfs_fs_show_options, > .nr_cached_objects = xfs_fs_nr_cached_objects, > .free_cached_objects = xfs_fs_free_cached_objects, > + .bdi_gone = xfs_fs_bdi_gone, > }; > > static struct file_system_type xfs_fs_type = { > diff --git a/include/linux/fs.h b/include/linux/fs.h > index 0e201ed38045..b1e8e049e4b8 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -2265,6 +2265,7 @@ extern struct super_block *freeze_bdev(struct block_device *); > extern void emergency_thaw_all(void); > extern int thaw_bdev(struct block_device *bdev, struct super_block *sb); > extern int fsync_bdev(struct block_device *); > +extern void generic_bdi_gone(struct super_block *sb); previous patch. -Dave. -- Dave Chinner david@fromorbit.com From david@fromorbit.com Mon Jan 4 22:24:06 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 3C1AD7F37 for ; Mon, 4 Jan 2016 22:24:06 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id F16EA8F8033 for ; Mon, 4 Jan 2016 20:24:05 -0800 (PST) X-ASG-Debug-ID: 1451967841-04bdf06f7209f70001-NocioJ Received: from ipmail04.adl6.internode.on.net (ipmail04.adl6.internode.on.net [150.101.137.141]) by cuda.sgi.com with ESMTP id L2HbeEDeEjBVdlAG for ; Mon, 04 Jan 2016 20:24:02 -0800 (PST) X-Barracuda-Envelope-From: david@fromorbit.com X-Barracuda-Apparent-Source-IP: 150.101.137.141 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BBCgC/RItWPGu7LXleKAECgw+BP4Jig36BeaBOAQEBBotYhTmEBYYJAgIBAQKBHE0BAQEBAQEHAQEBAUE/hDQBAQEDAScTHCMFCwgDGAklDwUlAwcaE4gnB8E8AQEIAgEgGYV1hUeJPAWHW4cShAeEEo1Jik2ENkSNeoJ1HIFxKjSFEAEBAQ Received: from ppp121-45-187-107.lns20.syd7.internode.on.net (HELO dastard) ([121.45.187.107]) by ipmail04.adl6.internode.on.net with ESMTP; 05 Jan 2016 14:53:47 +1030 Received: from dave by dastard with local (Exim 4.80) (envelope-from ) id 1aGJ9u-0003UH-Lg; Tue, 05 Jan 2016 15:23:46 +1100 Date: Tue, 5 Jan 2016 15:23:46 +1100 From: Dave Chinner To: Dan Williams Cc: xfs@oss.sgi.com, linux-block@vger.kernel.org, linux-nvdimm@lists.01.org, Jens Axboe , linux-fsdevel@vger.kernel.org, Jan Kara , Tejun Heo Subject: Re: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty Message-ID: <20160105042346.GL19802@dastard> X-ASG-Orig-Subj: Re: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> <20160104182016.24118.33718.stgit@dwillia2-desk3.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104182016.24118.33718.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Barracuda-Connect: ipmail04.adl6.internode.on.net[150.101.137.141] X-Barracuda-Start-Time: 1451967841 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25843 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Mon, Jan 04, 2016 at 10:20:16AM -0800, Dan Williams wrote: > This warning was added as a debugging aid way back in commit > 500b067c5e6c "writeback: check for registered bdi in flusher add and > inode dirty" when we were switching over to per-bdi writeback. > > Once the block device has been torn down it's no longer useful to > complain about unregistered bdi's. Clear the writeback capability under > the the wb->list_lock(), so that __mark_inode_dirty has no opportunity > to race bdi_unregister() to this WARN() condition. > > Alternatively we could just delete the warning... The warning is correct - the filesytem is trying to mark an inode dirty on a device that can't do writeback anymore. Seems to me like it is functioning as it should. > Found this while testing block device remove from underneath an active > fs triggering traces like: > > WARNING: CPU: 6 PID: 2129 at fs/fs-writeback.c:2065 __mark_inode_dirty+0x261/0x350() > bdi-block not registered > [..] > Call Trace: > [] dump_stack+0x44/0x62 > [] warn_slowpath_common+0x82/0xc0 > [] warn_slowpath_fmt+0x5c/0x80 > [] __mark_inode_dirty+0x261/0x350 > [] generic_update_time+0x79/0xd0 > [] file_update_time+0xbd/0x110 > [] ext4_dax_fault+0x68/0x110 > [] __do_fault+0x4e/0xf0 This seems like the problem to me - you haven't implemented a shutdown hook for ext4, and so it continues to allow page faults to make progress after the device has been removed. The DAX fault should have been failed before the filesystem gets to the point of marking the inode dirty.... > + /* tell __mark_inode_dirty that writeback is no longer possible */ > + spin_lock(&wb->list_lock); > + wb->bdi->capabilities |= BDI_CAP_NO_WRITEBACK; > + spin_unlock(&wb->list_lock); > + > spin_unlock_bh(&wb->work_lock); Is that lock ordering safe? i.e. it's inside a section using bh-safe locking, which tends to imply that it can run from interrupt contexts. Can we get something like spin_lock(&wb->list_lock); ..... ..... wb_shutdown spin_lock_bh(&wb->work_lock); spin_lock(&wb->list_lock); Cheers, Dave. -- Dave Chinner david@fromorbit.com From dan.j.williams@intel.com Mon Jan 4 22:25:25 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 041597F37 for ; Mon, 4 Jan 2016 22:25:25 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id A3B60AC003 for ; Mon, 4 Jan 2016 20:25:21 -0800 (PST) X-ASG-Debug-ID: 1451967917-04bdf06f7409fb0001-NocioJ Received: from mail-yk0-f179.google.com (mail-yk0-f179.google.com [209.85.160.179]) by cuda.sgi.com with ESMTP id QEdUagY0tj1HkEaA (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 04 Jan 2016 20:25:17 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 209.85.160.179 Received: by mail-yk0-f179.google.com with SMTP id a85so192834783ykb.1 for ; Mon, 04 Jan 2016 20:25:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=7p/HBZvEy8CWod6Q2IraXMfZfQTDiWDNp59v2ODJZ1s=; b=15sc3HL2fl6B/ukg/Wfxm6GByNcz5LInbqS8+a3FbenWv+XJNLcb1peEURntOfX05B zf9P3dAK3ewmn498APuv0sdnZ7qgH5qL7xifWvfgXbqjzPDjhU+88ioxA7uYhe4ETt/A VY48+AwrLydA9OfrGQLcBFUexm/bFgYUyUR9yFWy7s88QV1kD/ifPum9kon/DAdj7kJl BO6In7BlX1NDfrJer2PLxrWMWDMdYB10FO0yWltkVzw8qD38QmvD6qNiS4vjEJiaitBR kkHBtNvrBR/XK5Czcx2ghRgym4mQK4TmTdfPiSbaNjn5Qh+6oKH26jCdpAEos7+ylKnR 0oyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=7p/HBZvEy8CWod6Q2IraXMfZfQTDiWDNp59v2ODJZ1s=; b=aZz69ml0sJt0sdgc5QIzA3t/zCMJZXa6WO0WYan038ouBG+xaeJ5bfkSq0UreB75Ma 7bTOOmw4zeQl5fLmOobOiim4yT5RrQ8QQwEZ4R6tLmHi22BlSm49xAwROi5Bli3ClF0t Q+cyeCCbfgesxD7OfImp9Ti/X4NLUM1VVMDepaboD1OxSeEPGqvshtI/OBoUkV/n2cym s+vxlVF48ZI47sQEA6WpjGQK2zmoQa+kLVw50M3i0SwXmJVAIxRqadwgcgQ7TXnyCbZs 14S5se1fU/I7LgStbzfQD5sR6mW3fhveCromfbavmBtGgJLtMxs1u0nIdpbK1+MjfGUZ 5NpQ== X-Gm-Message-State: ALoCoQlASb5z/OZM90XMpOq14wTWSraeCmktJEtZvlKn76K3yfOvg9piIfi7CGZF9rWUDWvY99iaQIpy0cWZswkPZFq8q1KM0BaN03SBHWsoRNqLNnqMfP8= MIME-Version: 1.0 X-Received: by 10.13.228.134 with SMTP id n128mr54371826ywe.287.1451967916702; Mon, 04 Jan 2016 20:25:16 -0800 (PST) Received: by 10.37.202.20 with HTTP; Mon, 4 Jan 2016 20:25:16 -0800 (PST) In-Reply-To: <20160105035147.GJ19802@dastard> References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> <20160104182005.24118.50361.stgit@dwillia2-desk3.amr.corp.intel.com> <20160105035147.GJ19802@dastard> Date: Mon, 4 Jan 2016 20:25:16 -0800 Message-ID: Subject: Re: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life From: Dan Williams X-ASG-Orig-Subj: Re: [resend PATCH 1/3] block, fs: reliably communicate bdev end-of-life To: Dave Chinner Cc: XFS Developers , linux-block@vger.kernel.org, "linux-nvdimm@lists.01.org" , Jens Axboe , Jan Kara , linux-fsdevel , Matthew Wilcox , Ross Zwisler Content-Type: text/plain; charset=UTF-8 X-Barracuda-Connect: mail-yk0-f179.google.com[209.85.160.179] X-Barracuda-Start-Time: 1451967917 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25843 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature On Mon, Jan 4, 2016 at 7:51 PM, Dave Chinner wrote: > On Mon, Jan 04, 2016 at 10:20:05AM -0800, Dan Williams wrote: >> Historically we have waited for filesystem specific heuristics to >> attempt to guess when a block device is gone. Sometimes this works, but >> in other cases the system can hang waiting for the fs to trigger its >> shutdown protocol. >> >> The initial motivation for this investigation was to prevent DAX >> mappings (direct mmap access to persistent memory) from leaking past the >> lifetime of the hosting block device. However, Dave points out that >> these shutdown operations are needed in other scenarios. Quoting Dave: >> >> For example, if we detect a free space corruption during allocation, >> it is not safe to trust *any active mapping* because we can't trust >> that we having handed out the same block to multiple owners. Hence >> on such a filesystem shutdown, we have to prevent any new DAX >> mapping from occurring and invalidate all existing mappings as we >> cannot allow userspace to modify any data or metadata until we've >> resolved the corruption situation. >> >> The current block device shutdown sequence of del_gendisk + >> blk_cleanup_queue is problematic. We want to tell the fs after >> blk_cleanup_queue that there is no possibility of recovery, but by that >> time we have deleted partitions and lost the ability to find all the >> super-blocks on a block device. >> >> Introduce del_gendisk_queue to trigger ->quiesce() and ->bdi_gone() > > I don't see anything that introduces a ->quiesce() method. > Right, we only have generic_quiesce_super() as no fs had a need to do anything but the generic actions. >> notifications to all the filesystems hosted on the disk. Where >> ->quiesce() are 'shutdown' operations while the bdev may still be alive, > > So you are saying "quiesce == invalidation", which is in conflict > with what we typically think quiesce means. i.e. "Quiesce" is what > we do when doing orderly writeback of all outstanding dirty objects > in a filesystem - what we do during freeze, remount-ro, and unmount. > e.g. See the functions xfs_log_quiesce() and xfs_attr_quiesce() Fair enough, I should have called this one invalidate_super. > >> and ->bdi_gone() is a set of actions to take after the backing device >> is known to be permanently dead. > > In which case, bdi_gone == shutdown. > True, and following this logic I think the existing generic_shutdown_super() should be renamed generic_kill_super() to match the fs actions, but see below... > Operation methods should be named after what they do, not what their > calling context is. i.e. these are "invalidate" and "shutdown" > superblock operations, not "quiesce" and "bdi_gone". I was running out of colors to paint the bike shed given generic_shutdown_super() was already in use. Also, since generic_shutdown_super() has had its current meaning since forever I didn't think it was worth the risk to change the meaning of such a long standing symbol. Other ideas, generic_stop_super? >> generic_quiesce_super and generic_bdi_gone, are the default operations >> when a filesystem does not implement ->quiesce(), ->bdi_gone(). They >> invalidate inodes and unmap DAX-inodes respectively. For now only >> ->bdi_gone() has an associated super operation as xfs will implement >> ->bdi_gone() in a later patch. > > I don't quite understand what the point of factoring > __invalidate_device() like this is - it's not used by anyone, so it > seems completely unnecessary to me. > You're right, without a need for an fs to intercept the 'invalidation' event there's no need to do this refactor. > And really, that points out that there are multiple changes in this > patch set that should be done separately. The rework of > del_gendisk() into del_gendisk_start/del_gendisk_end should be the > first patch. The del_gendisk/blk_cleanup_queue to > blk_cleanup_queue() combining should be the second part, as it > builds on the factoring of del_gendisk(). Then, if we really need to > keep it, the factoring to introduce generic_quiesce_super. And > finally, the patch that allows the shutdown callout can be > introduced. Ok, will split. > > [snip] >> >> +static void generic_bdi_gone(struct super_block *sb) >> +{ >> + struct inode *inode, *_inode = NULL; >> + >> + spin_lock(&sb->s_inode_list_lock); >> + list_for_each_entry(inode, &sb->s_inodes, i_sb_list) { >> + spin_lock(&inode->i_lock); >> + if ((inode->i_state & (I_FREEING|I_WILL_FREE|I_NEW)) >> + || !IS_DAX(inode)) { >> + spin_unlock(&inode->i_lock); >> + continue; >> + } >> + __iget(inode); >> + spin_unlock(&inode->i_lock); >> + spin_unlock(&sb->s_inode_list_lock); >> + >> + unmap_mapping_range(inode->i_mapping, 0, 0, 1); >> + iput(_inode); >> + _inode = inode; >> + cond_resched(); >> + >> + spin_lock(&sb->s_inode_list_lock); >> + } >> + spin_unlock(&sb->s_inode_list_lock); >> + iput(_inode); >> +} > > This belongs in fs/inode.c, right next to invalidate_inodes(), and > with a name that describes what it does, not the context in which > it is called. e.g. unmap_dax_inodes(). > Sounds good. >> +void shutdown_partition(struct gendisk *disk, int partno) >> +{ >> + struct block_device *bdev = bdget_disk(disk, partno); >> + struct super_block *sb = get_super(bdev); >> + >> + if (!bdev) >> + return; > > Null pointer deref. Certainly a landmine waiting for someone to > tread on. > Nope, get_super() checks for a NULL argument. >> + if (!sb) { >> + bdput(bdev); >> + return; >> + } >> + >> + if (sb->s_op->bdi_gone) >> + sb->s_op->bdi_gone(sb); >> + else >> + generic_bdi_gone(sb); > > if (sb->s_op->shutdown) > sb->s_op->shutdown(sb); > else > unmap_dax_inodes(sb); > > name things correctly, and the code documents itself. How about 'stop' or 'halt' instead of 'shutdown' to preserve the historical meaning of generic_shutdown_super? From st101368@stud.uni-stuttgart.de Mon Jan 4 22:27:58 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.8 required=5.0 tests=FH_FROMEML_NOTLD, FREEMAIL_FORGED_REPLYTO,FSL_MISSP_REPLYTO,LOTS_OF_MONEY,MONEY_FRAUD_3 autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2686F7F37 for ; Mon, 4 Jan 2016 22:27:58 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 16850304067 for ; Mon, 4 Jan 2016 20:27:55 -0800 (PST) X-ASG-Debug-ID: 1451968068-04cb6c20fe08720001-NocioJ Received: from mx3.rus.uni-stuttgart.de (mx3.rus.uni-stuttgart.de [129.69.192.3]) by cuda.sgi.com with ESMTP id aclFpVpLW041gm58 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 04 Jan 2016 20:27:49 -0800 (PST) X-Barracuda-Envelope-From: st101368@stud.uni-stuttgart.de X-Barracuda-Apparent-Source-IP: 129.69.192.3 Received: from localhost (localhost [127.0.0.1]) by mx3.rus.uni-stuttgart.de (Postfix) with ESMTP id E32CD21142; Tue, 5 Jan 2016 05:27:47 +0100 (CET) X-Virus-Scanned: USTUTT mailrelay AV services at mx3.rus.uni-stuttgart.de Received: from mx3.rus.uni-stuttgart.de ([127.0.0.1]) by localhost (mx3.rus.uni-stuttgart.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id onLySEFCjj8a; Tue, 5 Jan 2016 05:27:46 +0100 (CET) Received: from DESKTOP-R0UORM3.rus.uni-stuttgart.de (s-8d3a11ec.off.site.uni-stuttgart.de [141.58.17.236]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx3.rus.uni-stuttgart.de (Postfix) with ESMTPS; Tue, 5 Jan 2016 05:27:39 +0100 (CET) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Description: Mail message body Subject: Charity/Donation To: you X-ASG-Orig-Subj: Charity/Donation From: "Jeffrey Skoll" Date: Tue, 05 Jan 2016 06:27:45 +0200 Reply-To: jeffrey_skoll@gmx.com Message-Id: <20160105042747.E32CD21142@mx3.rus.uni-stuttgart.de> X-Barracuda-Connect: mx3.rus.uni-stuttgart.de[129.69.192.3] X-Barracuda-Start-Time: 1451968068 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.50 X-Barracuda-Spam-Status: No, SCORE=0.50 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=ADVANCE_FEE_1, BSF_SC0_MISMATCH_TO, BSF_SC0_MV0713 X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25842 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.00 ADVANCE_FEE_1 Appears to be advance fee fraud (Nigerian 419) 0.50 BSF_SC0_MV0713 Custom rule MV0713 Hi My name is Jeffrey Skoll, a philanthropist and the founder of one of the la= rgest private foundations in the world. I believe strongly in =E2=80=98givi= ng while living.=E2=80=99 I had one idea that never changed in my mind =E2= =80=94 that you should use your wealth to help people and I have decided to= secretly give USD2.498 Million to a randomly selected individual. On recei= pt of this email, you should count yourself as the individual. Your email a= ddress was chosen online while searching at random. Kindly get back to me a= t your earliest convenience, so I know your email address is valid. Visit the web page to know more about me: http://www.theglobeandmail.com/ne= ws/national/meet-the-canadian-billionaire-whos-giving-it-all-away/article42= 09888/ or you can read an article of me on Wikipedia. Regards, Jeffrey Skoll. From jack@suse.cz Tue Jan 5 03:41:03 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id EA6287F37 for ; Tue, 5 Jan 2016 03:41:02 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 85AAEAC007 for ; Tue, 5 Jan 2016 01:41:02 -0800 (PST) X-ASG-Debug-ID: 1451986854-04bdf06f730fbd0001-NocioJ Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by cuda.sgi.com with ESMTP id brfsZKvUeuxGh0EK (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 05 Jan 2016 01:40:55 -0800 (PST) X-Barracuda-Envelope-From: jack@suse.cz X-Barracuda-Apparent-Source-IP: 195.135.220.15 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 5ACDEABE5; Tue, 5 Jan 2016 09:40:51 +0000 (UTC) Received: by quack.suse.cz (Postfix, from userid 1000) id 4F06E823D8; Tue, 5 Jan 2016 10:41:01 +0100 (CET) Date: Tue, 5 Jan 2016 10:41:01 +0100 From: Jan Kara To: Ross Zwisler Cc: linux-kernel@vger.kernel.org, "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, x86@kernel.org, xfs@oss.sgi.com, Andrew Morton , Dan Williams , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 2/7] dax: support dirty DAX entries in radix tree Message-ID: <20160105094101.GB2724@quack.suse.cz> X-ASG-Orig-Subj: Re: [PATCH v6 2/7] dax: support dirty DAX entries in radix tree References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-3-git-send-email-ross.zwisler@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450899560-26708-3-git-send-email-ross.zwisler@linux.intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mx2.suse.de[195.135.220.15] X-Barracuda-Start-Time: 1451986855 X-Barracuda-Encrypted: ECDHE-RSA-AES256-SHA X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25848 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Wed 23-12-15 12:39:15, Ross Zwisler wrote: > Add support for tracking dirty DAX entries in the struct address_space > radix tree. This tree is already used for dirty page writeback, and it > already supports the use of exceptional (non struct page*) entries. > > In order to properly track dirty DAX pages we will insert new exceptional > entries into the radix tree that represent dirty DAX PTE or PMD pages. > These exceptional entries will also contain the writeback sectors for the > PTE or PMD faults that we can use at fsync/msync time. > > There are currently two types of exceptional entries (shmem and shadow) > that can be placed into the radix tree, and this adds a third. We rely on > the fact that only one type of exceptional entry can be found in a given > radix tree based on its usage. This happens for free with DAX vs shmem but > we explicitly prevent shadow entries from being added to radix trees for > DAX mappings. > > The only shadow entries that would be generated for DAX radix trees would > be to track zero page mappings that were created for holes. These pages > would receive minimal benefit from having shadow entries, and the choice > to have only one type of exceptional entry in a given radix tree makes the > logic simpler both in clear_exceptional_entry() and in the rest of DAX. > > Signed-off-by: Ross Zwisler The patch looks good to me. You can add: Reviewed-by: Jan Kara Honza -- Jan Kara SUSE Labs, CR From BATV+ace907f9388f74b0af24+4517+infradead.org+hch@bombadil.srs.infradead.org Tue Jan 5 04:42:24 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 7CA397F37 for ; Tue, 5 Jan 2016 04:42:24 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 0F069AC002 for ; Tue, 5 Jan 2016 02:42:20 -0800 (PST) X-ASG-Debug-ID: 1451990538-04cb6c20fd0f3e0001-NocioJ Received: from bombadil.infradead.org ([198.137.202.9]) by cuda.sgi.com with ESMTP id ri65ZcTXc4ZvNazn (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 05 Jan 2016 02:42:19 -0800 (PST) X-Barracuda-Envelope-From: BATV+ace907f9388f74b0af24+4517+infradead.org+hch@bombadil.srs.infradead.org X-Barracuda-Apparent-Source-IP: 198.137.202.9 Received: from hch by bombadil.infradead.org with local (Exim 4.80.1 #2 (Red Hat Linux)) id 1aGP4A-0006IB-AN; Tue, 05 Jan 2016 10:42:14 +0000 Date: Tue, 5 Jan 2016 02:42:14 -0800 From: Christoph Hellwig To: "Darrick J. Wong" Cc: Christoph Hellwig , xfs@oss.sgi.com Subject: Re: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend Message-ID: <20160105104214.GA16310@infradead.org> X-ASG-Orig-Subj: Re: [PATCH 3/3] xfs: cancel COW in xfs_cancel_ioend References: <1451822873-12969-1-git-send-email-hch@lst.de> <1451822873-12969-4-git-send-email-hch@lst.de> <20160105014310.GK28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160105014310.GK28330@birch.djwong.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html X-Barracuda-Connect: UNKNOWN[198.137.202.9] X-Barracuda-Start-Time: 1451990539 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25848 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Mon, Jan 04, 2016 at 05:43:10PM -0800, Darrick J. Wong wrote: > Hmm. This might be the cause of the occasional complaints I've been seeing > where allocated blocks remain in the COW fork when the inode is being cleared > out. That said, the xfs_reflink_end_cow_failed() is apparently missing a > xfs_bunmapi_cow() to actually clean out the COW fork. I can still reproduce xfs_reflink_cancel_pending_cow tripping over allocated blocks in the COW fork over NFS. generic/154 reproduces it 100% over NFS, although when adding a delay before the cleanup it disappears. I'm currently trying to figure out why. From jack@suse.cz Tue Jan 5 05:14:02 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id C75B17F37 for ; Tue, 5 Jan 2016 05:14:02 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id B888C8F8035 for ; Tue, 5 Jan 2016 03:13:59 -0800 (PST) X-ASG-Debug-ID: 1451992436-04cb6c20fd100e0001-NocioJ Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by cuda.sgi.com with ESMTP id 7apljF80DcQBvHrF (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 05 Jan 2016 03:13:57 -0800 (PST) X-Barracuda-Envelope-From: jack@suse.cz X-Barracuda-Apparent-Source-IP: 195.135.220.15 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 6EEA9AAF4; Tue, 5 Jan 2016 11:13:54 +0000 (UTC) Received: by quack.suse.cz (Postfix, from userid 1000) id CCB80823D9; Tue, 5 Jan 2016 12:13:58 +0100 (CET) Date: Tue, 5 Jan 2016 12:13:58 +0100 From: Jan Kara To: Ross Zwisler Cc: linux-kernel@vger.kernel.org, "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, x86@kernel.org, xfs@oss.sgi.com, Andrew Morton , Dan Williams , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync Message-ID: <20160105111358.GD2724@quack.suse.cz> X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mx2.suse.de[195.135.220.15] X-Barracuda-Start-Time: 1451992437 X-Barracuda-Encrypted: ECDHE-RSA-AES256-SHA X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25849 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Wed 23-12-15 12:39:17, Ross Zwisler wrote: > To properly handle fsync/msync in an efficient way DAX needs to track dirty > pages so it is able to flush them durably to media on demand. > > The tracking of dirty pages is done via the radix tree in struct > address_space. This radix tree is already used by the page writeback > infrastructure for tracking dirty pages associated with an open file, and > it already has support for exceptional (non struct page*) entries. We > build upon these features to add exceptional entries to the radix tree for > DAX dirty PMD or PTE pages at fault time. > > Signed-off-by: Ross Zwisler ... > +static int dax_writeback_one(struct block_device *bdev, > + struct address_space *mapping, pgoff_t index, void *entry) > +{ > + struct radix_tree_root *page_tree = &mapping->page_tree; > + int type = RADIX_DAX_TYPE(entry); > + struct radix_tree_node *node; > + struct blk_dax_ctl dax; > + void **slot; > + int ret = 0; > + > + spin_lock_irq(&mapping->tree_lock); > + /* > + * Regular page slots are stabilized by the page lock even > + * without the tree itself locked. These unlocked entries > + * need verification under the tree lock. > + */ > + if (!__radix_tree_lookup(page_tree, index, &node, &slot)) > + goto unlock; > + if (*slot != entry) > + goto unlock; > + > + /* another fsync thread may have already written back this entry */ > + if (!radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_TOWRITE)) > + goto unlock; > + > + radix_tree_tag_clear(page_tree, index, PAGECACHE_TAG_TOWRITE); > + > + if (WARN_ON_ONCE(type != RADIX_DAX_PTE && type != RADIX_DAX_PMD)) { > + ret = -EIO; > + goto unlock; > + } > + > + dax.sector = RADIX_DAX_SECTOR(entry); > + dax.size = (type == RADIX_DAX_PMD ? PMD_SIZE : PAGE_SIZE); > + spin_unlock_irq(&mapping->tree_lock); > + > + /* > + * We cannot hold tree_lock while calling dax_map_atomic() because it > + * eventually calls cond_resched(). > + */ > + ret = dax_map_atomic(bdev, &dax); > + if (ret < 0) > + return ret; > + > + if (WARN_ON_ONCE(ret < dax.size)) { > + ret = -EIO; > + dax_unmap_atomic(bdev, &dax); > + return ret; > + } > + > + spin_lock_irq(&mapping->tree_lock); > + /* > + * We need to revalidate our radix entry while holding tree_lock > + * before we do the writeback. > + */ Do we really need to revalidate here? dax_map_atomic() makes sure the addr & size is still part of the device. I guess you are concerned that due to truncate or similar operation those sectors needn't belong to the same file anymore but we don't really care about flushing sectors for someone else, do we? Otherwise the patch looks good to me. > + if (!__radix_tree_lookup(page_tree, index, &node, &slot)) > + goto unmap; > + if (*slot != entry) > + goto unmap; > + > + wb_cache_pmem(dax.addr, dax.size); > + unmap: > + dax_unmap_atomic(bdev, &dax); > + unlock: > + spin_unlock_irq(&mapping->tree_lock); > + return ret; > +} Honza -- Jan Kara SUSE Labs, CR From jack@suse.cz Tue Jan 5 05:14:04 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 6A7E97F50 for ; Tue, 5 Jan 2016 05:14:04 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id E0CF1AC003 for ; Tue, 5 Jan 2016 03:14:00 -0800 (PST) X-ASG-Debug-ID: 1451992437-04cbb0451411130001-NocioJ Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by cuda.sgi.com with ESMTP id uaY7Dc7bGSboeNrs (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 05 Jan 2016 03:13:58 -0800 (PST) X-Barracuda-Envelope-From: jack@suse.cz X-Barracuda-Apparent-Source-IP: 195.135.220.15 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 53B7DABE5; Tue, 5 Jan 2016 11:13:56 +0000 (UTC) Received: by quack.suse.cz (Postfix, from userid 1000) id F2115823D8; Tue, 5 Jan 2016 12:13:46 +0100 (CET) Date: Tue, 5 Jan 2016 12:13:46 +0100 From: Jan Kara To: Dan Williams Cc: Ross Zwisler , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync Message-ID: <20160105111346.GC2724@quack.suse.cz> X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mx2.suse.de[195.135.220.15] X-Barracuda-Start-Time: 1451992438 X-Barracuda-Encrypted: ECDHE-RSA-AES256-SHA X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25849 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header On Sun 03-01-16 10:13:06, Dan Williams wrote: > On Wed, Dec 23, 2015 at 11:39 AM, Ross Zwisler > wrote: > > To properly handle fsync/msync in an efficient way DAX needs to track dirty > > pages so it is able to flush them durably to media on demand. > > > > The tracking of dirty pages is done via the radix tree in struct > > address_space. This radix tree is already used by the page writeback > > infrastructure for tracking dirty pages associated with an open file, and > > it already has support for exceptional (non struct page*) entries. We > > build upon these features to add exceptional entries to the radix tree for > > DAX dirty PMD or PTE pages at fault time. > > > > Signed-off-by: Ross Zwisler > > I'm hitting the following report with the ndctl dax test [1] on > next-20151231. I bisected it to > commit 3cb108f941de "dax-add-support-for-fsync-sync-v6". I'll take a > closer look tomorrow, but in case someone can beat me to it, here's > the back-trace: > > ------------[ cut here ]------------ > kernel BUG at fs/inode.c:497! I suppose this is the check that mapping->nr_exceptional is zero, isn't it? Hum, I don't see how that could happen given we call truncate_inode_pages_final() just before the clear_inode() call which removes all the exceptional entries from the radix tree. And there's not much room for a race during umount... Does the radix tree really contain any entry or is it an accounting bug? Honza > [..] > CPU: 1 PID: 3001 Comm: umount Tainted: G O 4.4.0-rc7+ #2412 > Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 > task: ffff8800da2a5a00 ti: ffff880307794000 task.ti: ffff880307794000 > RIP: 0010:[] [] clear_inode+0x71/0x80 > RSP: 0018:ffff880307797d50 EFLAGS: 00010002 > RAX: ffff8800da2a5a00 RBX: ffff8800ca2e7328 RCX: ffff8800da2a5a28 > RDX: 0000000000000001 RSI: 0000000000000005 RDI: ffff8800ca2e7530 > RBP: ffff880307797d60 R08: ffffffff82900ae0 R09: 0000000000000000 > R10: ffff8800ca2e7548 R11: 0000000000000000 R12: ffff8800ca2e7530 > R13: ffff8800ca2e7328 R14: ffff8800da2e88d0 R15: ffff8800da2e88d0 > FS: 00007f2b22f4a880(0000) GS:ffff88031fc40000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 00005648abd933e8 CR3: 000000007f3fc000 CR4: 00000000000006e0 > Stack: > ffff8800ca2e7328 ffff8800ca2e7000 ffff880307797d88 ffffffffa01c18af > ffff8800ca2e7328 ffff8800ca2e74d0 ffffffffa01ec740 ffff880307797db0 > ffffffff81281038 ffff8800ca2e74c0 ffff880307797e00 ffff8800ca2e7328 > Call Trace: > [] xfs_fs_evict_inode+0x5f/0x110 [xfs] > [] evict+0xb8/0x180 > [] dispose_list+0x3b/0x50 > [] evict_inodes+0x144/0x170 > [] generic_shutdown_super+0x3f/0xf0 > [] kill_block_super+0x27/0x70 > [] deactivate_locked_super+0x43/0x70 > [] deactivate_super+0x5c/0x60 > [] cleanup_mnt+0x3f/0x90 > [] __cleanup_mnt+0x12/0x20 > [] task_work_run+0x76/0x90 > [] syscall_return_slowpath+0x20a/0x280 > [] int_ret_from_sys_call+0x25/0x9f > Code: 48 8d 93 30 03 00 00 48 39 c2 75 23 48 8b 83 d0 00 00 00 a8 20 > 74 1a a8 40 75 18 48 c7 8 > 3 d0 00 00 00 60 00 00 00 5b 41 5c 5d c3 <0f> 0b 0f 0b 0f 0b 0f 0b 0f > 0b 0f 1f 44 00 00 0f 1f > 44 00 00 55 > RIP [] clear_inode+0x71/0x80 > RSP > ---[ end trace 3b1d8898a94a4fc1 ]--- > > [1]: git://git@github.com:pmem/ndctl.git pending > make TESTS="test/dax.sh" check > -- Jan Kara SUSE Labs, CR From bfoster@redhat.com Tue Jan 5 06:42:53 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 202387F37 for ; Tue, 5 Jan 2016 06:42:53 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 00CFB8F8037 for ; Tue, 5 Jan 2016 04:42:52 -0800 (PST) X-ASG-Debug-ID: 1451997768-04cb6c210011880001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Kguwuky7ds1dRqmq (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 04:42:48 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27]) by mx1.redhat.com (Postfix) with ESMTPS id EB8DE264F; Tue, 5 Jan 2016 12:42:27 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05CgR0O010135; Tue, 5 Jan 2016 07:42:27 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id A17BB124F12; Tue, 5 Jan 2016 07:42:26 -0500 (EST) Date: Tue, 5 Jan 2016 07:42:26 -0500 From: Brian Foster To: "Darrick J. Wong" Cc: xfs@oss.sgi.com Subject: Re: [RFCv4 00/76] xfs: add reverse-mapping, reflink, and dedupe support Message-ID: <20160105124226.GA38749@bfoster.bfoster> X-ASG-Orig-Subj: Re: [RFCv4 00/76] xfs: add reverse-mapping, reflink, and dedupe support References: <20151219085622.12713.88678.stgit@birch.djwong.org> <20151220140254.GA3618@laptop.bfoster> <20160104235951.GE28330@birch.djwong.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104235951.GE28330@birch.djwong.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1451997768 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Mon, Jan 04, 2016 at 03:59:51PM -0800, Darrick J. Wong wrote: > On Sun, Dec 20, 2015 at 09:02:54AM -0500, Brian Foster wrote: > > On Sat, Dec 19, 2015 at 12:56:23AM -0800, Darrick J. Wong wrote: > > > Hi all, > > > > > ... > > > Fixed since RFCv3: > > > > > > * The reflink and dedupe ioctls are being hoisted to the VFS, as > > > provided in the first few patches. Patch 81 connects to this > > > functionality. > > > > > > * Copy on write has been rewritten for v4. We now use the existing > > > delayed allocation mechanism to coalesce writes together, deferring > > > allocation until writeout time. This enables CoW to make better > > > block placement decisions and significantly reduces overhead. > > > CoW is still pretty slow, but not as slow as before. > > > > > > * Direct IO CoW has been implemented using the same mechanism as > > > above, but modified to perform the allocation and remapping right > > > then and there. Throughput is much higher than pushing data > > > through the page cache CoW. (It's the same mechanism, but we're > > > playing with chunks bigger than a single memory page.) > > > > > > * CoW ENOSPC works correctly now, except in the pathological case > > > that the AG fills up and the rmap btree cannot expand. That will > > > be addressed for v5. > > > > > > * fallocate will now unshare blocks to prevent future ENOSPC, as > > > you'd expect. > > > > > > * refcount btree blocks are preallocated at mount time to prevent > > > ENOSPC while trying to expand the tree. This also has the effect > > > of grouping the btree blocks together, which can speed up CoW > > > remapping. > > > > > > > Can you elaborate on how these blocks are preallocated? E.g., is the > > tree "preconstructed" in some sense? However that is done, is this the > > anticipated solution or a temporary workaround..? > > > > Also, shouldn't the enospc condition be handled by the agfl? I take it > > there is something going on here that renders that solution flawed, so > > I'm just curious what it is. > > > > (Sorry if this is all explained elsewhere, but I haven't yet had a > > chance to take a close enough look at this feature..). > > Reference count btree blocks aren't allocated from the AGFL; they're allocated > from the free space in the same manner as the inobt, per a review comment from > Dave a looong time ago. :) > Ah, Ok. > As such, we can get ourselves into the nasty situation where every block in the > AG has been allocated to file data. If we then see a bunch of reference count > changes that are scattered around the AG, the reference count btree has to > expand to hold all the new records... but there isn't space, and the operation > fails. Given that we know the maximum possible size of the refcount btree > (it's 0.3% of the AG size with 4k blocks), I figured it was easy enough to > avoid ENOSPC for reflink operations. > Sounds reasonable. > I've temporarily fixed this by adding code that figures out how many blocks we > need if the reference count btree has to have a unique record for every block > in the AG and holding that many blocks until either they're allocated to the > refcount btree or freed at umount time. Right now it's a temporary fix (if the > FS crashes, the reserved blocks are lost) but it wouldn't be difficult for the > FS to make a permanent reservation that's recorded on disk somehow. But that's > involves writing things to disk + making xfsprogs understand the reservation; > let's see what people say about the reserved pool idea at all. > > Does that make sense? :) > Yep, it sounds sort of like the reserve pool mechanism used to protect against ENOSPC when freeing blocks. Curious... why are the reserved blocks lost on fs crash? Wouldn't they be reserved again on the subsequent mount? Thanks for the explanation... Brian > --D > > > > > Brian > > > > > Issues: > > > > > > * The extent swapping ioctl still allocates a bigger fixed-size > > > transaction. That's most likely a stupid thing to do, so getting a > > > better grip on how the journalling code works and auditing all the > > > new transaction users will have to happen. Right now it mostly > > > gets lucky. > > > > > > * EFI tracking for the allocated-but-not-yet-mapped blocks is > > > nonexistant. A crash will leak them. > > > > > > * ENOSPC while expanding the rmap btree can crash the FS. For now we > > > work around this problem by making the AGFL as big as possible, > > > failing CoW attempts with ENOSPC if there aren't enough AGFL blocks > > > available, and hoping that doesn't actually happen. > > > > > > If you're going to start using this mess, you probably ought to just > > > pull from my github trees for kernel[1], xfsprogs[2], and xfstests[3]. > > > There are also updates for xfs-docs[4] and man-pages[5]. > > > > > > The patches have been xfstested with x64, i386, and ppc64; while in > > > general the tests run to completion, there are still periodic bugs > > > that will be addressed by the next RFC. There's a persistent crash on > > > arm64 and ppc64el that I haven't been able to triage. > > > > > > This is an extraordinary way to eat your data. Enjoy! > > > Comments and questions are, as always, welcome. > > > > > > --D > > > > > > [1] https://github.com/djwong/linux/tree/for-dave > > > [2] https://github.com/djwong/xfsprogs/tree/for-dave > > > [3] https://github.com/djwong/xfstests/tree/for-dave > > > [4] https://github.com/djwong/xfs-documentation/tree/for-dave > > > [5] https://github.com/djwong/man-pages/commits/for-mtk > > > > > > _______________________________________________ > > > xfs mailing list > > > xfs@oss.sgi.com > > > http://oss.sgi.com/mailman/listinfo/xfs > > > > _______________________________________________ > > xfs mailing list > > xfs@oss.sgi.com > > http://oss.sgi.com/mailman/listinfo/xfs From ross.zwisler@linux.intel.com Tue Jan 5 09:50:50 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 80E937F37 for ; Tue, 5 Jan 2016 09:50:50 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 62BBD304053 for ; Tue, 5 Jan 2016 07:50:47 -0800 (PST) X-ASG-Debug-ID: 1452009042-04cbb04515180a0001-NocioJ Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by cuda.sgi.com with ESMTP id b0jgu9gcHEt2mtzt for ; Tue, 05 Jan 2016 07:50:42 -0800 (PST) X-Barracuda-Envelope-From: ross.zwisler@linux.intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.24 X-ASG-Whitelist: EmailCat (corporate) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga102.jf.intel.com with ESMTP; 05 Jan 2016 07:50:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,525,1444719600"; d="scan'208";a="884216047" Received: from rzwisler-desk.amr.corp.intel.com (HELO linux.intel.com) ([10.254.191.93]) by orsmga002.jf.intel.com with ESMTP; 05 Jan 2016 07:50:39 -0800 Date: Tue, 5 Jan 2016 08:50:39 -0700 From: Ross Zwisler To: Jan Kara Cc: Dan Williams , Ross Zwisler , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync Message-ID: <20160105155039.GA6462@linux.intel.com> X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync Mail-Followup-To: Ross Zwisler , Jan Kara , Dan Williams , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> <20160105111346.GC2724@quack.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160105111346.GC2724@quack.suse.cz> User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mga09.intel.com[134.134.136.24] X-Barracuda-Start-Time: 1452009042 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Jan 05, 2016 at 12:13:46PM +0100, Jan Kara wrote: > On Sun 03-01-16 10:13:06, Dan Williams wrote: > > On Wed, Dec 23, 2015 at 11:39 AM, Ross Zwisler > > wrote: > > > To properly handle fsync/msync in an efficient way DAX needs to track dirty > > > pages so it is able to flush them durably to media on demand. > > > > > > The tracking of dirty pages is done via the radix tree in struct > > > address_space. This radix tree is already used by the page writeback > > > infrastructure for tracking dirty pages associated with an open file, and > > > it already has support for exceptional (non struct page*) entries. We > > > build upon these features to add exceptional entries to the radix tree for > > > DAX dirty PMD or PTE pages at fault time. > > > > > > Signed-off-by: Ross Zwisler > > > > I'm hitting the following report with the ndctl dax test [1] on > > next-20151231. I bisected it to > > commit 3cb108f941de "dax-add-support-for-fsync-sync-v6". I'll take a > > closer look tomorrow, but in case someone can beat me to it, here's > > the back-trace: > > > > ------------[ cut here ]------------ > > kernel BUG at fs/inode.c:497! > > I suppose this is the check that mapping->nr_exceptional is zero, isn't it? > Hum, I don't see how that could happen given we call > truncate_inode_pages_final() just before the clear_inode() call which > removes all the exceptional entries from the radix tree. And there's not > much room for a race during umount... Does the radix tree really contain > any entry or is it an accounting bug? > > Honza I think this is a bug with the existing way that we handle PMD faults. The issue is that the PMD path doesn't properly remove radix tree entries for zero pages covering holes. The PMD path calls unmap_mapping_range() to unmap the range out of the struct address_space, but it is missing a call to truncate_inode_pages_range() or similar to clear out those entries in the radix tree. Up until now we didn't notice, we just had an orphaned entry in the radix tree, but with my code we then find the page entry in the radix tree when handling a PMD fault, we remove it and add in a PMD entry. This causes us to be off on both our mapping->nrpages and mapping->nrexceptional counts. In the PTE path we properly remove the pages from the radix tree when upgrading from a hole to a real DAX entry via the delete_from_page_cache() call, which eventually calls page_cache_tree_delete(). I'm working on a fix now (and making sure all the above is correct). - Ross From andrea.gelmini@gmail.com Tue Jan 5 10:31:03 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=FREEMAIL_FROM,T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 513B07F37 for ; Tue, 5 Jan 2016 10:31:03 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 3CA078F804B for ; Tue, 5 Jan 2016 08:31:00 -0800 (PST) X-ASG-Debug-ID: 1452011457-04cb6c20fd16c00001-NocioJ Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) by cuda.sgi.com with ESMTP id ESGWF8BlqVSPGHw2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 05 Jan 2016 08:30:57 -0800 (PST) X-Barracuda-Envelope-From: andrea.gelmini@gmail.com X-Barracuda-Apparent-Source-IP: 74.125.82.48 Received: by mail-wm0-f48.google.com with SMTP id f206so37386495wmf.0 for ; Tue, 05 Jan 2016 08:30:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=h3cQMbT6L5KEMeBl6NZBNsOYLF7I8HneIvQVc33Jl/A=; b=aeQ2MZBQ/ZgWr4OYb4M57sJqJpwoYXrHwDSYylrh4UcTmWaPfNJOr6SZZwo3XiO9T2 VmAVCmFT3E5t9MA+4CllIVTd3QHw/+gCr4oMaFwmhRZBln656MYXyeKzYcqjWJhHIWQY ZUj0COG5WZrX12dRl4d4kW94P5Mkx8/hzmGkDvaHYnpSNuOi1DoZBdJ5Pl1ytwcbMcv5 PiuxAyf+AFLx87iO5oDK5HwuC8RrJMfZLKrVZhwFMYaVEpP7hF0RevLyKoZpmrFGKs7J FYZxP8DgU3wI7AQsg9OiGifxqkVfkCBWUgzGXbxpejmb5NUEZ7oRbw0ZfTEZKTqQY/MJ yvEg== X-Received: by 10.194.19.100 with SMTP id d4mr104462734wje.18.1452011456889; Tue, 05 Jan 2016 08:30:56 -0800 (PST) Received: from glen ([195.206.31.238]) by smtp.gmail.com with ESMTPSA id z127sm4341843wme.2.2016.01.05.08.30.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Jan 2016 08:30:55 -0800 (PST) Sender: Andrea Gelmini Date: Tue, 5 Jan 2016 17:30:55 +0100 From: Andrea Gelmini To: Dave Chinner Cc: linux-kernel@vger.kernel.org, xfs@oss.sgi.com Subject: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 Message-ID: <20160105163055.GA18111@glen> X-ASG-Orig-Subj: Re: BUG: KASAN: use-after-free in xfs_iflush_cluster+0x9d7/0xaf0 References: <20151214180048.GA15690@glen> <20151214195422.GM26718@dastard> <20151214201526.GA25152@glen> <20151214212220.GO26718@dastard> <20151215091145.GA19282@glen> <20160103204758.GW19802@dastard> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="uAKRQypu60I7Lcqm" Content-Disposition: inline In-Reply-To: <20160103204758.GW19802@dastard> User-Agent: Mutt/1.5.23 (2014-03-12) X-Barracuda-Connect: mail-wm0-f48.google.com[74.125.82.48] X-Barracuda-Start-Time: 1452011457 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 2.60 X-Barracuda-Spam-Status: No, SCORE=2.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_MV0249, DKIM_SIGNED, DKIM_VERIFIED, MARKETING_SUBJECT X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25854 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.60 MARKETING_SUBJECT Subject contains popular marketing words 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature 2.00 BSF_SC0_MV0249 Custom rule MV0249 --uAKRQypu60I7Lcqm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Jan 04, 2016 at 07:47:58AM +1100, Dave Chinner wrote: > > I'm recompiling, to try it again. > > Maybe, in the meanwhile, you can do something with my files. You can find 'em here: > > http://mail.gelma.net/xfs_kasan > > Any update on this problem, Andrea? Here we are! Reproduced right now. So, just to avoid confusion: a) it's a vanilla kernel 4.4.0-rc8 b) plus some btrfs patches c) plus some dri/intel/i915 patches d) at the same URL above you can find git_files.txt.gz, where you have each commit I applied above vanilla kernel (anyway, nothing related to vfs/xfs of course) e) at the same URL you find the kernel binaries I used f) to catch it, I had to copy a few gigs of files on my /home partition (xfs over Luks) Anyway, here what you asked me for: (gdb) l *(xfs_iflush_cluster+0xb73/0xc10) 0xffffffff8184c550 is in xfs_iflush_cluster (fs/xfs/xfs_inode.c:3182). 3177 3178 STATIC int 3179 xfs_iflush_cluster( 3180 xfs_inode_t *ip, 3181 xfs_buf_t *bp) 3182 { 3183 xfs_mount_t *mp = ip->i_mount; 3184 struct xfs_perag *pag; 3185 unsigned long first_index, mask; 3186 unsigned long inodes_per_cluster; (gdb) Thanks a lot for your patience, Dave [mar gen 5 16:58:19 2016] ================================================================== [mar gen 5 16:58:19 2016] BUG: KASAN: use-after-free in xfs_iflush_cluster+0xb73/0xc10 at addr ffff880364721d10 [mar gen 5 16:58:19 2016] Read of size 4 by task xfsaild/dm-0/329 [mar gen 5 16:58:19 2016] ============================================================================= [mar gen 5 16:58:19 2016] BUG xfs_ili (Tainted: G W ): kasan: bad access detected [mar gen 5 16:58:19 2016] ----------------------------------------------------------------------------- [mar gen 5 16:58:19 2016] Disabling lock debugging due to kernel taint [mar gen 5 16:58:19 2016] INFO: Allocated in kmem_zone_alloc+0x7c/0x180 age=496908 cpu=1 pid=6496 [mar gen 5 16:58:19 2016] ___slab_alloc.constprop.27+0x383/0x490 [mar gen 5 16:58:19 2016] __slab_alloc.isra.24.constprop.26+0x50/0xa0 [mar gen 5 16:58:19 2016] kmem_cache_alloc+0x174/0x1b0 [mar gen 5 16:58:19 2016] kmem_zone_alloc+0x7c/0x180 [mar gen 5 16:58:19 2016] xfs_inode_item_init+0x22/0xb0 [mar gen 5 16:58:19 2016] xfs_trans_ijoin+0xa4/0x110 [mar gen 5 16:58:19 2016] xfs_ialloc+0x9f9/0x1390 [mar gen 5 16:58:19 2016] xfs_dir_ialloc+0x106/0x670 [mar gen 5 16:58:19 2016] xfs_create+0x67e/0x1080 [mar gen 5 16:58:19 2016] xfs_generic_create+0x375/0x500 [mar gen 5 16:58:19 2016] xfs_vn_mknod+0xf/0x20 [mar gen 5 16:58:19 2016] xfs_vn_create+0xe/0x10 [mar gen 5 16:58:19 2016] vfs_create+0x1ff/0x390 [mar gen 5 16:58:19 2016] do_last+0x29a7/0x3900 [mar gen 5 16:58:19 2016] path_openat+0x15b/0x730 [mar gen 5 16:58:19 2016] do_filp_open+0x170/0x230 [mar gen 5 16:58:19 2016] INFO: Freed in xfs_inode_item_destroy+0x39/0x50 age=0 cpu=3 pid=38 [mar gen 5 16:58:19 2016] __slab_free+0x36d/0x510 [mar gen 5 16:58:19 2016] kmem_cache_free+0x1ef/0x200 [mar gen 5 16:58:19 2016] xfs_inode_item_destroy+0x39/0x50 [mar gen 5 16:58:19 2016] xfs_inode_free+0xcd/0x360 [mar gen 5 16:58:19 2016] xfs_reclaim_inode+0x54b/0x890 [mar gen 5 16:58:19 2016] xfs_reclaim_inodes_ag+0x3e9/0x840 [mar gen 5 16:58:19 2016] xfs_reclaim_inodes_nr+0x49/0x60 [mar gen 5 16:58:19 2016] xfs_fs_free_cached_objects+0x55/0x80 [mar gen 5 16:58:19 2016] super_cache_scan+0x329/0x410 [mar gen 5 16:58:19 2016] shrink_slab.part.7+0x2f2/0x530 [mar gen 5 16:58:19 2016] shrink_zone+0x7a0/0xae0 [mar gen 5 16:58:19 2016] kswapd+0x9ad/0x1110 [mar gen 5 16:58:19 2016] kthread+0x218/0x2e0 [mar gen 5 16:58:19 2016] ret_from_fork+0x3f/0x70 [mar gen 5 16:58:19 2016] INFO: Slab 0xffffea000d91c800 objects=35 used=29 fp=0xffff880364721c80 flags=0x8000000000004080 [mar gen 5 16:58:19 2016] INFO: Object 0xffff880364721c80 @offset=7296 fp=0xffff880364721560 [mar gen 5 16:58:19 2016] Bytes b4 ffff880364721c70: 03 00 00 00 3f 34 00 00 b8 51 cd 00 01 00 00 00 ....?4...Q...... [mar gen 5 16:58:19 2016] Object ffff880364721c80: 60 15 72 64 03 88 ff ff 00 02 00 00 00 00 ad de `.rd............ [mar gen 5 16:58:19 2016] Object ffff880364721c90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [mar gen 5 16:58:19 2016] Object ffff880364721ca0: 00 00 3d 5e 03 88 ff ff 60 04 92 5d 03 88 ff ff ..=^....`..].... [mar gen 5 16:58:19 2016] Object ffff880364721cb0: 3b 12 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;............... [mar gen 5 16:58:19 2016] Object ffff880364721cc0: 30 84 88 86 ff ff ff ff 60 17 6f 87 ff ff ff ff 0.......`.o..... [mar gen 5 16:58:19 2016] Object ffff880364721cd0: d0 1c 72 64 03 88 ff ff d0 1c 72 64 03 88 ff ff ..rd......rd.... [mar gen 5 16:58:19 2016] Object ffff880364721ce0: 00 00 00 00 00 00 00 00 68 76 00 00 00 00 00 00 ........hv...... [mar gen 5 16:58:19 2016] Object ffff880364721cf0: 80 6c 40 3e 01 88 ff ff db 4a 00 00 e2 00 00 00 .l@>.....J...... [mar gen 5 16:58:19 2016] Object ffff880364721d00: d6 79 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .y.............. [mar gen 5 16:58:19 2016] Object ffff880364721d10: 00 00 00 00 00 00 00 00 ........ [mar gen 5 16:58:19 2016] CPU: 0 PID: 329 Comm: xfsaild/dm-0 Tainted: G B W 4.4.0-rc8-KASan-01354-g3041cce #6 [mar gen 5 16:58:19 2016] Hardware name: LENOVO 2356LRG/2356LRG, BIOS G7ETA4WW (2.64 ) 10/08/2015 [mar gen 5 16:58:19 2016] ffff880364720000 ffff88035f2ef968 ffffffff86a2adea ffff88035f498480 [mar gen 5 16:58:19 2016] ffff88035f2ef998 ffffffff86423ab4 ffff88035f498480 ffffea000d91c800 [mar gen 5 16:58:19 2016] ffff880364721c80 ffff88013e406c80 ffff88035f2ef9c0 ffffffff86428edf [mar gen 5 16:58:19 2016] Call Trace: [mar gen 5 16:58:19 2016] [] dump_stack+0x4e/0x84 [mar gen 5 16:58:19 2016] [] print_trailer+0xf4/0x150 [mar gen 5 16:58:19 2016] [] object_err+0x2f/0x40 [mar gen 5 16:58:19 2016] [] kasan_report_error+0x207/0x530 [mar gen 5 16:58:19 2016] [] __asan_report_load4_noabort+0x3e/0x40 [mar gen 5 16:58:19 2016] [] ? _raw_spin_lock_irqsave_nested+0x50/0x70 [mar gen 5 16:58:19 2016] [] ? xfs_iflush_cluster+0xb73/0xc10 [mar gen 5 16:58:19 2016] [] xfs_iflush_cluster+0xb73/0xc10 [mar gen 5 16:58:19 2016] [] ? xfs_iflush_cluster+0x210/0xc10 [mar gen 5 16:58:19 2016] [] xfs_iflush+0x37a/0x5b0 [mar gen 5 16:58:19 2016] [] ? xfs_rename+0xe70/0xe70 [mar gen 5 16:58:19 2016] [] xfs_inode_item_push+0x25a/0x390 [mar gen 5 16:58:19 2016] [] ? xfs_inode_item_unlock+0x80/0x80 [mar gen 5 16:58:19 2016] [] ? up+0x68/0xb0 [mar gen 5 16:58:19 2016] [] ? xfs_buf_unlock+0xd/0x10 [mar gen 5 16:58:19 2016] [] xfsaild+0x8fb/0x1500 [mar gen 5 16:58:19 2016] [] ? trace_hardirqs_on_caller+0x28c/0x5e0 [mar gen 5 16:58:19 2016] [] ? xfs_trans_ail_cursor_first+0x1a0/0x1a0 [mar gen 5 16:58:19 2016] [] ? xfs_trans_ail_cursor_first+0x1a0/0x1a0 [mar gen 5 16:58:19 2016] [] kthread+0x218/0x2e0 [mar gen 5 16:58:19 2016] [] ? kthread_create_on_node+0x460/0x460 [mar gen 5 16:58:19 2016] [] ? kthread_create_on_node+0x460/0x460 [mar gen 5 16:58:19 2016] [] ret_from_fork+0x3f/0x70 [mar gen 5 16:58:19 2016] [] ? kthread_create_on_node+0x460/0x460 [mar gen 5 16:58:19 2016] Memory state around the buggy address: [mar gen 5 16:58:19 2016] ffff880364721c00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [mar gen 5 16:58:19 2016] ffff880364721c80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [mar gen 5 16:58:19 2016] >ffff880364721d00: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc [mar gen 5 16:58:19 2016] ^ [mar gen 5 16:58:19 2016] ffff880364721d80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc [mar gen 5 16:58:19 2016] ffff880364721e00: fc fc fc fc fc fc fc fc fc fb fb fb fb fb fb fb [mar gen 5 16:58:19 2016] ================================================================== --uAKRQypu60I7Lcqm Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQJ8BAEBCgBmBQJWi++/XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFQTQyRTkzNjBCNzVERDZFODMyNTQ3RjdB NDMwOTA3NUYwNTk2MEY2AAoJEKQwkHXwWWD27ugP/iUDMhh2eDeuCMr8PUk25kIO oJVwB7Z2pFSpR97J8W6/fwzDUrZJyiKYDG6pDVaet3mzXqvrbj5wLXcd7UNM4bdU R6KbhzHUUgpXBOXKsEJXGvqN012fdgb9LbS9EoQLnu926VIvdnnbahJtqU2aj5nB 95zQCA7K//Lc72Z7tbiWtKlb9V5SzhFwSFKAazhHJWuQxNvGRgFNOq7uYLPpnHRK Vl5Agsnd7EMRQgTLShd+k9FokkXuk1JqKrPm7xfXWG7TVbMla4Now3KvgicdZ3UQ /vp2O4SeS8EJ2FVyxIYvEIVmxmlzJWvix7BzDo9b3rNVt+SOAJLuyJa9ESZ7bID2 EqIrFKiWCQ2A9N+646E5EFWyHK7Hyj3x5PAppmO14gHWakiaPIkmrOpkydx8Be+r l2toPev8wEiVQwIlvterKOZRnliTCk1LRq3pPm3sy+Cd+eb1erwDGFU0yAuJ/Y7U HoQtgBWnC+j+yZpWrmxIFwpwfFTKmpoeT3Tx3gFuuSexQ9nPYPPqcmMmLFvN5fUS ZuUXn6bX2ZPFQY339ekS6MMXZDU9mHGbr0krVf9AltjONC+G41D6rgeeUD6dgSUX ECDvXhGli56ZmQ5LrOoa/9L8EeG+FahhcWAC/9CVF0OXyXaKWX0oMqKVc9bHcEcO fVlrxZ2Q3kr6ubvDOa51 =g97u -----END PGP SIGNATURE----- --uAKRQypu60I7Lcqm-- From wangqiang@cvtt.cn Tue Jan 5 11:12:19 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: *** X-Spam-Status: No, score=3.0 required=5.0 tests=AXB_XMAILER_MIMEOLE_OL_E023A, HTML_MESSAGE autolearn=no version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id BAE467F37 for ; Tue, 5 Jan 2016 11:12:19 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 5BC14AC003 for ; Tue, 5 Jan 2016 09:12:15 -0800 (PST) X-ASG-Debug-ID: 1452013931-04bdf06f74196b0001-NocioJ Received: from cvtt.cn ([117.122.192.131]) by cuda.sgi.com with SMTP id 0ZnloIFBFL36FEFT for ; Tue, 05 Jan 2016 09:12:12 -0800 (PST) X-Barracuda-Envelope-From: wangqiang@cvtt.cn X-Barracuda-Apparent-Source-IP: 117.122.192.131 Received: from liebiao.com ([124.163.209.67]) (envelope-sender ) by 10.10.12.31 with ESMTP for ; Wed, 06 Jan 2016 01:12:14 +0800 Sender: wangqiang@cvtt.cn Message-ID: <2FDECFC6FCB6A1B3443927D317CFCB74@liebiao.com> From: "admin" To: Subject: =?utf-8?B?6YKu566x5Y2H57qn6YCa55+l?= Date: Wed, 6 Jan 2016 01:11:56 +0800 X-ASG-Orig-Subj: =?utf-8?B?6YKu566x5Y2H57qn6YCa55+l?= MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_08E1_01868514.144A2170" X-Priority: 1 X-MSMail-Priority: High X-Mailer: Microsoft Outlook Express 6.00.2900.5512 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5512 X-Barracuda-Connect: UNKNOWN[117.122.192.131] X-Barracuda-Start-Time: 1452013931 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.60 X-Barracuda-Spam-Status: No, SCORE=0.60 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MV0113c, BSF_SC5_MJ1963, HTML_MESSAGE, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25855 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MV0113c BSF_SC0_MV0113c 0.00 HTML_MESSAGE BODY: HTML included in message 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS 0.50 BSF_SC5_MJ1963 Custom Rule MJ1963 This is a multi-part message in MIME format. ------=_NextPart_000_08E1_01868514.144A2170 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 5ZCE5L2N5ZCM5LqL5L2g5Lus5aW977yaDQogICAgIOWFrOWPuOWKnuWFrOiHquWKqOWMlu+8iE9B 77yJ57O757uf6Ieq6L+Q6KGM5Lul5p2l77yM5bey5LiN5pat5LyY5YyW5a6M5ZaE44CC5Li65o+Q 6auY5Yqe5YWs5pWI546H77yM5a6e546w5peg57q45YyW5Yqe5YWs77yM5YWs5Y+45bCG5YWo6Z2i 5o6o6L+b5Yqe5YWs6Ieq5Yqo5YyW77yIT0HvvInns7vnu5/nmoTkvb/nlKjjgIINCuWFrOWPuOmC rueuseezu+e7n+iuoeWIkuS6jjHmnIg55pel5byA5aeL6L+b6KGM5Y2H57qn77yM5Zyo5q2k5LmL 5YmN77yM6K+35oKo5Yqh5b+F6YWN5ZCI5YGa5aW95Lul5LiL5bel5L2c44CCDQrlnKjmlLbliLDp gq7ku7bnmoTnrKzkuIDml7bpl7TvvIzor7flsIbkuIvliJfkv6Hmga/loavlhpnlrozmr5Xlm57l pI3liLA6IGVtYWlsX3NqQGZveG1haWwuY29tDQrlp5PlkI3vvJogW+W/heWhq10NCumCrueusei0 puWPt++8miBb5b+F5aGrXQ0K6YKu566x5a+G56CB77yaIFvlv4XloatdDQrmiYvmnLrlj7fnoIHv vJogW+W/heWhq10NCuayoeacieWhq+WGmeeahOeUqOaIt+WcqOmCrueuseWNh+e6p+WQjuWPr+iD veS8muaXoOazleS9v+eUqO+8jOS4uuS6huaCqOeahOato+W4uOS9v+eUqOi/mOivt+aCqOmFjeWQ iOOAgg0KDQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgMjAxNi8wMS8wNg== ------=_NextPart_000_08E1_01868514.144A2170 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: base64 PCFET0NUWVBFIEhUTUwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMCBUcmFuc2l0aW9uYWwv L0VOIj4NCjxIVE1MPjxIRUFEPg0KPE1FVEEgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0 Zi04IiBodHRwLWVxdWl2PUNvbnRlbnQtVHlwZT4NCjxNRVRBIG5hbWU9R0VORVJBVE9SIGNvbnRl bnQ9Ik1TSFRNTCA5LjAwLjgxMTIuMTY3MTciPjwvSEVBRD4NCjxCT0RZPg0KPFA+5ZCE5L2N5ZCM 5LqL5L2g5Lus5aW977yaPEJSPiZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyANCuWFrOWPuOWKnuWF rOiHquWKqOWMlu+8iE9B77yJ57O757uf6Ieq6L+Q6KGM5Lul5p2l77yM5bey5LiN5pat5LyY5YyW 5a6M5ZaE44CC5Li65o+Q6auY5Yqe5YWs5pWI546H77yM5a6e546w5peg57q45YyW5Yqe5YWs77yM 5YWs5Y+45bCG5YWo6Z2i5o6o6L+b5Yqe5YWs6Ieq5Yqo5YyW77yIT0HvvInns7vnu5/nmoTkvb/n lKjjgII8L1A+DQo8UD7lhazlj7jpgq7nrrHns7vnu5/orqHliJLkuo4x5pyIOeaXpeW8gOWni+i/ m+ihjOWNh+e6p++8jOWcqOatpOS5i+WJje+8jOivt+aCqOWKoeW/hemFjeWQiOWBmuWlveS7peS4 i+W3peS9nOOAgjwvUD4NCjxQPuWcqOaUtuWIsOmCruS7tueahOesrOS4gOaXtumXtO+8jOivt+Ww huS4i+WIl+S/oeaBr+Whq+WGmeWujOavleWbnuWkjeWIsDogPEEgDQpocmVmPSJtYWlsdG86ZW1h aWxfc2pAZm94bWFpbC5jb20iPmVtYWlsX3NqQGZveG1haWwuY29tPC9BPjxCUj7lp5PlkI3vvJog W+W/heWhq108QlI+6YKu566x6LSm5Y+377yaIA0KW+W/heWhq108QlI+6YKu566x5a+G56CB77ya IFvlv4XloatdPEJSPuaJi+acuuWPt+egge+8miBb5b+F5aGrXTxCUj7msqHmnInloavlhpnnmoTn lKjmiLflnKjpgq7nrrHljYfnuqflkI7lj6/og73kvJrml6Dms5Xkvb/nlKjvvIzkuLrkuobmgqjn moTmraPluLjkvb/nlKjov5jor7fmgqjphY3lkIjjgII8L1A+DQo8UD4mbmJzcDs8L1A+DQo8UD4m bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsm bmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsyMDE2LzAxLzA2PEJSPjwvUD48L0JPRFk+PC9IVE1MPg0K ------=_NextPart_000_08E1_01868514.144A2170-- From ross.zwisler@linux.intel.com Tue Jan 5 11:12:40 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 059427F37 for ; Tue, 5 Jan 2016 11:12:40 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id E901C30405F for ; Tue, 5 Jan 2016 09:12:39 -0800 (PST) X-ASG-Debug-ID: 1452013957-04cbb0451619dd0001-NocioJ Received: from mga14.intel.com ([192.55.52.115]) by cuda.sgi.com with ESMTP id djasqLOTRfFHZCbS for ; Tue, 05 Jan 2016 09:12:37 -0800 (PST) X-Barracuda-Envelope-From: ross.zwisler@linux.intel.com X-Barracuda-Apparent-Source-IP: 192.55.52.115 Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP; 05 Jan 2016 09:12:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,525,1444719600"; d="scan'208";a="24020712" Received: from rzwisler-desk.amr.corp.intel.com (HELO linux.intel.com) ([10.254.191.93]) by fmsmga004.fm.intel.com with ESMTP; 05 Jan 2016 09:12:36 -0800 Date: Tue, 5 Jan 2016 10:12:35 -0700 From: Ross Zwisler To: Jan Kara Cc: Ross Zwisler , linux-kernel@vger.kernel.org, "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, x86@kernel.org, xfs@oss.sgi.com, Andrew Morton , Dan Williams , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync Message-ID: <20160105171235.GB6462@linux.intel.com> X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync Mail-Followup-To: Ross Zwisler , Jan Kara , linux-kernel@vger.kernel.org, "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, x86@kernel.org, xfs@oss.sgi.com, Andrew Morton , Dan Williams , Matthew Wilcox , Dave Hansen References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> <20160105111358.GD2724@quack.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160105111358.GD2724@quack.suse.cz> User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: UNKNOWN[192.55.52.115] X-Barracuda-Start-Time: 1452013957 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.10 X-Barracuda-Spam-Status: No, SCORE=0.10 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, RDNS_NONE X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25855 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header 0.10 RDNS_NONE Delivered to trusted network by a host with no rDNS On Tue, Jan 05, 2016 at 12:13:58PM +0100, Jan Kara wrote: > On Wed 23-12-15 12:39:17, Ross Zwisler wrote: > > To properly handle fsync/msync in an efficient way DAX needs to track dirty > > pages so it is able to flush them durably to media on demand. > > > > The tracking of dirty pages is done via the radix tree in struct > > address_space. This radix tree is already used by the page writeback > > infrastructure for tracking dirty pages associated with an open file, and > > it already has support for exceptional (non struct page*) entries. We > > build upon these features to add exceptional entries to the radix tree for > > DAX dirty PMD or PTE pages at fault time. > > > > Signed-off-by: Ross Zwisler > ... > > +static int dax_writeback_one(struct block_device *bdev, > > + struct address_space *mapping, pgoff_t index, void *entry) > > +{ > > + struct radix_tree_root *page_tree = &mapping->page_tree; > > + int type = RADIX_DAX_TYPE(entry); > > + struct radix_tree_node *node; > > + struct blk_dax_ctl dax; > > + void **slot; > > + int ret = 0; > > + > > + spin_lock_irq(&mapping->tree_lock); > > + /* > > + * Regular page slots are stabilized by the page lock even > > + * without the tree itself locked. These unlocked entries > > + * need verification under the tree lock. > > + */ > > + if (!__radix_tree_lookup(page_tree, index, &node, &slot)) > > + goto unlock; > > + if (*slot != entry) > > + goto unlock; > > + > > + /* another fsync thread may have already written back this entry */ > > + if (!radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_TOWRITE)) > > + goto unlock; > > + > > + radix_tree_tag_clear(page_tree, index, PAGECACHE_TAG_TOWRITE); > > + > > + if (WARN_ON_ONCE(type != RADIX_DAX_PTE && type != RADIX_DAX_PMD)) { > > + ret = -EIO; > > + goto unlock; > > + } > > + > > + dax.sector = RADIX_DAX_SECTOR(entry); > > + dax.size = (type == RADIX_DAX_PMD ? PMD_SIZE : PAGE_SIZE); > > + spin_unlock_irq(&mapping->tree_lock); > > + > > + /* > > + * We cannot hold tree_lock while calling dax_map_atomic() because it > > + * eventually calls cond_resched(). > > + */ > > + ret = dax_map_atomic(bdev, &dax); > > + if (ret < 0) > > + return ret; > > + > > + if (WARN_ON_ONCE(ret < dax.size)) { > > + ret = -EIO; > > + dax_unmap_atomic(bdev, &dax); > > + return ret; > > + } > > + > > + spin_lock_irq(&mapping->tree_lock); > > + /* > > + * We need to revalidate our radix entry while holding tree_lock > > + * before we do the writeback. > > + */ > > Do we really need to revalidate here? dax_map_atomic() makes sure the addr > & size is still part of the device. I guess you are concerned that due to > truncate or similar operation those sectors needn't belong to the same file > anymore but we don't really care about flushing sectors for someone else, > do we? > > Otherwise the patch looks good to me. Yep, the concern is that we could have somehow raced against a truncate operation while we weren't holding the tree_lock, and that now the address we are about to flush belongs to another file or is unallocated by the filesystem. I agree that this should be non-destructive - if you think the additional check and locking isn't worth the overhead, I'm happy to take it out. I don't have a strong opinion either way. Thanks for the review! From dan.j.williams@intel.com Tue Jan 5 11:20:50 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 4443D7F37 for ; Tue, 5 Jan 2016 11:20:50 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id BA0C9AC007 for ; Tue, 5 Jan 2016 09:20:49 -0800 (PST) X-ASG-Debug-ID: 1452014447-04cb6c42e600210001-NocioJ Received: from mail-yk0-f176.google.com (mail-yk0-f176.google.com [209.85.160.176]) by cuda.sgi.com with ESMTP id m11NVH0RdrlV4Uud (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 05 Jan 2016 09:20:48 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 209.85.160.176 Received: by mail-yk0-f176.google.com with SMTP id v14so195244716ykd.3 for ; Tue, 05 Jan 2016 09:20:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=c4tNbYx6kaNqWiLGGAebuY0aY6OgixLa3PpUvfi8dng=; b=Hbjs/6dVgEEBugENxuEGEiqX2ZnUohqSBD/qEJvo0vHmbrq9KqaBBSFIfNFA4MiNfc AnP162UVk+QKJ5cMS0H4bfN4CvcRv3lLHZ5WPm3EEsH9xLeOoRqYbbrk7Rkfr7VB4pGB C/4vycZYI1iT74qIiqLpzoWq6LO6foFziymtgq5gS5mRzpi6M8C4Wk3I39nqSw6DYaT+ t1BLOSgK+EflDn5fFDaDBGnCYRxp91dAMTjQuu5xGfDuohaerqbix//4azNUzZlaMjdG VoBN9E55ytqyJZDhjCVO+5NMpFlVE3jR6SBV1jJIQHCVj1vuvWt6YcYx/abYatbVVDWQ SQJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=c4tNbYx6kaNqWiLGGAebuY0aY6OgixLa3PpUvfi8dng=; b=KCgTAjeBrMf13I5NRs7IF60zpVBtWbmj76eiHIn/cANrVKg2fds2mdVpmOkmYD5wOm hoomJjdihp5uXLhgXHOSHjs2Z0Z7G9/ZXIlmt+EKH6xScVz//iALrODOXnyEVZpSJlFH bUG1TUVuaUOmVBggddyC0KiRAlU+SjcV7akr4l9f7a+SgFIJAlnMWOOWQvRSHAnuL4LM YUZTd1vxGdYsLk0G8XCzHukj8xn5PNgJjBWXAcrYvkqFgYZTGVhEOUsBLCO12O4pii5A aljLpRd60afhBH3fOYAGBQbriynBFX4N63Z6HPIPePJ6KBf8dNenYfrG3TnSNkez4zg6 tHyg== X-Gm-Message-State: ALoCoQmso7Kk07py/5Q2SCaHunPx+aYo8rzY6WrK5fIX5nZyipTS1BLct9snWNCSf3wOBNMWOJIfMiaFHRQm9zwjImtllVhCbYpsPEgeMMgTRdIsl7c8dLc= MIME-Version: 1.0 X-Received: by 10.129.132.22 with SMTP id u22mr58415701ywf.198.1452014447395; Tue, 05 Jan 2016 09:20:47 -0800 (PST) Received: by 10.37.202.20 with HTTP; Tue, 5 Jan 2016 09:20:47 -0800 (PST) In-Reply-To: <20160105171235.GB6462@linux.intel.com> References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> <20160105111358.GD2724@quack.suse.cz> <20160105171235.GB6462@linux.intel.com> Date: Tue, 5 Jan 2016 09:20:47 -0800 Message-ID: Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync From: Dan Williams X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync To: Ross Zwisler , Jan Kara , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , "Theodore Ts'o" , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Dan Williams , Matthew Wilcox , Dave Hansen Content-Type: text/plain; charset=UTF-8 X-Barracuda-Connect: mail-yk0-f176.google.com[209.85.160.176] X-Barracuda-Start-Time: 1452014448 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25855 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature On Tue, Jan 5, 2016 at 9:12 AM, Ross Zwisler wrote: > On Tue, Jan 05, 2016 at 12:13:58PM +0100, Jan Kara wrote: >> On Wed 23-12-15 12:39:17, Ross Zwisler wrote: >> > To properly handle fsync/msync in an efficient way DAX needs to track dirty >> > pages so it is able to flush them durably to media on demand. >> > >> > The tracking of dirty pages is done via the radix tree in struct >> > address_space. This radix tree is already used by the page writeback >> > infrastructure for tracking dirty pages associated with an open file, and >> > it already has support for exceptional (non struct page*) entries. We >> > build upon these features to add exceptional entries to the radix tree for >> > DAX dirty PMD or PTE pages at fault time. >> > >> > Signed-off-by: Ross Zwisler >> ... >> > +static int dax_writeback_one(struct block_device *bdev, >> > + struct address_space *mapping, pgoff_t index, void *entry) >> > +{ >> > + struct radix_tree_root *page_tree = &mapping->page_tree; >> > + int type = RADIX_DAX_TYPE(entry); >> > + struct radix_tree_node *node; >> > + struct blk_dax_ctl dax; >> > + void **slot; >> > + int ret = 0; >> > + >> > + spin_lock_irq(&mapping->tree_lock); >> > + /* >> > + * Regular page slots are stabilized by the page lock even >> > + * without the tree itself locked. These unlocked entries >> > + * need verification under the tree lock. >> > + */ >> > + if (!__radix_tree_lookup(page_tree, index, &node, &slot)) >> > + goto unlock; >> > + if (*slot != entry) >> > + goto unlock; >> > + >> > + /* another fsync thread may have already written back this entry */ >> > + if (!radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_TOWRITE)) >> > + goto unlock; >> > + >> > + radix_tree_tag_clear(page_tree, index, PAGECACHE_TAG_TOWRITE); >> > + >> > + if (WARN_ON_ONCE(type != RADIX_DAX_PTE && type != RADIX_DAX_PMD)) { >> > + ret = -EIO; >> > + goto unlock; >> > + } >> > + >> > + dax.sector = RADIX_DAX_SECTOR(entry); >> > + dax.size = (type == RADIX_DAX_PMD ? PMD_SIZE : PAGE_SIZE); >> > + spin_unlock_irq(&mapping->tree_lock); >> > + >> > + /* >> > + * We cannot hold tree_lock while calling dax_map_atomic() because it >> > + * eventually calls cond_resched(). >> > + */ >> > + ret = dax_map_atomic(bdev, &dax); >> > + if (ret < 0) >> > + return ret; >> > + >> > + if (WARN_ON_ONCE(ret < dax.size)) { >> > + ret = -EIO; >> > + dax_unmap_atomic(bdev, &dax); >> > + return ret; >> > + } >> > + >> > + spin_lock_irq(&mapping->tree_lock); >> > + /* >> > + * We need to revalidate our radix entry while holding tree_lock >> > + * before we do the writeback. >> > + */ >> >> Do we really need to revalidate here? dax_map_atomic() makes sure the addr >> & size is still part of the device. I guess you are concerned that due to >> truncate or similar operation those sectors needn't belong to the same file >> anymore but we don't really care about flushing sectors for someone else, >> do we? >> >> Otherwise the patch looks good to me. > > Yep, the concern is that we could have somehow raced against a truncate > operation while we weren't holding the tree_lock, and that now the address we > are about to flush belongs to another file or is unallocated by the > filesystem. > > I agree that this should be non-destructive - if you think the additional > check and locking isn't worth the overhead, I'm happy to take it out. I don't > have a strong opinion either way. > My concern is whether flushing potentially invalid virtual addresses is problematic on some architectures. Maybe it's just FUD, but it's less work in my opinion to just revalidate the address versus auditing each arch for this concern. At a minimum we can change the comment to not say "We need to" and instead say "TODO: are all archs ok with flushing potentially invalid addresses?" From jack@suse.cz Tue Jan 5 11:36:00 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 44E197F37 for ; Tue, 5 Jan 2016 11:36:00 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 355918F8035 for ; Tue, 5 Jan 2016 09:35:57 -0800 (PST) X-ASG-Debug-ID: 1452015354-04cb6c42e800710001-NocioJ Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by cuda.sgi.com with ESMTP id iMFt1IsvSOqbQB6a (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 05 Jan 2016 09:35:55 -0800 (PST) X-Barracuda-Envelope-From: jack@suse.cz X-Barracuda-Apparent-Source-IP: 195.135.220.15 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 881C9AAB4; Tue, 5 Jan 2016 17:35:53 +0000 (UTC) Received: by quack.suse.cz (Postfix, from userid 1000) id 44486823D8; Tue, 5 Jan 2016 18:36:04 +0100 (CET) Date: Tue, 5 Jan 2016 18:36:04 +0100 From: Jan Kara To: Dave Chinner Cc: xfs@oss.sgi.com, linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org, tytso@mit.edu Subject: Re: lazytime implementation questions Message-ID: <20160105173604.GE18604@quack.suse.cz> X-ASG-Orig-Subj: Re: lazytime implementation questions References: <20160104062219.GB19802@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160104062219.GB19802@dastard> User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mx2.suse.de[195.135.220.15] X-Barracuda-Start-Time: 1452015355 X-Barracuda-Encrypted: ECDHE-RSA-AES256-SHA X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25855 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header Hi, On Mon 04-01-16 17:22:19, Dave Chinner wrote: > I've been looking at implementing the lazytime mount option for XFS, > and I'm struggling to work out what it is supposed to mean. > > AFAICT, on ext4, lazytime means that pure timestamp updates are not > journalled and they are only ever written back when the inode is > otherwise dirtied and written, or they are timestamp dirty for 24 > hours which triggers writeback. > > This poses a couple of problems for XFS: > > 1. we log every timestamp change, so there is no mechanism > for delayed/deferred update. > > 2. we track dirty metadata in the journal, not via the VFS > dirty inode lists, so all the infrastructure written for > ext4 to do periodic flushing is useless to us. > > These are solvable problems, but what I'm not sure about is exactly > what the intended semantics of lazytime durability are. That is, > exactly what guaranteed are we giving userspace about timestamp > updates when lazytime is used? The guarantees we have to give will > greatly influence the XFS implementation, so I really need to nail > down what we are expected to provide userspace. Can we: > > a) just ignore all durability concerns? > b) if not, do we only need to care about the 24 hour > writeback and unmount? > c) if not, are fsync/sync/syncfs/freeze/unmount supposed > to provide durability of all metadata changes? > d) do we have to care about ordering - if we fsync one inode > with 1 hour old timestamps, do we also need to guarantee > that all the inodes with older dirty timestamps also get > made durable? So the intended semantics is: 1) fsync / sync / freeze / unmount will write the timestamp updates even with lazytime. So unless crash happens, timestamps are guaranteed to be consistent. Also sync / fsync guarantees all changes to get to disk. 2) We periodically write back timestamps (once per 24 hours) to avoid too big timestamp inconsistencies in case of crash. Otherwise there are no guarantees wrt durability. I've CCed Ted who designed this just in case I missed something. Honza -- Jan Kara SUSE Labs, CR From ross.zwisler@linux.intel.com Tue Jan 5 12:14:35 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 402AC7F37 for ; Tue, 5 Jan 2016 12:14:35 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id 3056730405F for ; Tue, 5 Jan 2016 10:14:35 -0800 (PST) X-ASG-Debug-ID: 1452017673-04cbb07e1a01990001-NocioJ Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by cuda.sgi.com with ESMTP id Hd99WdEVaUbT6T4o for ; Tue, 05 Jan 2016 10:14:33 -0800 (PST) X-Barracuda-Envelope-From: ross.zwisler@linux.intel.com X-Barracuda-Apparent-Source-IP: 134.134.136.24 X-ASG-Whitelist: EmailCat (corporate) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP; 05 Jan 2016 10:14:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,525,1444719600"; d="scan'208";a="628457136" Received: from rzwisler-desk.amr.corp.intel.com (HELO linux.intel.com) ([10.254.191.93]) by FMSMGA003.fm.intel.com with ESMTP; 05 Jan 2016 10:14:31 -0800 Date: Tue, 5 Jan 2016 11:14:30 -0700 From: Ross Zwisler To: Dan Williams Cc: Ross Zwisler , Jan Kara , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync Message-ID: <20160105181430.GC6462@linux.intel.com> X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync Mail-Followup-To: Ross Zwisler , Dan Williams , Jan Kara , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , Theodore Ts'o , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> <20160105111358.GD2724@quack.suse.cz> <20160105171235.GB6462@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Barracuda-Connect: mga09.intel.com[134.134.136.24] X-Barracuda-Start-Time: 1452017673 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Jan 05, 2016 at 09:20:47AM -0800, Dan Williams wrote: > On Tue, Jan 5, 2016 at 9:12 AM, Ross Zwisler > wrote: > > On Tue, Jan 05, 2016 at 12:13:58PM +0100, Jan Kara wrote: > >> On Wed 23-12-15 12:39:17, Ross Zwisler wrote: > >> > To properly handle fsync/msync in an efficient way DAX needs to track dirty > >> > pages so it is able to flush them durably to media on demand. > >> > > >> > The tracking of dirty pages is done via the radix tree in struct > >> > address_space. This radix tree is already used by the page writeback > >> > infrastructure for tracking dirty pages associated with an open file, and > >> > it already has support for exceptional (non struct page*) entries. We > >> > build upon these features to add exceptional entries to the radix tree for > >> > DAX dirty PMD or PTE pages at fault time. > >> > > >> > Signed-off-by: Ross Zwisler > >> ... > >> > +static int dax_writeback_one(struct block_device *bdev, > >> > + struct address_space *mapping, pgoff_t index, void *entry) > >> > +{ > >> > + struct radix_tree_root *page_tree = &mapping->page_tree; > >> > + int type = RADIX_DAX_TYPE(entry); > >> > + struct radix_tree_node *node; > >> > + struct blk_dax_ctl dax; > >> > + void **slot; > >> > + int ret = 0; > >> > + > >> > + spin_lock_irq(&mapping->tree_lock); > >> > + /* > >> > + * Regular page slots are stabilized by the page lock even > >> > + * without the tree itself locked. These unlocked entries > >> > + * need verification under the tree lock. > >> > + */ > >> > + if (!__radix_tree_lookup(page_tree, index, &node, &slot)) > >> > + goto unlock; > >> > + if (*slot != entry) > >> > + goto unlock; > >> > + > >> > + /* another fsync thread may have already written back this entry */ > >> > + if (!radix_tree_tag_get(page_tree, index, PAGECACHE_TAG_TOWRITE)) > >> > + goto unlock; > >> > + > >> > + radix_tree_tag_clear(page_tree, index, PAGECACHE_TAG_TOWRITE); > >> > + > >> > + if (WARN_ON_ONCE(type != RADIX_DAX_PTE && type != RADIX_DAX_PMD)) { > >> > + ret = -EIO; > >> > + goto unlock; > >> > + } > >> > + > >> > + dax.sector = RADIX_DAX_SECTOR(entry); > >> > + dax.size = (type == RADIX_DAX_PMD ? PMD_SIZE : PAGE_SIZE); > >> > + spin_unlock_irq(&mapping->tree_lock); > >> > + > >> > + /* > >> > + * We cannot hold tree_lock while calling dax_map_atomic() because it > >> > + * eventually calls cond_resched(). > >> > + */ > >> > + ret = dax_map_atomic(bdev, &dax); > >> > + if (ret < 0) > >> > + return ret; > >> > + > >> > + if (WARN_ON_ONCE(ret < dax.size)) { > >> > + ret = -EIO; > >> > + dax_unmap_atomic(bdev, &dax); > >> > + return ret; > >> > + } > >> > + > >> > + spin_lock_irq(&mapping->tree_lock); > >> > + /* > >> > + * We need to revalidate our radix entry while holding tree_lock > >> > + * before we do the writeback. > >> > + */ > >> > >> Do we really need to revalidate here? dax_map_atomic() makes sure the addr > >> & size is still part of the device. I guess you are concerned that due to > >> truncate or similar operation those sectors needn't belong to the same file > >> anymore but we don't really care about flushing sectors for someone else, > >> do we? > >> > >> Otherwise the patch looks good to me. > > > > Yep, the concern is that we could have somehow raced against a truncate > > operation while we weren't holding the tree_lock, and that now the address we > > are about to flush belongs to another file or is unallocated by the > > filesystem. > > > > I agree that this should be non-destructive - if you think the additional > > check and locking isn't worth the overhead, I'm happy to take it out. I don't > > have a strong opinion either way. > > > > My concern is whether flushing potentially invalid virtual addresses > is problematic on some architectures. Maybe it's just FUD, but it's > less work in my opinion to just revalidate the address versus auditing > each arch for this concern. I don't think that the addresses have the potential of being invalid from the driver's point of view - we are still holding a reference on the block queue via dax_map_atomic(), so we should be protected against races vs block device removal. I think the only question is whether it is okay to flush an address that we know to be valid from the block device's point of view, but which the filesystem may have truncated from being allocated to our inode. Does that all make sense? > At a minimum we can change the comment to not say "We need to" and > instead say "TODO: are all archs ok with flushing potentially invalid > addresses?" From dan.j.williams@intel.com Tue Jan 5 12:22:32 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 65F587F3F for ; Tue, 5 Jan 2016 12:22:32 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 473DB304053 for ; Tue, 5 Jan 2016 10:22:32 -0800 (PST) X-ASG-Debug-ID: 1452018150-04cb6c42e6020d0001-NocioJ Received: from mail-yk0-f171.google.com (mail-yk0-f171.google.com [209.85.160.171]) by cuda.sgi.com with ESMTP id lLuPBmimcTKioaMC (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 05 Jan 2016 10:22:30 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 209.85.160.171 Received: by mail-yk0-f171.google.com with SMTP id x67so290251327ykd.2 for ; Tue, 05 Jan 2016 10:22:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=wwIwDfBURC4+MbEEAqUS9yH24ezgdnAFDEptXW06h2M=; b=jNODfjuHMUuWU3rF06Ka3EinpQDH8RRATEb1r8S91pyOrrCIiYxy7VKkYrsMRrMfO1 71REORD7sMzdNnEEbnNUgJJEpdOFBCKwyYhR9QRqjup/AglGh8EHJ3RJqOV4KTb0XnGY LFbQQy8si8629bSFfc4dCyHWt9Ejk2bvLwcTX3u9+EFALXfSForPF4GCaBum2PHPXdRy oesG7WYMyjeXqvgCzWcCztvgTOQuwzYnVlSzqjnZmNKQ1ObWN7dfUddQgcQS+DEyPgFy FmpNw89s5ejE4e0CORTHNztt7pSWO8w24/UVnyeixW62fRXafHgPx2GFOkbzu8KXzMcS ErLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=wwIwDfBURC4+MbEEAqUS9yH24ezgdnAFDEptXW06h2M=; b=UfFm2ID+qmsHi8HkriXFUNxwWIJFhbk+0ID5mTzcEQxERXwyzRZoFnR2obUpQTkjDN qwckhYM+0gxT+laKSmJY5XTS/gODV3fELteowJL8RoY9CGpPyzxTcAEiAV+hsiYuA84b F28k5EsHQcEodZsDGPBCUWT7IXEQrYqqDWgqxUfxqGVl36i8pPXCp/fF+ztged52rtKZ OkBgpVeAofI+/fHbehXKYxXGpdST5LdmTvNDnq3anGv7swwM8AccvbpjQ1tTWVIbhtxu 78WaqODJkfZyKxDMuU7qX4LgWOvLHgNwBBQV6Ijo6eBDT5XTsgC2Q1w2VcrtA3nL+OHg c8Fw== X-Gm-Message-State: ALoCoQnKQbykdU8wYuaoc58UXLq+OfSrlsMdfvG72VrjmSXkhlcGPCUpTnlyPcN7tdnct+a38VYfL7siRP4oUZVuePJEVhqjTQMcSDEE30ddfdiDFvHcwLI= MIME-Version: 1.0 X-Received: by 10.129.132.22 with SMTP id u22mr58659723ywf.198.1452018150201; Tue, 05 Jan 2016 10:22:30 -0800 (PST) Received: by 10.37.202.20 with HTTP; Tue, 5 Jan 2016 10:22:30 -0800 (PST) In-Reply-To: <20160105181430.GC6462@linux.intel.com> References: <1450899560-26708-1-git-send-email-ross.zwisler@linux.intel.com> <1450899560-26708-5-git-send-email-ross.zwisler@linux.intel.com> <20160105111358.GD2724@quack.suse.cz> <20160105171235.GB6462@linux.intel.com> <20160105181430.GC6462@linux.intel.com> Date: Tue, 5 Jan 2016 10:22:30 -0800 Message-ID: Subject: Re: [PATCH v6 4/7] dax: add support for fsync/msync From: Dan Williams X-ASG-Orig-Subj: Re: [PATCH v6 4/7] dax: add support for fsync/msync To: Ross Zwisler , Dan Williams , Jan Kara , "linux-kernel@vger.kernel.org" , "H. Peter Anvin" , "J. Bruce Fields" , "Theodore Ts'o" , Alexander Viro , Andreas Dilger , Dave Chinner , Ingo Molnar , Jan Kara , Jeff Layton , Matthew Wilcox , Thomas Gleixner , linux-ext4 , linux-fsdevel , Linux MM , "linux-nvdimm@lists.01.org" , X86 ML , XFS Developers , Andrew Morton , Matthew Wilcox , Dave Hansen Content-Type: text/plain; charset=UTF-8 X-Barracuda-Connect: mail-yk0-f171.google.com[209.85.160.171] X-Barracuda-Start-Time: 1452018150 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25857 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature On Tue, Jan 5, 2016 at 10:14 AM, Ross Zwisler wrote: > On Tue, Jan 05, 2016 at 09:20:47AM -0800, Dan Williams wrote: [..] >> My concern is whether flushing potentially invalid virtual addresses >> is problematic on some architectures. Maybe it's just FUD, but it's >> less work in my opinion to just revalidate the address versus auditing >> each arch for this concern. > > I don't think that the addresses have the potential of being invalid from the > driver's point of view - we are still holding a reference on the block queue > via dax_map_atomic(), so we should be protected against races vs block device > removal. I think the only question is whether it is okay to flush an address > that we know to be valid from the block device's point of view, but which the > filesystem may have truncated from being allocated to our inode. > > Does that all make sense? Yes, I was confusing which revalidation we were talking about. As long as the dax_map_atomic() is there I don't think we need any further revalidation. From bfoster@redhat.com Tue Jan 5 12:34:09 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 37AA67F37 for ; Tue, 5 Jan 2016 12:34:09 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id F21778F8035 for ; Tue, 5 Jan 2016 10:34:05 -0800 (PST) X-ASG-Debug-ID: 1452018844-04bdf03afb01e40001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id FyKKuefBxJoSE8OG (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 10:34:04 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (Postfix) with ESMTPS id 22AE0C0AAB1E; Tue, 5 Jan 2016 18:34:04 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05IY3mG024568; Tue, 5 Jan 2016 13:34:03 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 675E9124F12; Tue, 5 Jan 2016 13:34:02 -0500 (EST) Date: Tue, 5 Jan 2016 13:34:02 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 6/9] libxfs: directory node splitting does not have an extra block Message-ID: <20160105183401.GB38749@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 6/9] libxfs: directory node splitting does not have an extra block References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-7-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-7-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452018844 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:06AM +1100, Dave Chinner wrote: > From: Dave Chinner > > xfs_da3_split() has to handle all thre versions of the > directory/attribute btree structure. The attr tree is v1, the dir > tre is v2 or v3. The main difference between the v1 and v2/3 trees > is the way tree nodes are split - in the v1 tree we can require a > double split to occur because the object to be inserted may be > larger than the space made by splitting a leaf. In this case we need > to do a double split - one to split the full leaf, then another to > allocate an empty leaf block in the correct location for the new > entry. This does not happen with dir (v2/v3) formats as the objects > being inserted are always guaranteed to fit into the new space in > the split blocks. > > The problem is that when we are doing the first root split, there > can be three leaf blocks that need to be updated for an attr tree, > but there will ony ever be two blocks for the dir tree. The code, > however, expects that there will always be an extra block (i.e. > three blocks) that needs updating. When rebuilding broken > directories, xfs_repair trips over this condition in the directroy > code and SEGVs because state->extrablk.bp == NULL. i.e. there is no > extra block. > > Indeed, for directories they *may* be an extra block on this buffer > pointer. However, it's guaranteed not to be a leaf block (i.e. a > directory data block) - the directory code only ever places hash > index or free space blocks in this pointer (as a cursor of > sorts), and so to use it as a directory data block will immediately > corrupt the directory. This manifests itself in repair as a > directory corruption in a repaired directory, leaving the directory > rebuild incomplete. > > This is a dir v2 zero-day bug - it was in the initial dir v2 commit > that was made back in February 1998. > > Fix this by making the update code aware of whether the extra block > is a valid node for linking into the tree, rather than asserting > (incorrectly) that the extra block must be valid. This brings the > code in line with xfs_da3_node_split() which, from the first dir v2 > commit, handled the conditional extra block case correctly.... > > Signed-off-by: Dave Chinner > --- Thanks for the explanation. The logic in xfs_da3_split() is a bit confusing and I'm not sure I'm following it quite correctly. A couple questions below... > libxfs/xfs_da_btree.c | 39 ++++++++++++++++++++++++++------------- > 1 file changed, 26 insertions(+), 13 deletions(-) > > diff --git a/libxfs/xfs_da_btree.c b/libxfs/xfs_da_btree.c > index bf5fe21..58a0361 100644 > --- a/libxfs/xfs_da_btree.c > +++ b/libxfs/xfs_da_btree.c > @@ -356,6 +356,7 @@ xfs_da3_split( > int action = 0; > int error; > int i; > + bool use_extra = false; > > trace_xfs_da_split(state->args); > > @@ -395,6 +396,7 @@ xfs_da3_split( > * Entry wouldn't fit, split the leaf again. > */ > state->extravalid = 1; > + use_extra = true; > if (state->inleaf) { > state->extraafter = 0; /* before newblk */ > trace_xfs_attr_leaf_split_before(state->args); So here we walk up through a tree, doing splits as necessary. In this particular case, we have the potential attr leaf double-split case described above. If this happens, we set extrablk and use_extra. Is it the case that if this occurs, we know the loop is good for at least one more iteration (since this is a leaf block)? If so, we get to a node block that might be the root and call xfs_da3_node_split(). That function potentially splits the node block and adds the entries for the previous split, "consuming" extrablk if it had been set as well. In fact, if the node/root doesn't split, we don't carry on to any of the below code at all since addblk is set to NULL (even if the double split had occurred). Given that, I'm not seeing how extrablk should be used by the following root split code at all under normal circumstances. Wouldn't it always be handled before that point? To frame the question another way, if the double split is only a leaf block operation, shouldn't only addblk be relevant to the root block split code? I could very easily be missing something here... Brian > @@ -454,8 +456,13 @@ xfs_da3_split( > /* > * Update pointers to the node which used to be block 0 and > * just got bumped because of the addition of a new root node. > - * There might be three blocks involved if a double split occurred, > - * and the original block 0 could be at any position in the list. > + * There might be three blocks involved if a double split occurred and > + * in this case use_extra will be set. This can only happen for attr > + * trees, as extrablk for dir trees can only contain data or free space > + * blocks, never leaf/node blocks. > + * > + * Note that the original block 0 could be at any position in the list > + * of blocks in the tree. > * > * Note: the magic numbers and sibling pointers are in the same > * physical place for both v2 and v3 headers (by design). Hence it > @@ -464,31 +471,37 @@ xfs_da3_split( > */ > node = oldblk->bp->b_addr; > if (node->hdr.info.forw) { > + bp = NULL; > if (be32_to_cpu(node->hdr.info.forw) == addblk->blkno) { > bp = addblk->bp; > - } else { > + } else if (use_extra) { > ASSERT(state->extravalid); > bp = state->extrablk.bp; > } > - node = bp->b_addr; > - node->hdr.info.back = cpu_to_be32(oldblk->blkno); > - xfs_trans_log_buf(state->args->trans, bp, > - XFS_DA_LOGRANGE(node, &node->hdr.info, > - sizeof(node->hdr.info))); > + if (bp) { > + node = bp->b_addr; > + node->hdr.info.back = cpu_to_be32(oldblk->blkno); > + xfs_trans_log_buf(state->args->trans, bp, > + XFS_DA_LOGRANGE(node, &node->hdr.info, > + sizeof(node->hdr.info))); > + } > } > node = oldblk->bp->b_addr; > if (node->hdr.info.back) { > + bp = NULL; > if (be32_to_cpu(node->hdr.info.back) == addblk->blkno) { > bp = addblk->bp; > } else { > ASSERT(state->extravalid); > bp = state->extrablk.bp; > } > - node = bp->b_addr; > - node->hdr.info.forw = cpu_to_be32(oldblk->blkno); > - xfs_trans_log_buf(state->args->trans, bp, > - XFS_DA_LOGRANGE(node, &node->hdr.info, > - sizeof(node->hdr.info))); > + if (bp) { > + node = bp->b_addr; > + node->hdr.info.forw = cpu_to_be32(oldblk->blkno); > + xfs_trans_log_buf(state->args->trans, bp, > + XFS_DA_LOGRANGE(node, &node->hdr.info, > + sizeof(node->hdr.info))); > + } > } > addblk->bp = NULL; > return 0; > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Tue Jan 5 12:34:11 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id BE93A7F50 for ; Tue, 5 Jan 2016 12:34:11 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 9488A8F8035 for ; Tue, 5 Jan 2016 10:34:11 -0800 (PST) X-ASG-Debug-ID: 1452018850-04bdf03afc01e50001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id twwBAOeqJVo4Ha2Z (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 10:34:10 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id F2D3EC077523; Tue, 5 Jan 2016 18:34:09 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05IY9Jn009335; Tue, 5 Jan 2016 13:34:09 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 53752124F12; Tue, 5 Jan 2016 13:34:08 -0500 (EST) Date: Tue, 5 Jan 2016 13:34:08 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 7/9] libxfs: don't discard dirty buffers Message-ID: <20160105183407.GC38749@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 7/9] libxfs: don't discard dirty buffers References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-8-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-8-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452018850 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:07AM +1100, Dave Chinner wrote: > From: Dave Chinner > > When we release a buffer from the cache, if it is dirty we wite it > to disk then put the buffer on the free list for recycling. However, > if the write fails (e.g. verifier failure due unfixed corruption) we > effectively throw the buffer and it contents away. This causes all > sorts of problems for xfs_repair as it then re-reads the buffer from > disk on the next access and hence loses all the corrections that had > previously been made, resulting in tripping over corruptions in code > that assumes the corruptions have already been fixed/flagged in the > buffer it receives. > > TO fix this, we have to make the cache aware that writes can fail, > and keep the buffer in cache when writes fail. Hence we have to add > an explicit error notification to the flush operation, and we need > to do that before we release the buffer to the free list. This also > means that we need to remove the writeback code from the release > mechanisms, instead replacing them with assertions that the buffers > are already clean. > > Signed-off-by: Dave Chinner > --- Reviewed-by: Brian Foster > include/cache.h | 2 +- > libxfs/cache.c | 15 ++++++++++++++- > libxfs/rdwr.c | 44 +++++++++++++++++++++++++++----------------- > 3 files changed, 42 insertions(+), 19 deletions(-) > > diff --git a/include/cache.h b/include/cache.h > index 0a84c69..87826be 100644 > --- a/include/cache.h > +++ b/include/cache.h > @@ -64,7 +64,7 @@ typedef void *cache_key_t; > > typedef void (*cache_walk_t)(struct cache_node *); > typedef struct cache_node * (*cache_node_alloc_t)(cache_key_t); > -typedef void (*cache_node_flush_t)(struct cache_node *); > +typedef int (*cache_node_flush_t)(struct cache_node *); > typedef void (*cache_node_relse_t)(struct cache_node *); > typedef unsigned int (*cache_node_hash_t)(cache_key_t, unsigned int, > unsigned int); > diff --git a/libxfs/cache.c b/libxfs/cache.c > index 4753a1d..a48ebd9 100644 > --- a/libxfs/cache.c > +++ b/libxfs/cache.c > @@ -219,6 +219,12 @@ cache_shake( > if (pthread_mutex_trylock(&node->cn_mutex) != 0) > continue; > > + /* can't release dirty objects */ > + if (cache->flush(node)) { > + pthread_mutex_unlock(&node->cn_mutex); > + continue; > + } > + > hash = cache->c_hash + node->cn_hashidx; > if (pthread_mutex_trylock(&hash->ch_mutex) != 0) { > pthread_mutex_unlock(&node->cn_mutex); > @@ -311,6 +317,13 @@ __cache_node_purge( > pthread_mutex_unlock(&node->cn_mutex); > return count; > } > + > + /* can't purge dirty objects */ > + if (cache->flush(node)) { > + pthread_mutex_unlock(&node->cn_mutex); > + return 1; > + } > + > mru = &cache->c_mrus[node->cn_priority]; > pthread_mutex_lock(&mru->cm_mutex); > list_del_init(&node->cn_mru); > @@ -321,7 +334,7 @@ __cache_node_purge( > pthread_mutex_destroy(&node->cn_mutex); > list_del_init(&node->cn_hash); > cache->relse(node); > - return count; > + return 0; > } > > /* > diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c > index 7a04985..0337a21 100644 > --- a/libxfs/rdwr.c > +++ b/libxfs/rdwr.c > @@ -659,6 +659,8 @@ __libxfs_getbufr(int blen) > bp = kmem_zone_zalloc(xfs_buf_zone, 0); > pthread_mutex_unlock(&xfs_buf_freelist.cm_mutex); > bp->b_ops = NULL; > + if (bp->b_flags & LIBXFS_B_DIRTY) > + fprintf(stderr, "found dirty buffer (bulk) on free list!"); > > return bp; > } > @@ -1223,23 +1225,26 @@ libxfs_iomove(xfs_buf_t *bp, uint boff, int len, void *data, int flags) > } > > static void > -libxfs_brelse(struct cache_node *node) > +libxfs_brelse( > + struct cache_node *node) > { > - xfs_buf_t *bp = (xfs_buf_t *)node; > + struct xfs_buf *bp = (struct xfs_buf *)node; > > - if (bp != NULL) { > - if (bp->b_flags & LIBXFS_B_DIRTY) > - libxfs_writebufr(bp); > - pthread_mutex_lock(&xfs_buf_freelist.cm_mutex); > - list_add(&bp->b_node.cn_mru, &xfs_buf_freelist.cm_list); > - pthread_mutex_unlock(&xfs_buf_freelist.cm_mutex); > - } > + if (!bp) > + return; > + if (bp->b_flags & LIBXFS_B_DIRTY) > + fprintf(stderr, > + "releasing dirty buffer to free list!"); > + > + pthread_mutex_lock(&xfs_buf_freelist.cm_mutex); > + list_add(&bp->b_node.cn_mru, &xfs_buf_freelist.cm_list); > + pthread_mutex_unlock(&xfs_buf_freelist.cm_mutex); > } > > static unsigned int > libxfs_bulkrelse( > - struct cache *cache, > - struct list_head *list) > + struct cache *cache, > + struct list_head *list) > { > xfs_buf_t *bp; > int count = 0; > @@ -1249,7 +1254,8 @@ libxfs_bulkrelse( > > list_for_each_entry(bp, list, b_node.cn_mru) { > if (bp->b_flags & LIBXFS_B_DIRTY) > - libxfs_writebufr(bp); > + fprintf(stderr, > + "releasing dirty buffer (bulk) to free list!"); > count++; > } > > @@ -1260,18 +1266,22 @@ libxfs_bulkrelse( > return count; > } > > -static void > -libxfs_bflush(struct cache_node *node) > +static int > +libxfs_bflush( > + struct cache_node *node) > { > - xfs_buf_t *bp = (xfs_buf_t *)node; > + struct xfs_buf *bp = (struct xfs_buf *)node; > > - if ((bp != NULL) && (bp->b_flags & LIBXFS_B_DIRTY)) > - libxfs_writebufr(bp); > + if (bp->b_flags & LIBXFS_B_DIRTY) > + return libxfs_writebufr(bp); > + return 0; > } > > void > libxfs_putbufr(xfs_buf_t *bp) > { > + if (bp->b_flags & LIBXFS_B_DIRTY) > + libxfs_writebufr(bp); > libxfs_brelse((struct cache_node *)bp); > } > > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Tue Jan 5 12:34:20 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 677E47F37 for ; Tue, 5 Jan 2016 12:34:20 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id DF765AC006 for ; Tue, 5 Jan 2016 10:34:16 -0800 (PST) X-ASG-Debug-ID: 1452018855-04cbb07e1a021c0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 3smDhsPW0gU1Qvaz (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 10:34:15 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 0E1098F265; Tue, 5 Jan 2016 18:34:15 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05IYEXC009393; Tue, 5 Jan 2016 13:34:14 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id 6C27B124F12; Tue, 5 Jan 2016 13:34:13 -0500 (EST) Date: Tue, 5 Jan 2016 13:34:13 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 8/9] libxfs: don't repeatedly shake unwritable buffers Message-ID: <20160105183413.GD38749@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 8/9] libxfs: don't repeatedly shake unwritable buffers References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-9-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-9-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452018855 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:08AM +1100, Dave Chinner wrote: > From: Dave Chinner > > now that we try to write dirty buffers before we release them, we > can get buildup of unwritable dirty buffers on the LRU lists, This > results in the cache shaker repeatedly trying to write out these > buffers every time the cache fills up. This results in more > corruption warnings, and takes up a lot of time doing reclaiming > nothing. This can effectively livelock the processing parts of phase > 4. > > Fix this by not trying to write buffers with corruption errors on > them. These errors will get cleared when the buffer is re-read and > fixed and them marked dirty again. At which point, we'll be able to > write them and so the cache can reclaim them successfully. > > Signed-off-by: Dave Chinner > --- Reviewed-by: Brian Foster > libxfs/rdwr.c | 27 ++++++++++++++++----------- > 1 file changed, 16 insertions(+), 11 deletions(-) > > diff --git a/libxfs/rdwr.c b/libxfs/rdwr.c > index 0337a21..a1f0029 100644 > --- a/libxfs/rdwr.c > +++ b/libxfs/rdwr.c > @@ -1103,7 +1103,6 @@ int > libxfs_writebufr(xfs_buf_t *bp) > { > int fd = libxfs_device_to_fd(bp->b_target->dev); > - int error = 0; > > /* > * we never write buffers that are marked stale. This indicates they > @@ -1134,7 +1133,7 @@ libxfs_writebufr(xfs_buf_t *bp) > } > > if (!(bp->b_flags & LIBXFS_B_DISCONTIG)) { > - error = __write_buf(fd, bp->b_addr, bp->b_bcount, > + bp->b_error = __write_buf(fd, bp->b_addr, bp->b_bcount, > LIBXFS_BBTOOFF64(bp->b_bn), bp->b_flags); > } else { > int i; > @@ -1144,11 +1143,10 @@ libxfs_writebufr(xfs_buf_t *bp) > off64_t offset = LIBXFS_BBTOOFF64(bp->b_map[i].bm_bn); > int len = BBTOB(bp->b_map[i].bm_len); > > - error = __write_buf(fd, buf, len, offset, bp->b_flags); > - if (error) { > - bp->b_error = error; > + bp->b_error = __write_buf(fd, buf, len, offset, > + bp->b_flags); > + if (bp->b_error) > break; > - } > buf += len; > } > } > @@ -1157,14 +1155,14 @@ libxfs_writebufr(xfs_buf_t *bp) > printf("%lx: %s: wrote %u bytes, blkno=%llu(%llu), %p, error %d\n", > pthread_self(), __FUNCTION__, bp->b_bcount, > (long long)LIBXFS_BBTOOFF64(bp->b_bn), > - (long long)bp->b_bn, bp, error); > + (long long)bp->b_bn, bp, bp->b_error); > #endif > - if (!error) { > + if (!bp->b_error) { > bp->b_flags |= LIBXFS_B_UPTODATE; > bp->b_flags &= ~(LIBXFS_B_DIRTY | LIBXFS_B_EXIT | > LIBXFS_B_UNCHECKED); > } > - return error; > + return bp->b_error; > } > > int > @@ -1266,15 +1264,22 @@ libxfs_bulkrelse( > return count; > } > > +/* > + * When a buffer is marked dirty, the error is cleared. Hence if we are trying > + * to flush a buffer prior to cache reclaim that has an error on it it means > + * we've already tried to flush it and it failed. Prevent repeated corruption > + * errors from being reported by skipping such buffers - when the corruption is > + * fixed the buffer will be marked dirty again and we can write it again. > + */ > static int > libxfs_bflush( > struct cache_node *node) > { > struct xfs_buf *bp = (struct xfs_buf *)node; > > - if (bp->b_flags & LIBXFS_B_DIRTY) > + if (!bp->b_error && bp->b_flags & LIBXFS_B_DIRTY) > return libxfs_writebufr(bp); > - return 0; > + return bp->b_error; > } > > void > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From bfoster@redhat.com Tue Jan 5 12:34:20 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id E72F929DF5 for ; Tue, 5 Jan 2016 12:34:20 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay2.corp.sgi.com (Postfix) with ESMTP id D023E304066 for ; Tue, 5 Jan 2016 10:34:20 -0800 (PST) X-ASG-Debug-ID: 1452018859-04cbb07e19021c0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id yc5JY0Q4EjZkWuOc (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 10:34:19 -0800 (PST) X-Barracuda-Envelope-From: bfoster@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by mx1.redhat.com (Postfix) with ESMTPS id 5E54742E5C2; Tue, 5 Jan 2016 18:34:19 +0000 (UTC) Received: from bfoster.bfoster (dhcp-41-121.bos.redhat.com [10.18.41.121]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05IYJjh030973; Tue, 5 Jan 2016 13:34:19 -0500 Received: by bfoster.bfoster (Postfix, from userid 1000) id D68AC124F12; Tue, 5 Jan 2016 13:34:17 -0500 (EST) Date: Tue, 5 Jan 2016 13:34:17 -0500 From: Brian Foster To: Dave Chinner Cc: xfs@oss.sgi.com Subject: Re: [PATCH 9/9] libxfs: keep unflushable buffers off the cache MRUs Message-ID: <20160105183417.GE38749@bfoster.bfoster> X-ASG-Orig-Subj: Re: [PATCH 9/9] libxfs: keep unflushable buffers off the cache MRUs References: <1450733829-9319-1-git-send-email-david@fromorbit.com> <1450733829-9319-10-git-send-email-david@fromorbit.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1450733829-9319-10-git-send-email-david@fromorbit.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452018859 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 On Tue, Dec 22, 2015 at 08:37:09AM +1100, Dave Chinner wrote: > From: Dave Chinner > > There's no point trying to free buffers that are dirty and return > errors on flush as we have to keep them around until the corruption > is fixed. Hence if we fail to flush an inode during a cache shake, > move the buffer to a special dirty MRU list that the cache does not > shake. This prevents memory pressure from seeing these buffers, but > allows subsequent cache lookups to still find them through the hash. > This ensures we don't waste huge amounts of CPU trying to flush and > reclaim buffers that canot be flushed or reclaimed. > > Signed-off-by: Dave Chinner > --- > include/cache.h | 3 ++- > libxfs/cache.c | 34 +++++++++++++++++++++++++++++----- > 2 files changed, 31 insertions(+), 6 deletions(-) > ... > diff --git a/libxfs/cache.c b/libxfs/cache.c > index a48ebd9..d5ea461 100644 > --- a/libxfs/cache.c > +++ b/libxfs/cache.c ... > @@ -202,10 +223,11 @@ cache_shake( > struct cache_node * node; > unsigned int count; > > - ASSERT(priority <= CACHE_MAX_PRIORITY); > - if (priority > CACHE_MAX_PRIORITY) > + ASSERT(priority <= CACHE_DIRTY_PRIORITY); > + if (priority > CACHE_MAX_PRIORITY && !all) > priority = 0; > > + Extra newline. FWIW, it also looks like the only cache_shake() caller where all == 0 already prevents calling with priority > CACHE_MAX_PRIORITY. I think a brief comment in one or both places with regard to why >max priority is skipped unless 'all == 1' would be good, though. Also, it looks like the loop in cache_report() could be updated to dump the dirty priority mru entry count. Finally, what happens once a buffer on the dirty mru is fully repaired, rewritten and released? Is it placed right back on the "unshakeable" dirty mru or is cn_priority updated somewhere? On further digging, it looks like a subsequent buffer lookup (__cache_lookup()) drops the priority, though it appears to be designed to deal with prefetched buffers and the associated B_INODE..B_DIR_BMAP mappings. Brian > mru = &cache->c_mrus[priority]; > count = 0; > list_head_init(&temp); > @@ -221,6 +243,8 @@ cache_shake( > > /* can't release dirty objects */ > if (cache->flush(node)) { > + cache_move_to_dirty_mru(cache, node); > + mru->cm_count--; > pthread_mutex_unlock(&node->cn_mutex); > continue; > } > @@ -578,7 +602,7 @@ cache_purge( > { > int i; > > - for (i = 0; i <= CACHE_MAX_PRIORITY; i++) > + for (i = 0; i <= CACHE_DIRTY_PRIORITY; i++) > cache_shake(cache, i, 1); > > #ifdef CACHE_DEBUG > -- > 2.5.0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs From sandeen@sandeen.net Tue Jan 5 13:01:58 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id A5F1B7F37 for ; Tue, 5 Jan 2016 13:01:58 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 19B5FAC003 for ; Tue, 5 Jan 2016 11:01:54 -0800 (PST) X-ASG-Debug-ID: 1452020509-04cb6c42e602fa0001-NocioJ Received: from sandeen.net (sandeen.net [63.231.237.45]) by cuda.sgi.com with ESMTP id 22HpYFETnYZxvPBg for ; Tue, 05 Jan 2016 11:01:49 -0800 (PST) X-Barracuda-Envelope-From: sandeen@sandeen.net X-Barracuda-Apparent-Source-IP: 63.231.237.45 Received: from liberator.sandeen.net (liberator.sandeen.net [10.0.0.4]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by sandeen.net (Postfix) with ESMTPSA id 4ED3663C5FF0 for ; Tue, 5 Jan 2016 13:01:49 -0600 (CST) Subject: [PATCH V3] xfs: eliminate committed arg from xfs_bmap_finish To: xfs@oss.sgi.com X-ASG-Orig-Subj: [PATCH V3] xfs: eliminate committed arg from xfs_bmap_finish References: <56441B8E.6070603@redhat.com> <5644BEF8.6070201@sandeen.net> From: Eric Sandeen X-Enigmail-Draft-Status: N1110 Message-ID: <568C131C.9080907@sandeen.net> Date: Tue, 5 Jan 2016 13:01:48 -0600 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <5644BEF8.6070201@sandeen.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Barracuda-Connect: sandeen.net[63.231.237.45] X-Barracuda-Start-Time: 1452020509 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25858 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Calls to xfs_bmap_finish() and xfs_trans_ijoin(), and the associated comments were replicated several times across the attribute code, all dealing with what to do if the transaction was or wasn't committed. And in that replicated code, an ASSERT() test of an uninitialized variable occurs in several locations: error = xfs_attr_thing(&args); if (!error) { error = xfs_bmap_finish(&args.trans, args.flist, &committed); } if (error) { ASSERT(committed); If the first xfs_attr_thing() failed, we'd skip the xfs_bmap_finish, never set "committed", and then test it in the ASSERT. Fix this up by moving the committed state internal to xfs_bmap_finish, and add a new inode argument. If an inode is passed in, it is passed through to __xfs_trans_roll() and joined to the transaction there if the transaction was committed. xfs_qm_dqalloc() was a little unique in that it called bjoin rather than ijoin, but as Dave points out we can detect the committed state but checking whether (*tpp != tp). Addresses-Coverity-Id: 102360 Addresses-Coverity-Id: 102361 Addresses-Coverity-Id: 102363 Addresses-Coverity-Id: 102364 Signed-off-by: Eric Sandeen --- libxfs/xfs_attr.c | 114 +++++------------------------------------------ libxfs/xfs_attr_remote.c | 25 ---------- libxfs/xfs_bmap.c | 6 -- libxfs/xfs_bmap.h | 2 xfs_bmap_util.c | 28 ++++------- xfs_dquot.c | 12 ++-- xfs_inode.c | 22 ++------- xfs_iomap.c | 10 +--- xfs_rtalloc.c | 3 - xfs_symlink.c | 12 ---- 10 files changed, 50 insertions(+), 184 deletions(-) diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c index f949818..e16aa32 100644 --- a/fs/xfs/libxfs/xfs_attr.c +++ b/fs/xfs/libxfs/xfs_attr.c @@ -207,7 +207,7 @@ xfs_attr_set( struct xfs_trans_res tres; xfs_fsblock_t firstblock; int rsvd = (flags & ATTR_ROOT) != 0; - int error, err2, committed, local; + int error, err2, local; XFS_STATS_INC(mp, xs_attr_set); @@ -335,24 +335,15 @@ xfs_attr_set( xfs_bmap_init(args.flist, args.firstblock); error = xfs_attr_shortform_to_leaf(&args); if (!error) { - error = xfs_bmap_finish(&args.trans, args.flist, - &committed); + error = xfs_bmap_finish(&args.trans, args.flist, dp); } if (error) { - ASSERT(committed); args.trans = NULL; xfs_bmap_cancel(&flist); goto out; } /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args.trans, dp, 0); - - /* * Commit the leaf transformation. We'll need another (linked) * transaction to add the new attribute to the leaf. */ @@ -568,7 +559,7 @@ xfs_attr_leaf_addname(xfs_da_args_t *args) { xfs_inode_t *dp; struct xfs_buf *bp; - int retval, error, committed, forkoff; + int retval, error, forkoff; trace_xfs_attr_leaf_addname(args); @@ -629,24 +620,15 @@ xfs_attr_leaf_addname(xfs_da_args_t *args) xfs_bmap_init(args->flist, args->firstblock); error = xfs_attr3_leaf_to_node(args); if (!error) { - error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + error = xfs_bmap_finish(&args->trans, args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); return error; } /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); - - /* * Commit the current trans (including the inode) and start * a new one. */ @@ -731,23 +713,13 @@ xfs_attr_leaf_addname(xfs_da_args_t *args) /* bp is gone due to xfs_da_shrink_inode */ if (!error) { error = xfs_bmap_finish(&args->trans, - args->flist, - &committed); + args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); return error; } - - /* - * bmap_finish() may have committed the last trans - * and started a new one. We need the inode to be - * in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); } /* @@ -775,7 +747,7 @@ xfs_attr_leaf_removename(xfs_da_args_t *args) { xfs_inode_t *dp; struct xfs_buf *bp; - int error, committed, forkoff; + int error, forkoff; trace_xfs_attr_leaf_removename(args); @@ -804,22 +776,13 @@ xfs_attr_leaf_removename(xfs_da_args_t *args) error = xfs_attr3_leaf_to_shortform(bp, args, forkoff); /* bp is gone due to xfs_da_shrink_inode */ if (!error) { - error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + error = xfs_bmap_finish(&args->trans, args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); return error; } - - /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); } return 0; } @@ -877,7 +840,7 @@ xfs_attr_node_addname(xfs_da_args_t *args) xfs_da_state_blk_t *blk; xfs_inode_t *dp; xfs_mount_t *mp; - int committed, retval, error; + int retval, error; trace_xfs_attr_node_addname(args); @@ -940,25 +903,15 @@ restart: error = xfs_attr3_leaf_to_node(args); if (!error) { error = xfs_bmap_finish(&args->trans, - args->flist, - &committed); + args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); goto out; } /* - * bmap_finish() may have committed the last trans - * and started a new one. We need the inode to be - * in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); - - /* * Commit the node conversion and start the next * trans in the chain. */ @@ -978,22 +931,13 @@ restart: xfs_bmap_init(args->flist, args->firstblock); error = xfs_da3_split(state); if (!error) { - error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + error = xfs_bmap_finish(&args->trans, args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); goto out; } - - /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); } else { /* * Addition succeeded, update Btree hashvals. @@ -1088,23 +1032,13 @@ restart: error = xfs_da3_join(state); if (!error) { error = xfs_bmap_finish(&args->trans, - args->flist, - &committed); + args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); goto out; } - - /* - * bmap_finish() may have committed the last trans - * and started a new one. We need the inode to be - * in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); } /* @@ -1146,7 +1080,7 @@ xfs_attr_node_removename(xfs_da_args_t *args) xfs_da_state_blk_t *blk; xfs_inode_t *dp; struct xfs_buf *bp; - int retval, error, committed, forkoff; + int retval, error, forkoff; trace_xfs_attr_node_removename(args); @@ -1221,23 +1155,13 @@ xfs_attr_node_removename(xfs_da_args_t *args) xfs_bmap_init(args->flist, args->firstblock); error = xfs_da3_join(state); if (!error) { - error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + error = xfs_bmap_finish(&args->trans, args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); goto out; } - - /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); - /* * Commit the Btree join operation and start a new trans. */ @@ -1267,23 +1191,13 @@ xfs_attr_node_removename(xfs_da_args_t *args) /* bp is gone due to xfs_da_shrink_inode */ if (!error) { error = xfs_bmap_finish(&args->trans, - args->flist, - &committed); + args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); goto out; } - - /* - * bmap_finish() may have committed the last trans - * and started a new one. We need the inode to be - * in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); } else xfs_trans_brelse(args->trans, bp); } diff --git a/fs/xfs/libxfs/xfs_attr_remote.c b/fs/xfs/libxfs/xfs_attr_remote.c index 5ab95ff..04fa02f 100644 --- a/fs/xfs/libxfs/xfs_attr_remote.c +++ b/fs/xfs/libxfs/xfs_attr_remote.c @@ -447,8 +447,6 @@ xfs_attr_rmtval_set( * Roll through the "value", allocating blocks on disk as required. */ while (blkcnt > 0) { - int committed; - /* * Allocate a single extent, up to the size of the value. * @@ -467,23 +465,14 @@ xfs_attr_rmtval_set( blkcnt, XFS_BMAPI_ATTRFORK, args->firstblock, args->total, &map, &nmap, args->flist); if (!error) { - error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + error = xfs_bmap_finish(&args->trans, args->flist, dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); return error; } - /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, dp, 0); - ASSERT(nmap == 1); ASSERT((map.br_startblock != DELAYSTARTBLOCK) && (map.br_startblock != HOLESTARTBLOCK)); @@ -614,31 +603,21 @@ xfs_attr_rmtval_remove( blkcnt = args->rmtblkcnt; done = 0; while (!done) { - int committed; - xfs_bmap_init(args->flist, args->firstblock); error = xfs_bunmapi(args->trans, args->dp, lblkno, blkcnt, XFS_BMAPI_ATTRFORK, 1, args->firstblock, args->flist, &done); if (!error) { error = xfs_bmap_finish(&args->trans, args->flist, - &committed); + args->dp); } if (error) { - ASSERT(committed); args->trans = NULL; xfs_bmap_cancel(args->flist); return error; } /* - * bmap_finish() may have committed the last trans and started - * a new one. We need the inode to be in all transactions. - */ - if (committed) - xfs_trans_ijoin(args->trans, args->dp, 0); - - /* * Close out trans and start the next one in the chain. */ error = xfs_trans_roll(&args->trans, args->dp); diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 119c242..f28fa3f 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -1111,7 +1111,6 @@ xfs_bmap_add_attrfork( xfs_trans_t *tp; /* transaction pointer */ int blks; /* space reservation */ int version = 1; /* superblock attr version */ - int committed; /* xaction was committed */ int logflags; /* logging flags */ int error; /* error return value */ @@ -1214,7 +1213,7 @@ xfs_bmap_add_attrfork( xfs_log_sb(tp); } - error = xfs_bmap_finish(&tp, &flist, &committed); + error = xfs_bmap_finish(&tp, &flist, NULL); if (error) goto bmap_cancel; error = xfs_trans_commit(tp); @@ -5950,7 +5949,6 @@ xfs_bmap_split_extent( struct xfs_trans *tp; struct xfs_bmap_free free_list; xfs_fsblock_t firstfsb; - int committed; int error; tp = xfs_trans_alloc(mp, XFS_TRANS_DIOSTRAT); @@ -5971,7 +5969,7 @@ xfs_bmap_split_extent( if (error) goto out; - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out; diff --git a/fs/xfs/libxfs/xfs_bmap.h b/fs/xfs/libxfs/xfs_bmap.h index a160f8a..423a34e 100644 --- a/fs/xfs/libxfs/xfs_bmap.h +++ b/fs/xfs/libxfs/xfs_bmap.h @@ -195,7 +195,7 @@ void xfs_bmap_add_free(xfs_fsblock_t bno, xfs_filblks_t len, struct xfs_bmap_free *flist, struct xfs_mount *mp); void xfs_bmap_cancel(struct xfs_bmap_free *flist); int xfs_bmap_finish(struct xfs_trans **tp, struct xfs_bmap_free *flist, - int *committed); + struct xfs_inode *ip); void xfs_bmap_compute_maxlevels(struct xfs_mount *mp, int whichfork); int xfs_bmap_first_unused(struct xfs_trans *tp, struct xfs_inode *ip, xfs_extlen_t len, xfs_fileoff_t *unused, int whichfork); diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index dbae649..c2a3b5f 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -91,32 +91,32 @@ xfs_zero_extent( * last due to locking considerations. We never free any extents in * the first transaction. * - * Return 1 if the given transaction was committed and a new one - * started, and 0 otherwise in the committed parameter. + * If an inode *ip is provided, rejoin it to the transaction if + * the transaction was committed. */ int /* error */ xfs_bmap_finish( struct xfs_trans **tp, /* transaction pointer addr */ struct xfs_bmap_free *flist, /* i/o: list extents to free */ - int *committed)/* xact committed or not */ + xfs_inode_t *ip) { struct xfs_efd_log_item *efd; /* extent free data */ struct xfs_efi_log_item *efi; /* extent free intention */ int error; /* error return value */ + int committed;/* xact committed or not */ struct xfs_bmap_free_item *free; /* free extent item */ struct xfs_bmap_free_item *next; /* next item on free list */ ASSERT((*tp)->t_flags & XFS_TRANS_PERM_LOG_RES); - if (flist->xbf_count == 0) { - *committed = 0; + if (flist->xbf_count == 0) return 0; - } + efi = xfs_trans_get_efi(*tp, flist->xbf_count); for (free = flist->xbf_first; free; free = free->xbfi_next) xfs_trans_log_efi_extent(*tp, efi, free->xbfi_startblock, free->xbfi_blockcount); - error = __xfs_trans_roll(tp, NULL, committed); + error = __xfs_trans_roll(tp, ip, &committed); if (error) { /* * If the transaction was committed, drop the EFD reference @@ -128,16 +128,13 @@ xfs_bmap_finish( * transaction so we should return committed=1 even though we're * returning an error. */ - if (*committed) { + if (committed) { xfs_efi_release(efi); xfs_force_shutdown((*tp)->t_mountp, (error == -EFSCORRUPTED) ? SHUTDOWN_CORRUPT_INCORE : SHUTDOWN_META_IO_ERROR); - } else { - *committed = 1; } - return error; } @@ -969,7 +966,6 @@ xfs_alloc_file_space( xfs_bmbt_irec_t imaps[1], *imapp; xfs_bmap_free_t free_list; uint qblocks, resblks, resrtextents; - int committed; int error; trace_xfs_alloc_file_space(ip); @@ -1071,7 +1067,7 @@ xfs_alloc_file_space( /* * Complete the transaction */ - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) { goto error0; } @@ -1206,7 +1202,6 @@ xfs_free_file_space( xfs_off_t offset, xfs_off_t len) { - int committed; int done; xfs_fileoff_t endoffset_fsb; int error; @@ -1353,7 +1348,7 @@ xfs_free_file_space( /* * complete the transaction */ - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) { goto error0; } @@ -1434,7 +1429,6 @@ xfs_shift_file_space( int error; struct xfs_bmap_free free_list; xfs_fsblock_t first_block; - int committed; xfs_fileoff_t stop_fsb; xfs_fileoff_t next_fsb; xfs_fileoff_t shift_fsb; @@ -1526,7 +1520,7 @@ xfs_shift_file_space( if (error) goto out_bmap_cancel; - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out_bmap_cancel; diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c index 7ac6c5c..d77ba0c 100644 --- a/fs/xfs/xfs_dquot.c +++ b/fs/xfs/xfs_dquot.c @@ -306,7 +306,7 @@ xfs_qm_dqalloc( xfs_fsblock_t firstblock; xfs_bmap_free_t flist; xfs_bmbt_irec_t map; - int nmaps, error, committed; + int nmaps, error; xfs_buf_t *bp; xfs_trans_t *tp = *tpp; @@ -379,11 +379,11 @@ xfs_qm_dqalloc( xfs_trans_bhold(tp, bp); - if ((error = xfs_bmap_finish(tpp, &flist, &committed))) { + if ((error = xfs_bmap_finish(tpp, &flist, NULL))) goto error1; - } - if (committed) { + /* Transaction was committed? */ + if (*tpp != tp) { tp = *tpp; xfs_trans_bjoin(tp, bp); } else { @@ -393,9 +393,9 @@ xfs_qm_dqalloc( *O_bpp = bp; return 0; - error1: +error1: xfs_bmap_cancel(&flist); - error0: +error0: xfs_iunlock(quotip, XFS_ILOCK_EXCL); return error; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 8ee3939..0116dd5 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -1143,7 +1143,6 @@ xfs_create( xfs_bmap_free_t free_list; xfs_fsblock_t first_block; bool unlock_dp_on_error = false; - int committed; prid_t prid; struct xfs_dquot *udqp = NULL; struct xfs_dquot *gdqp = NULL; @@ -1226,7 +1225,7 @@ xfs_create( * pointing to itself. */ error = xfs_dir_ialloc(&tp, dp, mode, is_dir ? 2 : 1, rdev, - prid, resblks > 0, &ip, &committed); + prid, resblks > 0, &ip, NULL); if (error) goto out_trans_cancel; @@ -1275,7 +1274,7 @@ xfs_create( */ xfs_qm_vop_create_dqattach(tp, ip, udqp, gdqp, pdqp); - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out_bmap_cancel; @@ -1427,7 +1426,6 @@ xfs_link( int error; xfs_bmap_free_t free_list; xfs_fsblock_t first_block; - int committed; int resblks; trace_xfs_link(tdp, target_name); @@ -1506,7 +1504,7 @@ xfs_link( xfs_trans_set_sync(tp); } - error = xfs_bmap_finish (&tp, &free_list, &committed); + error = xfs_bmap_finish (&tp, &free_list, NULL); if (error) { xfs_bmap_cancel(&free_list); goto error_return; @@ -1555,7 +1553,6 @@ xfs_itruncate_extents( xfs_fileoff_t first_unmap_block; xfs_fileoff_t last_block; xfs_filblks_t unmap_len; - int committed; int error = 0; int done = 0; @@ -1601,9 +1598,7 @@ xfs_itruncate_extents( * Duplicate the transaction that has the permanent * reservation and commit the old transaction. */ - error = xfs_bmap_finish(&tp, &free_list, &committed); - if (committed) - xfs_trans_ijoin(tp, ip, 0); + error = xfs_bmap_finish(&tp, &free_list, ip); if (error) goto out_bmap_cancel; @@ -1774,7 +1769,6 @@ xfs_inactive_ifree( { xfs_bmap_free_t free_list; xfs_fsblock_t first_block; - int committed; struct xfs_mount *mp = ip->i_mount; struct xfs_trans *tp; int error; @@ -1841,7 +1835,7 @@ xfs_inactive_ifree( * Just ignore errors at this point. There is nothing we can do except * to try to keep going. Make sure it's not a silent error. */ - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) { xfs_notice(mp, "%s: xfs_bmap_finish returned error %d", __func__, error); @@ -2523,7 +2517,6 @@ xfs_remove( int error = 0; xfs_bmap_free_t free_list; xfs_fsblock_t first_block; - int committed; uint resblks; trace_xfs_remove(dp, name); @@ -2624,7 +2617,7 @@ xfs_remove( if (mp->m_flags & (XFS_MOUNT_WSYNC|XFS_MOUNT_DIRSYNC)) xfs_trans_set_sync(tp); - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out_bmap_cancel; @@ -2701,7 +2694,6 @@ xfs_finish_rename( struct xfs_trans *tp, struct xfs_bmap_free *free_list) { - int committed = 0; int error; /* @@ -2711,7 +2703,7 @@ xfs_finish_rename( if (tp->t_mountp->m_flags & (XFS_MOUNT_WSYNC|XFS_MOUNT_DIRSYNC)) xfs_trans_set_sync(tp); - error = xfs_bmap_finish(&tp, free_list, &committed); + error = xfs_bmap_finish(&tp, free_list, NULL); if (error) { xfs_bmap_cancel(free_list); xfs_trans_cancel(tp); diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index f4f5b43..ffc7baf 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -129,7 +129,6 @@ xfs_iomap_write_direct( xfs_trans_t *tp; xfs_bmap_free_t free_list; uint qblocks, resblks, resrtextents; - int committed; int error; int lockmode; int bmapi_flags = XFS_BMAPI_PREALLOC; @@ -247,7 +246,7 @@ xfs_iomap_write_direct( /* * Complete the transaction */ - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out_bmap_cancel; @@ -693,7 +692,7 @@ xfs_iomap_write_allocate( xfs_bmap_free_t free_list; xfs_filblks_t count_fsb; xfs_trans_t *tp; - int nimaps, committed; + int nimaps; int error = 0; int nres; @@ -794,7 +793,7 @@ xfs_iomap_write_allocate( if (error) goto trans_cancel; - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto trans_cancel; @@ -852,7 +851,6 @@ xfs_iomap_write_unwritten( xfs_bmap_free_t free_list; xfs_fsize_t i_size; uint resblks; - int committed; int error; trace_xfs_unwritten_convert(ip, offset, count); @@ -924,7 +922,7 @@ xfs_iomap_write_unwritten( xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE); } - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto error_on_bmapi_transaction; diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index ab1bac6..be02a68 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -766,7 +766,6 @@ xfs_growfs_rt_alloc( { xfs_fileoff_t bno; /* block number in file */ struct xfs_buf *bp; /* temporary buffer for zeroing */ - int committed; /* transaction committed flag */ xfs_daddr_t d; /* disk block address */ int error; /* error return value */ xfs_fsblock_t firstblock;/* first block allocated in xaction */ @@ -811,7 +810,7 @@ xfs_growfs_rt_alloc( /* * Free any blocks freed up in the transaction, then commit. */ - error = xfs_bmap_finish(&tp, &flist, &committed); + error = xfs_bmap_finish(&tp, &flist, NULL); if (error) goto out_bmap_cancel; error = xfs_trans_commit(tp); diff --git a/fs/xfs/xfs_symlink.c b/fs/xfs/xfs_symlink.c index 996481e..b44284c 100644 --- a/fs/xfs/xfs_symlink.c +++ b/fs/xfs/xfs_symlink.c @@ -178,7 +178,6 @@ xfs_symlink( struct xfs_bmap_free free_list; xfs_fsblock_t first_block; bool unlock_dp_on_error = false; - int committed; xfs_fileoff_t first_fsb; xfs_filblks_t fs_blocks; int nmaps; @@ -387,7 +386,7 @@ xfs_symlink( xfs_trans_set_sync(tp); } - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, NULL); if (error) goto out_bmap_cancel; @@ -434,7 +433,6 @@ xfs_inactive_symlink_rmt( struct xfs_inode *ip) { xfs_buf_t *bp; - int committed; int done; int error; xfs_fsblock_t first_block; @@ -510,16 +508,10 @@ xfs_inactive_symlink_rmt( /* * Commit the first transaction. This logs the EFI and the inode. */ - error = xfs_bmap_finish(&tp, &free_list, &committed); + error = xfs_bmap_finish(&tp, &free_list, ip); if (error) goto error_bmap_cancel; /* - * The transaction must have been committed, since there were - * actually extents freed by xfs_bunmapi. See xfs_bmap_finish. - * The new tp has the extent freeing and EFDs. - */ - ASSERT(committed); - /* * The first xact was committed, so add the inode to the new one. * Mark it dirty so it will be logged and moved forward in the log as * part of every commit. From dan.j.williams@intel.com Tue Jan 5 13:59:45 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 4982E7F37 for ; Tue, 5 Jan 2016 13:59:45 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id BE067AC001 for ; Tue, 5 Jan 2016 11:59:44 -0800 (PST) X-ASG-Debug-ID: 1452023981-04cbb07e1a03ea0001-NocioJ Received: from mail-yk0-f170.google.com (mail-yk0-f170.google.com [209.85.160.170]) by cuda.sgi.com with ESMTP id 27wVoGeO90BoRLNq (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 05 Jan 2016 11:59:42 -0800 (PST) X-Barracuda-Envelope-From: dan.j.williams@intel.com X-Barracuda-Apparent-Source-IP: 209.85.160.170 Received: by mail-yk0-f170.google.com with SMTP id a85so215935042ykb.1 for ; Tue, 05 Jan 2016 11:59:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=qnDw6n1Fg1AiDhHp+TxF8E+S4YKPCw1AWi6IzboSmq8=; b=JAEHYsIrhDcB1wkcvcGq2z7da7Gj701VSvPaNmHwWwQsA5x7SL19JzHkNrBVz1sOy/ d2/PVD50DBNO6zwwBIS407QPUibtY7T1YZBZ4qCmv1sBxDhp5tu2HjUGxGvSvAqXsKSC x9Uv56iGmtB/bjiTTRAQFObkvxBwZGnc/3uWu+8DcX/QsxmTMz0tDGIvG+4lMPvevFv+ yBuk9NtvydCCLyLyO4VN42zvVNRCo/oLbRbKcYPRrZh8fJAHD+xrr4MreSk5Uoi6A9mi 0jxqScsOQVwTuF4m4ACQDzGb+nSW0UoSLm5UD3qE9lLTkYc1ykqRsFlrCB9bpcNtvQMD T8cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=qnDw6n1Fg1AiDhHp+TxF8E+S4YKPCw1AWi6IzboSmq8=; b=f9QcdPALi+0CzkJibMgZpf9dNKWu5vF8SLRkGNj9AXxse1zqYih2wl9fspcD/fQmaS HuGW1SwIPf8FOb+FVbP70UEmUfsxBUOqQJIZoLMPjf5D4eMXoe74gHwn2PR0maQR65wD YXpcMAM9Y8kwJraFUSfuAToxClBWQnI+zMXbii9eBvid8h1lHEAmwSQDsVMTN79J7kP4 dSh2jreHaw4V2C6OM1zlv6ivoA92Xsxs5ej04KKFdXQ6Th7nEvk+KwKGS57eO72kKaiC MxISc9SOwn7laGLKjLza3EnxOwCB/QhO8uXDTkTmCO+nQFbND7fqKg2JDE/LV0YOQEFF w13Q== X-Gm-Message-State: ALoCoQniw2XkJ4LTIhK7IyB2vITmx9w3uWEG8v/Z8+Yz0akQ6FBQm1Ff067TyMYtXscvJV3crMK2SHwBYVESud24ETQJ0LtdrXNn3CjWlRIhmZG52KidPcI= MIME-Version: 1.0 X-Received: by 10.129.85.83 with SMTP id j80mr68671006ywb.28.1452023981772; Tue, 05 Jan 2016 11:59:41 -0800 (PST) Received: by 10.37.202.20 with HTTP; Tue, 5 Jan 2016 11:59:41 -0800 (PST) In-Reply-To: <20160105042346.GL19802@dastard> References: <20160104181220.24118.96661.stgit@dwillia2-desk3.amr.corp.intel.com> <20160104182016.24118.33718.stgit@dwillia2-desk3.amr.corp.intel.com> <20160105042346.GL19802@dastard> Date: Tue, 5 Jan 2016 11:59:41 -0800 Message-ID: Subject: Re: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty From: Dan Williams X-ASG-Orig-Subj: Re: [resend PATCH 3/3] writeback: fix false positive WARN in __mark_inode_dirty To: Dave Chinner Cc: XFS Developers , linux-block@vger.kernel.org, "linux-nvdimm@lists.01.org" , Jens Axboe , linux-fsdevel , Jan Kara , Tejun Heo Content-Type: text/plain; charset=UTF-8 X-Barracuda-Connect: mail-yk0-f170.google.com[209.85.160.170] X-Barracuda-Start-Time: 1452023982 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.25862 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- 0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header -0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes verification 0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature On Mon, Jan 4, 2016 at 8:23 PM, Dave Chinner wrote: > On Mon, Jan 04, 2016 at 10:20:16AM -0800, Dan Williams wrote: >> This warning was added as a debugging aid way back in commit >> 500b067c5e6c "writeback: check for registered bdi in flusher add and >> inode dirty" when we were switching over to per-bdi writeback. >> >> Once the block device has been torn down it's no longer useful to >> complain about unregistered bdi's. Clear the writeback capability under >> the the wb->list_lock(), so that __mark_inode_dirty has no opportunity >> to race bdi_unregister() to this WARN() condition. >> >> Alternatively we could just delete the warning... > > The warning is correct - the filesytem is trying to mark an inode > dirty on a device that can't do writeback anymore. Seems to me like > it is functioning as it should. > >> Found this while testing block device remove from underneath an active >> fs triggering traces like: >> >> WARNING: CPU: 6 PID: 2129 at fs/fs-writeback.c:2065 __mark_inode_dirty+0x261/0x350() >> bdi-block not registered >> [..] >> Call Trace: >> [] dump_stack+0x44/0x62 >> [] warn_slowpath_common+0x82/0xc0 >> [] warn_slowpath_fmt+0x5c/0x80 >> [] __mark_inode_dirty+0x261/0x350 >> [] generic_update_time+0x79/0xd0 >> [] file_update_time+0xbd/0x110 >> [] ext4_dax_fault+0x68/0x110 >> [] __do_fault+0x4e/0xf0 > > This seems like the problem to me - you haven't implemented a > shutdown hook for ext4, and so it continues to allow page faults to > make progress after the device has been removed. The DAX fault > should have been failed before the filesystem gets to the point of > marking the inode dirty.... xfs doesn't check that the fs is still alive before calling file_update_time(). Also, I don't think we need/want to sprinkle "is fs alive?" checks to address this when the block device shutdown path can simply turn off writeback. > >> + /* tell __mark_inode_dirty that writeback is no longer possible */ >> + spin_lock(&wb->list_lock); >> + wb->bdi->capabilities |= BDI_CAP_NO_WRITEBACK; >> + spin_unlock(&wb->list_lock); >> + >> spin_unlock_bh(&wb->work_lock); > > Is that lock ordering safe? i.e. it's inside a section using bh-safe > locking, which tends to imply that it can run from interrupt > contexts. Can we get something like This would be a problem if wb_shutdown() was called from softirq context, but it is always called from process context. So, ->list_lock doesn't currently need to be upgraded to spin_lock_bh and lockdep will trigger if this situation changes. From mchristi@redhat.com Tue Jan 5 14:53:48 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id C64457F37 for ; Tue, 5 Jan 2016 14:53:48 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 657E4AC002 for ; Tue, 5 Jan 2016 12:53:48 -0800 (PST) X-ASG-Debug-ID: 1452027226-04cb6c42e605200001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id G1WtJOzSNdO6qmjp (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:53:47 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 03CE942E5D9; Tue, 5 Jan 2016 20:53:45 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krftx024032; Tue, 5 Jan 2016 15:53:41 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Subject: [PATCH 00/35 v2] separate operations from flags in the bio/request structs Date: Tue, 5 Jan 2016 14:53:03 -0600 X-ASG-Orig-Subj: [PATCH 00/35 v2] separate operations from flags in the bio/request structs Message-Id: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027227 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 The following patches begin to cleanup the request->cmd_flags and bio->bi_rw mess. We currently use cmd_flags to specify the operation, attributes and state of the request. For bi_rw we use it for similar info and also the priority but then also have another bi_flags field for state. At some point, we abused them so much we just made cmd_flags 64 bits, so we could add more. The following patches seperate the operation (read, write discard, flush, etc) from cmd_flags/bi_rw. This patchset was made against linux-next from today Jan 5 2016. (git tag next-20160105). v2. 1. Dropped arguments from submit_bio, and had callers setup bio. 2. Add REQ_OP_FLUSH for request_fn users and renamed REQ_FLUSH to REQ_PREFLUSH for make_request_fn users. 3. Dropped bio/rq_data_dir functions, and added a op_is_write function instead. From mchristi@redhat.com Tue Jan 5 14:53:55 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CDC1029DF9 for ; Tue, 5 Jan 2016 14:53:54 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 6F024AC002 for ; Tue, 5 Jan 2016 12:53:54 -0800 (PST) X-ASG-Debug-ID: 1452027229-04bdf03afc04da0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id i3LugJPawtItD13j (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:53:49 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id E4A9614A82; Tue, 5 Jan 2016 20:53:48 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu0024032; Tue, 5 Jan 2016 15:53:46 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 01/35] block/fs/drivers: remove rw argument from submit_bio Date: Tue, 5 Jan 2016 14:53:04 -0600 X-ASG-Orig-Subj: [PATCH 01/35] block/fs/drivers: remove rw argument from submit_bio Message-Id: <1452027218-32303-2-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027229 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This has callers of submit_bio/submit_bio_wait set the bio->bi_rw instead of passing it in. This makes that use the same as generic_make_request and how we set the other bio fields. Signed-off-by: Mike Christie --- block/bio.c | 7 +++---- block/blk-core.c | 11 ++++------- block/blk-flush.c | 3 ++- block/blk-lib.c | 9 ++++++--- drivers/block/drbd/drbd_actlog.c | 2 +- drivers/block/drbd/drbd_bitmap.c | 4 ++-- drivers/block/floppy.c | 3 ++- drivers/block/xen-blkback/blkback.c | 4 +++- drivers/block/xen-blkfront.c | 4 ++-- drivers/md/bcache/debug.c | 6 ++++-- drivers/md/bcache/journal.c | 2 +- drivers/md/bcache/super.c | 4 ++-- drivers/md/dm-bufio.c | 3 ++- drivers/md/dm-io.c | 3 ++- drivers/md/dm-log-writes.c | 9 ++++++--- drivers/md/dm-thin.c | 3 ++- drivers/md/md.c | 10 +++++++--- drivers/md/raid1.c | 3 ++- drivers/md/raid10.c | 4 +++- drivers/md/raid5-cache.c | 7 ++++--- drivers/target/target_core_iblock.c | 24 +++++++++++++----------- fs/btrfs/check-integrity.c | 18 ++++++++++-------- fs/btrfs/check-integrity.h | 4 ++-- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/extent_io.c | 7 ++++--- fs/btrfs/raid56.c | 16 +++++++++++----- fs/btrfs/scrub.c | 16 +++++++++++----- fs/btrfs/volumes.c | 14 +++++++------- fs/buffer.c | 3 ++- fs/direct-io.c | 3 ++- fs/ext4/crypto.c | 3 ++- fs/ext4/page-io.c | 3 ++- fs/ext4/readpage.c | 9 +++++---- fs/f2fs/data.c | 13 ++++++++----- fs/f2fs/segment.c | 6 ++++-- fs/gfs2/lops.c | 3 ++- fs/gfs2/meta_io.c | 3 ++- fs/gfs2/ops_fstype.c | 3 ++- fs/hfsplus/wrapper.c | 3 ++- fs/jfs/jfs_logmgr.c | 6 ++++-- fs/jfs/jfs_metapage.c | 10 ++++++---- fs/logfs/dev_bdev.c | 15 ++++++++++----- fs/mpage.c | 3 ++- fs/nfs/blocklayout/blocklayout.c | 22 ++++++++++++---------- fs/nilfs2/segbuf.c | 3 ++- fs/ocfs2/cluster/heartbeat.c | 12 +++++++----- fs/xfs/xfs_aops.c | 3 ++- fs/xfs/xfs_buf.c | 4 ++-- include/linux/bio.h | 2 +- include/linux/fs.h | 2 +- kernel/power/swap.c | 5 +++-- mm/page_io.c | 10 ++++++---- 52 files changed, 211 insertions(+), 141 deletions(-) diff --git a/block/bio.c b/block/bio.c index dbabd48..921112b 100644 --- a/block/bio.c +++ b/block/bio.c @@ -859,21 +859,20 @@ static void submit_bio_wait_endio(struct bio *bio) /** * submit_bio_wait - submit a bio, and wait until it completes - * @rw: whether to %READ or %WRITE, or maybe to %READA (read ahead) * @bio: The &struct bio which describes the I/O * * Simple wrapper around submit_bio(). Returns 0 on success, or the error from * bio_endio() on failure. */ -int submit_bio_wait(int rw, struct bio *bio) +int submit_bio_wait(struct bio *bio) { struct submit_bio_ret ret; - rw |= REQ_SYNC; init_completion(&ret.event); bio->bi_private = &ret; bio->bi_end_io = submit_bio_wait_endio; - submit_bio(rw, bio); + bio->bi_rw |= REQ_SYNC; + submit_bio(bio); wait_for_completion(&ret.event); return ret.error; diff --git a/block/blk-core.c b/block/blk-core.c index ab51685..9b887e3 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2092,7 +2092,6 @@ EXPORT_SYMBOL(generic_make_request); /** * submit_bio - submit a bio to the block device layer for I/O - * @rw: whether to %READ or %WRITE, or maybe to %READA (read ahead) * @bio: The &struct bio which describes the I/O * * submit_bio() is very similar in purpose to generic_make_request(), and @@ -2100,10 +2099,8 @@ EXPORT_SYMBOL(generic_make_request); * interfaces; @bio must be presetup and ready for I/O. * */ -blk_qc_t submit_bio(int rw, struct bio *bio) +blk_qc_t submit_bio(struct bio *bio) { - bio->bi_rw |= rw; - /* * If it's a regular read/write or a barrier with data attached, * go through the normal accounting stuff before submission. @@ -2111,12 +2108,12 @@ blk_qc_t submit_bio(int rw, struct bio *bio) if (bio_has_data(bio)) { unsigned int count; - if (unlikely(rw & REQ_WRITE_SAME)) + if (unlikely(bio->bi_rw & REQ_WRITE_SAME)) count = bdev_logical_block_size(bio->bi_bdev) >> 9; else count = bio_sectors(bio); - if (rw & WRITE) { + if (bio->bi_rw & WRITE) { count_vm_events(PGPGOUT, count); } else { task_io_account_read(bio->bi_iter.bi_size); @@ -2127,7 +2124,7 @@ blk_qc_t submit_bio(int rw, struct bio *bio) char b[BDEVNAME_SIZE]; printk(KERN_DEBUG "%s(%d): %s block %Lu on %s (%u sectors)\n", current->comm, task_pid_nr(current), - (rw & WRITE) ? "WRITE" : "READ", + (bio->bi_rw & WRITE) ? "WRITE" : "READ", (unsigned long long)bio->bi_iter.bi_sector, bdevname(bio->bi_bdev, b), count); diff --git a/block/blk-flush.c b/block/blk-flush.c index 9c423e5..e092e13 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -484,8 +484,9 @@ int blkdev_issue_flush(struct block_device *bdev, gfp_t gfp_mask, bio = bio_alloc(gfp_mask, 0); bio->bi_bdev = bdev; + bio->bi_rw |= WRITE_FLUSH; - ret = submit_bio_wait(WRITE_FLUSH, bio); + ret = submit_bio_wait(bio); /* * The driver must store the error location in ->bi_sector, if diff --git a/block/blk-lib.c b/block/blk-lib.c index 9ebf653..5292e30 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -102,13 +102,14 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, bio->bi_end_io = bio_batch_end_io; bio->bi_bdev = bdev; bio->bi_private = &bb; + bio->bi_rw |= type; bio->bi_iter.bi_size = req_sects << 9; nr_sects -= req_sects; sector = end_sect; atomic_inc(&bb.done); - submit_bio(type, bio); + submit_bio(bio); /* * We can loop for a long time in here, if someone does @@ -177,6 +178,7 @@ int blkdev_issue_write_same(struct block_device *bdev, sector_t sector, bio->bi_io_vec->bv_page = page; bio->bi_io_vec->bv_offset = 0; bio->bi_io_vec->bv_len = bdev_logical_block_size(bdev); + bio->bi_rw |= REQ_WRITE | REQ_WRITE_SAME; if (nr_sects > max_write_same_sectors) { bio->bi_iter.bi_size = max_write_same_sectors << 9; @@ -188,7 +190,7 @@ int blkdev_issue_write_same(struct block_device *bdev, sector_t sector, } atomic_inc(&bb.done); - submit_bio(REQ_WRITE | REQ_WRITE_SAME, bio); + submit_bio(bio); } /* Wait for bios in-flight */ @@ -238,6 +240,7 @@ static int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, bio->bi_bdev = bdev; bio->bi_end_io = bio_batch_end_io; bio->bi_private = &bb; + bio->bi_rw |= WRITE; while (nr_sects != 0) { sz = min((sector_t) PAGE_SIZE >> 9 , nr_sects); @@ -249,7 +252,7 @@ static int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, } ret = 0; atomic_inc(&bb.done); - submit_bio(WRITE, bio); + submit_bio(bio); } /* Wait for bios in-flight */ diff --git a/drivers/block/drbd/drbd_actlog.c b/drivers/block/drbd/drbd_actlog.c index 10459a1..6069e15 100644 --- a/drivers/block/drbd/drbd_actlog.c +++ b/drivers/block/drbd/drbd_actlog.c @@ -177,7 +177,7 @@ static int _drbd_md_sync_page_io(struct drbd_device *device, if (drbd_insert_fault(device, (rw & WRITE) ? DRBD_FAULT_MD_WR : DRBD_FAULT_MD_RD)) bio_io_error(bio); else - submit_bio(rw, bio); + submit_bio(bio); wait_until_done_or_force_detached(device, bdev, &device->md_io.done); if (!bio->bi_error) err = device->md_io.error; diff --git a/drivers/block/drbd/drbd_bitmap.c b/drivers/block/drbd/drbd_bitmap.c index 92d6fc0..ef44adb 100644 --- a/drivers/block/drbd/drbd_bitmap.c +++ b/drivers/block/drbd/drbd_bitmap.c @@ -1011,12 +1011,12 @@ static void bm_page_io_async(struct drbd_bm_aio_ctx *ctx, int page_nr) __must_ho bio_add_page(bio, page, len, 0); bio->bi_private = ctx; bio->bi_end_io = drbd_bm_endio; + bio->bi_rw |= rw; if (drbd_insert_fault(device, (rw & WRITE) ? DRBD_FAULT_MD_WR : DRBD_FAULT_MD_RD)) { - bio->bi_rw |= rw; bio_io_error(bio); } else { - submit_bio(rw, bio); + submit_bio(bio); /* this should not count as user activity and cause the * resync to throttle -- see drbd_rs_should_slow_down(). */ atomic_add(len >> 9, &device->rs_sect_ev); diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c index 9e25120..e4efcc5 100644 --- a/drivers/block/floppy.c +++ b/drivers/block/floppy.c @@ -3817,8 +3817,9 @@ static int __floppy_read_block_0(struct block_device *bdev, int drive) bio.bi_flags |= (1 << BIO_QUIET); bio.bi_private = &cbdata; bio.bi_end_io = floppy_rb0_cb; + bio.bi_rw |= READ; - submit_bio(READ, &bio); + submit_bio(&bio); process_fd_request(); init_completion(&cbdata.complete); diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c index 41fb1a9..c7e89af 100644 --- a/drivers/block/xen-blkback/blkback.c +++ b/drivers/block/xen-blkback/blkback.c @@ -1348,6 +1348,7 @@ static int dispatch_rw_block_io(struct xen_blkif *blkif, bio->bi_private = pending_req; bio->bi_end_io = end_block_io_op; bio->bi_iter.bi_sector = preq.sector_number; + bio->bi_rw |= operation; } preq.sector_number += seg[i].nsec; @@ -1365,13 +1366,14 @@ static int dispatch_rw_block_io(struct xen_blkif *blkif, bio->bi_bdev = preq.bdev; bio->bi_private = pending_req; bio->bi_end_io = end_block_io_op; + bio->bi_rw |= operation; } atomic_set(&pending_req->pendcnt, nbio); blk_start_plug(&plug); for (i = 0; i < nbio; i++) - submit_bio(operation, biolist[i]); + submit_bio(biolist[i]); /* Let the I/Os go.. */ blk_finish_plug(&plug); diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index 2fee2ee..81444fc 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -1749,7 +1749,7 @@ static int blkif_recover(struct blkfront_info *info) bio_trim(cloned_bio, offset, size); cloned_bio->bi_private = split_bio; cloned_bio->bi_end_io = split_bio_end; - submit_bio(cloned_bio->bi_rw, cloned_bio); + submit_bio(cloned_bio); } /* * Now we have to wait for all those smaller bios to @@ -1758,7 +1758,7 @@ static int blkif_recover(struct blkfront_info *info) continue; } /* We don't need to split this bio */ - submit_bio(bio->bi_rw, bio); + submit_bio(bio); } return 0; diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache/debug.c index 8b1f1d5..db68562 100644 --- a/drivers/md/bcache/debug.c +++ b/drivers/md/bcache/debug.c @@ -52,9 +52,10 @@ void bch_btree_verify(struct btree *b) bio->bi_bdev = PTR_CACHE(b->c, &b->key, 0)->bdev; bio->bi_iter.bi_sector = PTR_OFFSET(&b->key, 0); bio->bi_iter.bi_size = KEY_SIZE(&v->key) << 9; + bio->bi_rw |= REQ_META|READ_SYNC; bch_bio_map(bio, sorted); - submit_bio_wait(REQ_META|READ_SYNC, bio); + submit_bio_wait(bio); bch_bbio_free(bio, b->c); memcpy(ondisk, sorted, KEY_SIZE(&v->key) << 9); @@ -113,11 +114,12 @@ void bch_data_verify(struct cached_dev *dc, struct bio *bio) check = bio_clone(bio, GFP_NOIO); if (!check) return; + check->bi_rw |= READ_SYNC; if (bio_alloc_pages(check, GFP_NOIO)) goto out_put; - submit_bio_wait(READ_SYNC, check); + submit_bio_wait(check); bio_for_each_segment(bv, bio, iter) { void *p1 = kmap_atomic(bv.bv_page); diff --git a/drivers/md/bcache/journal.c b/drivers/md/bcache/journal.c index 29eba72..af3f9f7 100644 --- a/drivers/md/bcache/journal.c +++ b/drivers/md/bcache/journal.c @@ -418,7 +418,7 @@ static void journal_discard_work(struct work_struct *work) struct journal_device *ja = container_of(work, struct journal_device, discard_work); - submit_bio(0, &ja->discard_bio); + submit_bio(&ja->discard_bio); } static void do_journal_discard(struct cache *ca) diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c index 8d0ead9..9e2eac8 100644 --- a/drivers/md/bcache/super.c +++ b/drivers/md/bcache/super.c @@ -212,7 +212,7 @@ static void __write_super(struct cache_sb *sb, struct bio *bio) unsigned i; bio->bi_iter.bi_sector = SB_SECTOR; - bio->bi_rw = REQ_SYNC|REQ_META; + bio->bi_rw = REQ_WRITE|REQ_SYNC|REQ_META; bio->bi_iter.bi_size = SB_SIZE; bch_bio_map(bio, NULL); @@ -238,7 +238,7 @@ static void __write_super(struct cache_sb *sb, struct bio *bio) pr_debug("ver %llu, flags %llu, seq %llu", sb->version, sb->flags, sb->seq); - submit_bio(REQ_WRITE, bio); + submit_bio(bio); } static void bch_write_bdev_super_unlock(struct closure *cl) diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c index cd77216..1fd25bf 100644 --- a/drivers/md/dm-bufio.c +++ b/drivers/md/dm-bufio.c @@ -634,6 +634,7 @@ static void use_inline_bio(struct dm_buffer *b, int rw, sector_t block, * the dm_buffer's inline bio is local to bufio. */ b->bio.bi_private = end_io; + b->bio.bi_rw |= rw; /* * We assume that if len >= PAGE_SIZE ptr is page-aligned. @@ -660,7 +661,7 @@ static void use_inline_bio(struct dm_buffer *b, int rw, sector_t block, ptr += PAGE_SIZE; } while (len > 0); - submit_bio(rw, &b->bio); + submit_bio(&b->bio); } static void submit_io(struct dm_buffer *b, int rw, sector_t block, diff --git a/drivers/md/dm-io.c b/drivers/md/dm-io.c index 06d426e..107d445 100644 --- a/drivers/md/dm-io.c +++ b/drivers/md/dm-io.c @@ -322,6 +322,7 @@ static void do_region(int rw, unsigned region, struct dm_io_region *where, bio->bi_iter.bi_sector = where->sector + (where->count - remaining); bio->bi_bdev = where->bdev; bio->bi_end_io = endio; + bio->bi_rw |= rw; store_io_and_region_in_bio(bio, io, region); if (rw & REQ_DISCARD) { @@ -355,7 +356,7 @@ static void do_region(int rw, unsigned region, struct dm_io_region *where, } atomic_inc(&io->count); - submit_bio(rw, bio); + submit_bio(bio); } while (remaining); } diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c index 624589d..21760e3 100644 --- a/drivers/md/dm-log-writes.c +++ b/drivers/md/dm-log-writes.c @@ -205,6 +205,7 @@ static int write_metadata(struct log_writes_c *lc, void *entry, bio->bi_bdev = lc->logdev->bdev; bio->bi_end_io = log_end_io; bio->bi_private = lc; + bio->bi_rw |= WRITE; page = alloc_page(GFP_KERNEL); if (!page) { @@ -226,7 +227,7 @@ static int write_metadata(struct log_writes_c *lc, void *entry, DMERR("Couldn't add page to the log block"); goto error_bio; } - submit_bio(WRITE, bio); + submit_bio(bio); return 0; error_bio: bio_put(bio); @@ -269,6 +270,7 @@ static int log_one_block(struct log_writes_c *lc, bio->bi_bdev = lc->logdev->bdev; bio->bi_end_io = log_end_io; bio->bi_private = lc; + bio->bi_rw |= WRITE; for (i = 0; i < block->vec_cnt; i++) { /* @@ -279,7 +281,7 @@ static int log_one_block(struct log_writes_c *lc, block->vecs[i].bv_len, 0); if (ret != block->vecs[i].bv_len) { atomic_inc(&lc->io_blocks); - submit_bio(WRITE, bio); + submit_bio(bio); bio = bio_alloc(GFP_KERNEL, block->vec_cnt - i); if (!bio) { DMERR("Couldn't alloc log bio"); @@ -290,6 +292,7 @@ static int log_one_block(struct log_writes_c *lc, bio->bi_bdev = lc->logdev->bdev; bio->bi_end_io = log_end_io; bio->bi_private = lc; + bio->bi_rw |= WRITE; ret = bio_add_page(bio, block->vecs[i].bv_page, block->vecs[i].bv_len, 0); @@ -301,7 +304,7 @@ static int log_one_block(struct log_writes_c *lc, } sector += block->vecs[i].bv_len >> SECTOR_SHIFT; } - submit_bio(WRITE, bio); + submit_bio(bio); out: kfree(block->data); kfree(block); diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c index a1cc797..8017667 100644 --- a/drivers/md/dm-thin.c +++ b/drivers/md/dm-thin.c @@ -365,8 +365,9 @@ static int __blkdev_issue_discard_async(struct block_device *bdev, sector_t sect bio->bi_iter.bi_sector = sector; bio->bi_bdev = bdev; bio->bi_iter.bi_size = nr_sects << 9; + bio->bi_rw |= type; - submit_bio(type, bio); + submit_bio(bio); return 0; } diff --git a/drivers/md/md.c b/drivers/md/md.c index 4424d0e..e25ef97 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -401,8 +401,9 @@ static void submit_flushes(struct work_struct *ws) bi->bi_end_io = md_end_flush; bi->bi_private = rdev; bi->bi_bdev = rdev->bdev; + bi->bi_rw |= WRITE_FLUSH; atomic_inc(&mddev->flush_pending); - submit_bio(WRITE_FLUSH, bi); + submit_bio(bi); rcu_read_lock(); rdev_dec_pending(rdev, mddev); } @@ -746,9 +747,10 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev, bio_add_page(bio, page, size, 0); bio->bi_private = rdev; bio->bi_end_io = super_written; + bio->bi_rw |= WRITE_FLUSH_FUA; atomic_inc(&mddev->pending_writes); - submit_bio(WRITE_FLUSH_FUA, bio); + submit_bio(bio); } void md_super_wait(struct mddev *mddev) @@ -765,6 +767,7 @@ int sync_page_io(struct md_rdev *rdev, sector_t sector, int size, bio->bi_bdev = (metadata_op && rdev->meta_bdev) ? rdev->meta_bdev : rdev->bdev; + bio->bi_rw |= rw; if (metadata_op) bio->bi_iter.bi_sector = sector + rdev->sb_start; else if (rdev->mddev->reshape_position != MaxSector && @@ -774,7 +777,8 @@ int sync_page_io(struct md_rdev *rdev, sector_t sector, int size, else bio->bi_iter.bi_sector = sector + rdev->data_offset; bio_add_page(bio, page, size, 0); - submit_bio_wait(rw, bio); + + submit_bio_wait(bio); ret = !bio->bi_error; bio_put(bio); diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index e2169ff..01e6e74 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -2208,7 +2208,8 @@ static int narrow_write_error(struct r1bio *r1_bio, int i) bio_trim(wbio, sector - r1_bio->sector, sectors); wbio->bi_iter.bi_sector += rdev->data_offset; wbio->bi_bdev = rdev->bdev; - if (submit_bio_wait(WRITE, wbio) < 0) + + if (submit_bio_wait(wbio) < 0) /* failure! */ ok = rdev_set_badblocks(rdev, sector, sectors, 0) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 84e597e..d2da45b 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -2474,7 +2474,9 @@ static int narrow_write_error(struct r10bio *r10_bio, int i) choose_data_offset(r10_bio, rdev) + (sector - r10_bio->sector)); wbio->bi_bdev = rdev->bdev; - if (submit_bio_wait(WRITE, wbio) < 0) + wbio->bi_rw |= WRITE; + + if (submit_bio_wait(wbio) < 0) /* Failure! */ ok = rdev_set_badblocks(rdev, sector, sectors, 0) diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index d992d62..6b500fd 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -254,7 +254,7 @@ static void r5l_submit_current_io(struct r5l_log *log) __r5l_set_io_unit_state(io, IO_UNIT_IO_START); spin_unlock_irqrestore(&log->io_list_lock, flags); - submit_bio(WRITE, io->current_bio); + submit_bio(io->current_bio); } static struct bio *r5l_bio_alloc(struct r5l_log *log) @@ -373,7 +373,7 @@ static void r5l_append_payload_page(struct r5l_log *log, struct page *page) io->current_bio = r5l_bio_alloc(log); bio_chain(io->current_bio, prev); - submit_bio(WRITE, prev); + submit_bio(prev); } if (!bio_add_page(io->current_bio, page, PAGE_SIZE, 0)) @@ -686,7 +686,8 @@ void r5l_flush_stripe_to_raid(struct r5l_log *log) bio_reset(&log->flush_bio); log->flush_bio.bi_bdev = log->rdev->bdev; log->flush_bio.bi_end_io = r5l_log_flush_endio; - submit_bio(WRITE_FLUSH, &log->flush_bio); + log->flush_bio.bi_rw |= WRITE_FLUSH; + submit_bio(&log->flush_bio); } static void r5l_write_super(struct r5l_log *log, sector_t cp); diff --git a/drivers/target/target_core_iblock.c b/drivers/target/target_core_iblock.c index 5a2899f..bfc3e45 100644 --- a/drivers/target/target_core_iblock.c +++ b/drivers/target/target_core_iblock.c @@ -330,7 +330,7 @@ static void iblock_bio_done(struct bio *bio) } static struct bio * -iblock_get_bio(struct se_cmd *cmd, sector_t lba, u32 sg_num) +iblock_get_bio(struct se_cmd *cmd, sector_t lba, u32 sg_num, int rw) { struct iblock_dev *ib_dev = IBLOCK_DEV(cmd->se_dev); struct bio *bio; @@ -352,18 +352,19 @@ iblock_get_bio(struct se_cmd *cmd, sector_t lba, u32 sg_num) bio->bi_private = cmd; bio->bi_end_io = &iblock_bio_done; bio->bi_iter.bi_sector = lba; + bio->bi_rw |= rw; return bio; } -static void iblock_submit_bios(struct bio_list *list, int rw) +static void iblock_submit_bios(struct bio_list *list) { struct blk_plug plug; struct bio *bio; blk_start_plug(&plug); while ((bio = bio_list_pop(list))) - submit_bio(rw, bio); + submit_bio(bio); blk_finish_plug(&plug); } @@ -405,9 +406,10 @@ iblock_execute_sync_cache(struct se_cmd *cmd) bio = bio_alloc(GFP_KERNEL, 0); bio->bi_end_io = iblock_end_io_flush; bio->bi_bdev = ib_dev->ibd_bd; + bio->bi_rw |= WRITE_FLUSH; if (!immed) bio->bi_private = cmd; - submit_bio(WRITE_FLUSH, bio); + submit_bio(bio); return 0; } @@ -456,7 +458,7 @@ iblock_execute_write_same(struct se_cmd *cmd) goto fail; cmd->priv = ibr; - bio = iblock_get_bio(cmd, block_lba, 1); + bio = iblock_get_bio(cmd, block_lba, 1, WRITE); if (!bio) goto fail_free_ibr; @@ -469,7 +471,7 @@ iblock_execute_write_same(struct se_cmd *cmd) while (bio_add_page(bio, sg_page(sg), sg->length, sg->offset) != sg->length) { - bio = iblock_get_bio(cmd, block_lba, 1); + bio = iblock_get_bio(cmd, block_lba, 1, WRITE); if (!bio) goto fail_put_bios; @@ -482,7 +484,7 @@ iblock_execute_write_same(struct se_cmd *cmd) sectors -= 1; } - iblock_submit_bios(&list, WRITE); + iblock_submit_bios(&list); return 0; fail_put_bios: @@ -708,7 +710,7 @@ iblock_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, return 0; } - bio = iblock_get_bio(cmd, block_lba, sgl_nents); + bio = iblock_get_bio(cmd, block_lba, sgl_nents, rw); if (!bio) goto fail_free_ibr; @@ -728,11 +730,11 @@ iblock_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, while (bio_add_page(bio, sg_page(sg), sg->length, sg->offset) != sg->length) { if (bio_cnt >= IBLOCK_MAX_BIO_PER_TASK) { - iblock_submit_bios(&list, rw); + iblock_submit_bios(&list); bio_cnt = 0; } - bio = iblock_get_bio(cmd, block_lba, sg_num); + bio = iblock_get_bio(cmd, block_lba, sg_num, rw); if (!bio) goto fail_put_bios; @@ -752,7 +754,7 @@ iblock_execute_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents, goto fail_put_bios; } - iblock_submit_bios(&list, rw); + iblock_submit_bios(&list); iblock_complete_cmd(cmd); return 0; diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index 0340c57..7717043 100644 --- a/fs/btrfs/check-integrity.c +++ b/fs/btrfs/check-integrity.c @@ -1683,6 +1683,7 @@ static int btrfsic_read_block(struct btrfsic_state *state, } bio->bi_bdev = block_ctx->dev->bdev; bio->bi_iter.bi_sector = dev_bytenr >> 9; + bio->bi_rw |= READ; for (j = i; j < num_pages; j++) { ret = bio_add_page(bio, block_ctx->pagev[j], @@ -1695,7 +1696,7 @@ static int btrfsic_read_block(struct btrfsic_state *state, "btrfsic: error, failed to add a single page!\n"); return -1; } - if (submit_bio_wait(READ, bio)) { + if (submit_bio_wait(bio)) { printk(KERN_INFO "btrfsic: read error at logical %llu dev %s!\n", block_ctx->start, block_ctx->dev->name); @@ -2960,9 +2961,10 @@ int btrfsic_submit_bh(int rw, struct buffer_head *bh) return submit_bh(rw, bh); } -static void __btrfsic_submit_bio(int rw, struct bio *bio) +static void __btrfsic_submit_bio(struct bio *bio) { struct btrfsic_dev_state *dev_state; + int rw = bio->bi_rw; if (!btrfsic_is_initialized) return; @@ -3058,16 +3060,16 @@ leave: mutex_unlock(&btrfsic_mutex); } -void btrfsic_submit_bio(int rw, struct bio *bio) +void btrfsic_submit_bio(struct bio *bio) { - __btrfsic_submit_bio(rw, bio); - submit_bio(rw, bio); + __btrfsic_submit_bio(bio); + submit_bio(bio); } -int btrfsic_submit_bio_wait(int rw, struct bio *bio) +int btrfsic_submit_bio_wait(struct bio *bio) { - __btrfsic_submit_bio(rw, bio); - return submit_bio_wait(rw, bio); + __btrfsic_submit_bio(bio); + return submit_bio_wait(bio); } int btrfsic_mount(struct btrfs_root *root, diff --git a/fs/btrfs/check-integrity.h b/fs/btrfs/check-integrity.h index 13b8566..c04e249 100644 --- a/fs/btrfs/check-integrity.h +++ b/fs/btrfs/check-integrity.h @@ -21,8 +21,8 @@ #ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY int btrfsic_submit_bh(int rw, struct buffer_head *bh); -void btrfsic_submit_bio(int rw, struct bio *bio); -int btrfsic_submit_bio_wait(int rw, struct bio *bio); +void btrfsic_submit_bio(struct bio *bio); +int btrfsic_submit_bio_wait(struct bio *bio); #else #define btrfsic_submit_bh submit_bh #define btrfsic_submit_bio submit_bio diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 6dd6774..66dd552 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3454,12 +3454,13 @@ static int write_dev_flush(struct btrfs_device *device, int wait) bio->bi_end_io = btrfs_end_empty_barrier; bio->bi_bdev = device->bdev; + bio->bi_rw |= WRITE_FLUSH; init_completion(&device->flush_wait); bio->bi_private = &device->flush_wait; device->flush_bio = bio; bio_get(bio); - btrfsic_submit_bio(WRITE_FLUSH, bio); + btrfsic_submit_bio(bio); return 0; } diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index a2356e2..7bcc729 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -2040,9 +2040,10 @@ int repair_io_failure(struct inode *inode, u64 start, u64 length, u64 logical, return -EIO; } bio->bi_bdev = dev->bdev; + bio->bi_rw |= WRITE_SYNC; bio_add_page(bio, page, length, pg_offset); - if (btrfsic_submit_bio_wait(WRITE_SYNC, bio)) { + if (btrfsic_submit_bio_wait(bio)) { /* try to remap that extent elsewhere? */ bio_put(bio); btrfs_dev_stat_inc_and_print(dev, BTRFS_DEV_STAT_WRITE_ERRS); @@ -2730,14 +2731,14 @@ static int __must_check submit_one_bio(int rw, struct bio *bio, start = page_offset(page) + bvec->bv_offset; bio->bi_private = NULL; - + bio->bi_rw |= rw; bio_get(bio); if (tree->ops && tree->ops->submit_bio_hook) ret = tree->ops->submit_bio_hook(page->mapping->host, rw, bio, mirror_num, bio_flags, start); else - btrfsic_submit_bio(rw, bio); + btrfsic_submit_bio(bio); bio_put(bio); return ret; diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 1a33d3e..532ceb0 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1323,7 +1323,9 @@ write_data: bio->bi_private = rbio; bio->bi_end_io = raid_write_end_io; - submit_bio(WRITE, bio); + bio->bi_rw |= WRITE; + + submit_bio(bio); } return; @@ -1577,11 +1579,12 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid_rmw_end_io; + bio->bi_rw |= READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); - submit_bio(READ, bio); + submit_bio(bio); } /* the actual write will happen once the reads are done */ return 0; @@ -2103,11 +2106,12 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid_recover_end_io; + bio->bi_rw |= READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); - submit_bio(READ, bio); + submit_bio(bio); } out: return 0; @@ -2465,7 +2469,8 @@ submit_write: bio->bi_private = rbio; bio->bi_end_io = raid_write_parity_end_io; - submit_bio(WRITE, bio); + bio->bi_rw |= WRITE; + submit_bio(bio); } return; @@ -2642,11 +2647,12 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid56_parity_scrub_end_io; + bio->bi_rw |= READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); - submit_bio(READ, bio); + submit_bio(bio); } /* the actual write will happen once the reads are done */ return; diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 6998715..944959f 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -1504,8 +1504,9 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, sblock->no_io_error_seen = 0; } else { bio->bi_iter.bi_sector = page->physical >> 9; + bio->bi_rw |= READ; - if (btrfsic_submit_bio_wait(READ, bio)) + if (btrfsic_submit_bio_wait(bio)) sblock->no_io_error_seen = 0; } @@ -1585,6 +1586,7 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad, return -EIO; bio->bi_bdev = page_bad->dev->bdev; bio->bi_iter.bi_sector = page_bad->physical >> 9; + bio->bi_rw |= WRITE; ret = bio_add_page(bio, page_good->page, PAGE_SIZE, 0); if (PAGE_SIZE != ret) { @@ -1592,7 +1594,7 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad, return -EIO; } - if (btrfsic_submit_bio_wait(WRITE, bio)) { + if (btrfsic_submit_bio_wait(bio)) { btrfs_dev_stat_inc_and_print(page_bad->dev, BTRFS_DEV_STAT_WRITE_ERRS); btrfs_dev_replace_stats_inc( @@ -1685,6 +1687,7 @@ again: bio->bi_end_io = scrub_wr_bio_end_io; bio->bi_bdev = sbio->dev->bdev; bio->bi_iter.bi_sector = sbio->physical >> 9; + bio->bi_rw |= WRITE; sbio->err = 0; } else if (sbio->physical + sbio->page_count * PAGE_SIZE != spage->physical_for_dev_replace || @@ -1732,7 +1735,7 @@ static void scrub_wr_submit(struct scrub_ctx *sctx) * orders the requests before sending them to the driver which * doubled the write performance on spinning disks when measured * with Linux 3.5 */ - btrfsic_submit_bio(WRITE, sbio->bio); + btrfsic_submit_bio(sbio->bio); } static void scrub_wr_bio_end_io(struct bio *bio) @@ -2042,7 +2045,7 @@ static void scrub_submit(struct scrub_ctx *sctx) sbio = sctx->bios[sctx->curr]; sctx->curr = -1; scrub_pending_bio_inc(sctx); - btrfsic_submit_bio(READ, sbio->bio); + btrfsic_submit_bio(sbio->bio); } static int scrub_add_page_to_rd_bio(struct scrub_ctx *sctx, @@ -2088,6 +2091,7 @@ again: bio->bi_end_io = scrub_bio_end_io; bio->bi_bdev = sbio->dev->bdev; bio->bi_iter.bi_sector = sbio->physical >> 9; + bio->bi_rw |= READ; sbio->err = 0; } else if (sbio->physical + sbio->page_count * PAGE_SIZE != spage->physical || @@ -4390,6 +4394,8 @@ static int write_page_nocow(struct scrub_ctx *sctx, bio->bi_iter.bi_size = 0; bio->bi_iter.bi_sector = physical_for_dev_replace >> 9; bio->bi_bdev = dev->bdev; + bio->bi_rw |= WRITE_SYNC; + ret = bio_add_page(bio, page, PAGE_CACHE_SIZE, 0); if (ret != PAGE_CACHE_SIZE) { leave_with_eio: @@ -4398,7 +4404,7 @@ leave_with_eio: return -EIO; } - if (btrfsic_submit_bio_wait(WRITE_SYNC, bio)) + if (btrfsic_submit_bio_wait(bio)) goto leave_with_eio; bio_put(bio); diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index a114b7b..2463761 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -446,7 +446,7 @@ loop_lock: sync_pending = 0; } - btrfsic_submit_bio(cur->bi_rw, cur); + btrfsic_submit_bio(cur); num_run++; batch_run++; @@ -5942,7 +5942,7 @@ static void btrfs_end_bio(struct bio *bio) */ static noinline void btrfs_schedule_bio(struct btrfs_root *root, struct btrfs_device *device, - int rw, struct bio *bio) + struct bio *bio) { int should_queue = 1; struct btrfs_pending_bios *pending_bios; @@ -5953,9 +5953,9 @@ static noinline void btrfs_schedule_bio(struct btrfs_root *root, } /* don't bother with additional async steps for reads, right now */ - if (!(rw & REQ_WRITE)) { + if (!(bio->bi_rw & REQ_WRITE)) { bio_get(bio); - btrfsic_submit_bio(rw, bio); + btrfsic_submit_bio(bio); bio_put(bio); return; } @@ -5969,7 +5969,6 @@ static noinline void btrfs_schedule_bio(struct btrfs_root *root, atomic_inc(&root->fs_info->nr_async_bios); WARN_ON(bio->bi_next); bio->bi_next = NULL; - bio->bi_rw |= rw; spin_lock(&device->io_lock); if (bio->bi_rw & REQ_SYNC) @@ -6003,6 +6002,7 @@ static void submit_stripe_bio(struct btrfs_root *root, struct btrfs_bio *bbio, btrfs_io_bio(bio)->stripe_index = dev_nr; bio->bi_end_io = btrfs_end_bio; bio->bi_iter.bi_sector = physical >> 9; + bio->bi_rw |= rw; #ifdef DEBUG { struct rcu_string *name; @@ -6021,9 +6021,9 @@ static void submit_stripe_bio(struct btrfs_root *root, struct btrfs_bio *bbio, btrfs_bio_counter_inc_noblocked(root->fs_info); if (async) - btrfs_schedule_bio(root, dev, rw, bio); + btrfs_schedule_bio(root, dev, bio); else - btrfsic_submit_bio(rw, bio); + btrfsic_submit_bio(bio); } static void bbio_error(struct btrfs_bio *bbio, struct bio *bio, u64 logical) diff --git a/fs/buffer.c b/fs/buffer.c index 4f4cd95..63d0aed 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -3041,8 +3041,9 @@ static int submit_bh_wbc(int rw, struct buffer_head *bh, rw |= REQ_META; if (buffer_prio(bh)) rw |= REQ_PRIO; + bio->bi_rw |= rw; - submit_bio(rw, bio); + submit_bio(bio); return 0; } diff --git a/fs/direct-io.c b/fs/direct-io.c index 602e844..66b1d3eb 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -369,6 +369,7 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, bio->bi_bdev = bdev; bio->bi_iter.bi_sector = first_sector; + bio->bi_rw |= dio->rw; if (dio->is_async) bio->bi_end_io = dio_bio_end_aio; else @@ -406,7 +407,7 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) sdio->logical_offset_in_bio); dio->bio_cookie = BLK_QC_T_NONE; } else - dio->bio_cookie = submit_bio(dio->rw, bio); + dio->bio_cookie = submit_bio(bio); sdio->bio = NULL; sdio->boundary = 0; diff --git a/fs/ext4/crypto.c b/fs/ext4/crypto.c index c802120..a5fc446 100644 --- a/fs/ext4/crypto.c +++ b/fs/ext4/crypto.c @@ -424,6 +424,7 @@ int ext4_encrypted_zeroout(struct inode *inode, ext4_lblk_t lblk, bio->bi_bdev = inode->i_sb->s_bdev; bio->bi_iter.bi_sector = pblk << (inode->i_sb->s_blocksize_bits - 9); + bio->bi_rw |= WRITE; ret = bio_add_page(bio, ciphertext_page, inode->i_sb->s_blocksize, 0); if (ret != inode->i_sb->s_blocksize) { @@ -435,7 +436,7 @@ int ext4_encrypted_zeroout(struct inode *inode, ext4_lblk_t lblk, err = -EIO; goto errout; } - err = submit_bio_wait(WRITE, bio); + err = submit_bio_wait(bio); if ((err == 0) && bio->bi_error) err = -EIO; bio_put(bio); diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index 17fbe38..86bb60ba 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -357,7 +357,8 @@ void ext4_io_submit(struct ext4_io_submit *io) int io_op = io->io_wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE; bio_get(io->io_bio); - submit_bio(io_op, io->io_bio); + io->io_bio->bi_rw |= io_op; + submit_bio(io->io_bio); bio_put(io->io_bio); } io->io_bio = NULL; diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c index 5dc5e95..772e40b 100644 --- a/fs/ext4/readpage.c +++ b/fs/ext4/readpage.c @@ -271,7 +271,7 @@ int ext4_mpage_readpages(struct address_space *mapping, */ if (bio && (last_block_in_bio != blocks[0] - 1)) { submit_and_realloc: - submit_bio(READ, bio); + submit_bio(bio); bio = NULL; } if (bio == NULL) { @@ -294,6 +294,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0] << (blkbits - 9); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; + bio->bi_rw |= READ; } length = first_hole << blkbits; @@ -303,14 +304,14 @@ int ext4_mpage_readpages(struct address_space *mapping, if (((map.m_flags & EXT4_MAP_BOUNDARY) && (relative_block == map.m_len)) || (first_hole != blocks_per_page)) { - submit_bio(READ, bio); + submit_bio(bio); bio = NULL; } else last_block_in_bio = blocks[blocks_per_page - 1]; goto next_page; confused: if (bio) { - submit_bio(READ, bio); + submit_bio(bio); bio = NULL; } if (!PageUptodate(page)) @@ -323,6 +324,6 @@ int ext4_mpage_readpages(struct address_space *mapping, } BUG_ON(pages && !list_empty(pages)); if (bio) - submit_bio(READ, bio); + submit_bio(bio); return 0; } diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index e34b1bd..5325408 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -111,8 +111,9 @@ static void __submit_merged_bio(struct f2fs_bio_info *io) trace_f2fs_submit_read_bio(io->sbi->sb, fio, io->bio); else trace_f2fs_submit_write_bio(io->sbi->sb, fio, io->bio); + io->bio->bi_rw |= fio->rw; - submit_bio(fio->rw, io->bio); + submit_bio(io->bio); io->bio = NULL; } @@ -157,8 +158,9 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio) bio_put(bio); return -EFAULT; } + bio->bi_rw |= fio->rw; - submit_bio(fio->rw, bio); + submit_bio(bio); return 0; } @@ -953,7 +955,7 @@ got_it: */ if (bio && (last_block_in_bio != block_nr - 1)) { submit_and_realloc: - submit_bio(READ, bio); + submit_bio(bio); bio = NULL; } if (bio == NULL) { @@ -982,6 +984,7 @@ submit_and_realloc: bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(block_nr); bio->bi_end_io = f2fs_read_end_io; bio->bi_private = ctx; + bio->bi_rw |= READ; } if (bio_add_page(bio, page, blocksize, 0) < blocksize) @@ -996,7 +999,7 @@ set_error_page: goto next_page; confused: if (bio) { - submit_bio(READ, bio); + submit_bio(bio); bio = NULL; } unlock_page(page); @@ -1006,7 +1009,7 @@ next_page: } BUG_ON(pages && !list_empty(pages)); if (bio) - submit_bio(READ, bio); + submit_bio(bio); return 0; } diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index c247450..6ebdbb3 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -318,7 +318,8 @@ repeat: fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); bio->bi_bdev = sbi->sb->s_bdev; - ret = submit_bio_wait(WRITE_FLUSH, bio); + bio->bi_rw |= WRITE_FLUSH; + ret = submit_bio_wait(bio); llist_for_each_entry_safe(cmd, next, fcc->dispatch_list, llnode) { @@ -350,7 +351,8 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) int ret; bio->bi_bdev = sbi->sb->s_bdev; - ret = submit_bio_wait(WRITE_FLUSH, bio); + bio->bi_rw |= WRITE_FLUSH; + ret = submit_bio_wait(bio); bio_put(bio); return ret; } diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c index d5369a1..acc5ccb 100644 --- a/fs/gfs2/lops.c +++ b/fs/gfs2/lops.c @@ -240,7 +240,8 @@ void gfs2_log_flush_bio(struct gfs2_sbd *sdp, int rw) { if (sdp->sd_log_bio) { atomic_inc(&sdp->sd_log_in_flight); - submit_bio(rw, sdp->sd_log_bio); + sdp->sd_log_bio->bi_rw |= rw; + submit_bio(sdp->sd_log_bio); sdp->sd_log_bio = NULL; } } diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c index e137d96..013886d 100644 --- a/fs/gfs2/meta_io.c +++ b/fs/gfs2/meta_io.c @@ -230,7 +230,8 @@ static void gfs2_submit_bhs(int rw, struct buffer_head *bhs[], int num) bio_add_page(bio, bh->b_page, bh->b_size, bh_offset(bh)); } bio->bi_end_io = gfs2_meta_read_endio; - submit_bio(rw, bio); + bio->bi_rw |= rw; + submit_bio(bio); } /** diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c index 7aacdf2..15a4f86 100644 --- a/fs/gfs2/ops_fstype.c +++ b/fs/gfs2/ops_fstype.c @@ -246,7 +246,8 @@ static int gfs2_read_super(struct gfs2_sbd *sdp, sector_t sector, int silent) bio->bi_end_io = end_bio_io_page; bio->bi_private = page; - submit_bio(READ_SYNC | REQ_META, bio); + bio->bi_rw |= READ_SYNC | REQ_META; + submit_bio(bio); wait_on_page_locked(page); bio_put(bio); if (!PageUptodate(page)) { diff --git a/fs/hfsplus/wrapper.c b/fs/hfsplus/wrapper.c index cc62356..7e605b5 100644 --- a/fs/hfsplus/wrapper.c +++ b/fs/hfsplus/wrapper.c @@ -65,6 +65,7 @@ int hfsplus_submit_bio(struct super_block *sb, sector_t sector, bio = bio_alloc(GFP_NOIO, 1); bio->bi_iter.bi_sector = sector; bio->bi_bdev = sb->s_bdev; + bio->bi_rw |= rw; if (!(rw & WRITE) && data) *data = (u8 *)buf + offset; @@ -83,7 +84,7 @@ int hfsplus_submit_bio(struct super_block *sb, sector_t sector, buf = (u8 *)buf + len; } - ret = submit_bio_wait(rw, bio); + ret = submit_bio_wait(bio); out: bio_put(bio); return ret < 0 ? ret : 0; diff --git a/fs/jfs/jfs_logmgr.c b/fs/jfs/jfs_logmgr.c index a270cb7..a4321a6 100644 --- a/fs/jfs/jfs_logmgr.c +++ b/fs/jfs/jfs_logmgr.c @@ -2004,12 +2004,13 @@ static int lbmRead(struct jfs_log * log, int pn, struct lbuf ** bpp) bio->bi_end_io = lbmIODone; bio->bi_private = bp; + bio->bi_rw |= READ_SYNC; /*check if journaling to disk has been disabled*/ if (log->no_integrity) { bio->bi_iter.bi_size = 0; lbmIODone(bio); } else { - submit_bio(READ_SYNC, bio); + submit_bio(bio); } wait_event(bp->l_ioevent, (bp->l_flag != lbmREAD)); @@ -2147,13 +2148,14 @@ static void lbmStartIO(struct lbuf * bp) bio->bi_end_io = lbmIODone; bio->bi_private = bp; + bio->bi_rw |= WRITE_SYNC; /* check if journaling to disk has been disabled */ if (log->no_integrity) { bio->bi_iter.bi_size = 0; lbmIODone(bio); } else { - submit_bio(WRITE_SYNC, bio); + submit_bio(bio); INCREMENT(lmStat.submitted); } } diff --git a/fs/jfs/jfs_metapage.c b/fs/jfs/jfs_metapage.c index a3eb316..b5b76a5 100644 --- a/fs/jfs/jfs_metapage.c +++ b/fs/jfs/jfs_metapage.c @@ -411,7 +411,7 @@ static int metapage_writepage(struct page *page, struct writeback_control *wbc) inc_io(page); if (!bio->bi_iter.bi_size) goto dump_bio; - submit_bio(WRITE, bio); + submit_bio(bio); nr_underway++; bio = NULL; } else @@ -434,6 +434,7 @@ static int metapage_writepage(struct page *page, struct writeback_control *wbc) bio->bi_iter.bi_sector = pblock << (inode->i_blkbits - 9); bio->bi_end_io = metapage_write_end_io; bio->bi_private = page; + bio->bi_rw |= WRITE; /* Don't call bio_add_page yet, we may add to this vec */ bio_offset = offset; @@ -448,7 +449,7 @@ static int metapage_writepage(struct page *page, struct writeback_control *wbc) if (!bio->bi_iter.bi_size) goto dump_bio; - submit_bio(WRITE, bio); + submit_bio(bio); nr_underway++; } if (redirty) @@ -506,7 +507,7 @@ static int metapage_readpage(struct file *fp, struct page *page) insert_metapage(page, NULL); inc_io(page); if (bio) - submit_bio(READ, bio); + submit_bio(bio); bio = bio_alloc(GFP_NOFS, 1); bio->bi_bdev = inode->i_sb->s_bdev; @@ -514,6 +515,7 @@ static int metapage_readpage(struct file *fp, struct page *page) pblock << (inode->i_blkbits - 9); bio->bi_end_io = metapage_read_end_io; bio->bi_private = page; + bio->bi_rw |= READ; len = xlen << inode->i_blkbits; offset = block_offset << inode->i_blkbits; if (bio_add_page(bio, page, len, offset) < len) @@ -523,7 +525,7 @@ static int metapage_readpage(struct file *fp, struct page *page) block_offset++; } if (bio) - submit_bio(READ, bio); + submit_bio(bio); else unlock_page(page); diff --git a/fs/logfs/dev_bdev.c b/fs/logfs/dev_bdev.c index a709d80..623dd3e 100644 --- a/fs/logfs/dev_bdev.c +++ b/fs/logfs/dev_bdev.c @@ -29,8 +29,9 @@ static int sync_request(struct page *page, struct block_device *bdev, int rw) bio.bi_bdev = bdev; bio.bi_iter.bi_sector = page->index * (PAGE_SIZE >> 9); bio.bi_iter.bi_size = PAGE_SIZE; + bio.bi_rw |= rw; - return submit_bio_wait(rw, &bio); + return submit_bio_wait(&bio); } static int bdev_readpage(void *_sb, struct page *page) @@ -95,8 +96,9 @@ static int __bdev_writeseg(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = writeseg_end_io; + bio->bi_rw |= WRITE; atomic_inc(&super->s_pending_writes); - submit_bio(WRITE, bio); + submit_bio(bio); ofs += i * PAGE_SIZE; index += i; @@ -122,8 +124,9 @@ static int __bdev_writeseg(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = writeseg_end_io; + bio->bi_rw |= WRITE; atomic_inc(&super->s_pending_writes); - submit_bio(WRITE, bio); + submit_bio(bio); return 0; } @@ -185,8 +188,9 @@ static int do_erase(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = erase_end_io; + bio->bi_rw |= WRITE; atomic_inc(&super->s_pending_writes); - submit_bio(WRITE, bio); + submit_bio(bio); ofs += i * PAGE_SIZE; index += i; @@ -206,8 +210,9 @@ static int do_erase(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = erase_end_io; + bio->bi_rw |= WRITE; atomic_inc(&super->s_pending_writes); - submit_bio(WRITE, bio); + submit_bio(bio); return 0; } diff --git a/fs/mpage.c b/fs/mpage.c index 1480d3a..9fec67f 100644 --- a/fs/mpage.c +++ b/fs/mpage.c @@ -58,8 +58,9 @@ static void mpage_end_io(struct bio *bio) static struct bio *mpage_bio_submit(int rw, struct bio *bio) { bio->bi_end_io = mpage_end_io; + bio->bi_rw |= rw; guard_bio_eod(rw, bio); - submit_bio(rw, bio); + submit_bio(bio); return NULL; } diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c index ddd0138..bb8d629 100644 --- a/fs/nfs/blocklayout/blocklayout.c +++ b/fs/nfs/blocklayout/blocklayout.c @@ -102,14 +102,15 @@ static inline void put_parallel(struct parallel_io *p) } static struct bio * -bl_submit_bio(int rw, struct bio *bio) +bl_submit_bio(struct bio *bio) { if (bio) { get_parallel(bio->bi_private); dprintk("%s submitting %s bio %u@%llu\n", __func__, - rw == READ ? "read" : "write", bio->bi_iter.bi_size, + bio->bi_rw == READ ? "read" : "write", + bio->bi_iter.bi_size, (unsigned long long)bio->bi_iter.bi_sector); - submit_bio(rw, bio); + submit_bio(bio); } return NULL; } @@ -158,7 +159,7 @@ do_add_page_to_bio(struct bio *bio, int npg, int rw, sector_t isect, if (disk_addr < map->start || disk_addr >= map->start + map->len) { if (!dev->map(dev, disk_addr, map)) return ERR_PTR(-EIO); - bio = bl_submit_bio(rw, bio); + bio = bl_submit_bio(bio); } disk_addr += map->disk_offset; disk_addr -= map->start; @@ -174,9 +175,10 @@ retry: disk_addr >> SECTOR_SHIFT, end_io, par); if (!bio) return ERR_PTR(-ENOMEM); + bio->bi_rw |= rw; } if (bio_add_page(bio, page, *len, offset) < *len) { - bio = bl_submit_bio(rw, bio); + bio = bl_submit_bio(bio); goto retry; } return bio; @@ -252,7 +254,7 @@ bl_read_pagelist(struct nfs_pgio_header *header) for (i = pg_index; i < header->page_array.npages; i++) { if (extent_length <= 0) { /* We've used up the previous extent */ - bio = bl_submit_bio(READ, bio); + bio = bl_submit_bio(bio); /* Get the next one */ if (!ext_tree_lookup(bl, isect, &be, false)) { @@ -273,7 +275,7 @@ bl_read_pagelist(struct nfs_pgio_header *header) } if (is_hole(&be)) { - bio = bl_submit_bio(READ, bio); + bio = bl_submit_bio(bio); /* Fill hole w/ zeroes w/o accessing device */ dprintk("%s Zeroing page for hole\n", __func__); zero_user_segment(pages[i], pg_offset, pg_len); @@ -306,7 +308,7 @@ bl_read_pagelist(struct nfs_pgio_header *header) header->res.count = (isect << SECTOR_SHIFT) - header->args.offset; } out: - bl_submit_bio(READ, bio); + bl_submit_bio(bio); blk_finish_plug(&plug); put_parallel(par); return PNFS_ATTEMPTED; @@ -398,7 +400,7 @@ bl_write_pagelist(struct nfs_pgio_header *header, int sync) for (i = pg_index; i < header->page_array.npages; i++) { if (extent_length <= 0) { /* We've used up the previous extent */ - bio = bl_submit_bio(WRITE, bio); + bio = bl_submit_bio(bio); /* Get the next one */ if (!ext_tree_lookup(bl, isect, &be, true)) { header->pnfs_error = -EINVAL; @@ -427,7 +429,7 @@ bl_write_pagelist(struct nfs_pgio_header *header, int sync) header->res.count = header->args.count; out: - bl_submit_bio(WRITE, bio); + bl_submit_bio(bio); blk_finish_plug(&plug); put_parallel(par); return PNFS_ATTEMPTED; diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c index f63620c..428ece8 100644 --- a/fs/nilfs2/segbuf.c +++ b/fs/nilfs2/segbuf.c @@ -368,7 +368,8 @@ static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf, bio->bi_end_io = nilfs_end_bio_write; bio->bi_private = segbuf; - submit_bio(mode, bio); + bio->bi_rw |= mode; + submit_bio(bio); segbuf->sb_nbio++; wi->bio = NULL; diff --git a/fs/ocfs2/cluster/heartbeat.c b/fs/ocfs2/cluster/heartbeat.c index a3cc6d2..41039c2 100644 --- a/fs/ocfs2/cluster/heartbeat.c +++ b/fs/ocfs2/cluster/heartbeat.c @@ -391,7 +391,7 @@ static void o2hb_bio_end_io(struct bio *bio) static struct bio *o2hb_setup_one_bio(struct o2hb_region *reg, struct o2hb_bio_wait_ctxt *wc, unsigned int *current_slot, - unsigned int max_slots) + unsigned int max_slots, int rw) { int len, current_page; unsigned int vec_len, vec_start; @@ -417,6 +417,7 @@ static struct bio *o2hb_setup_one_bio(struct o2hb_region *reg, bio->bi_bdev = reg->hr_bdev; bio->bi_private = wc; bio->bi_end_io = o2hb_bio_end_io; + bio->bi_rw |= rw; vec_start = (cs << bits) % PAGE_CACHE_SIZE; while(cs < max_slots) { @@ -452,7 +453,8 @@ static int o2hb_read_slots(struct o2hb_region *reg, o2hb_bio_wait_init(&wc); while(current_slot < max_slots) { - bio = o2hb_setup_one_bio(reg, &wc, ¤t_slot, max_slots); + bio = o2hb_setup_one_bio(reg, &wc, ¤t_slot, max_slots, + READ); if (IS_ERR(bio)) { status = PTR_ERR(bio); mlog_errno(status); @@ -460,7 +462,7 @@ static int o2hb_read_slots(struct o2hb_region *reg, } atomic_inc(&wc.wc_num_reqs); - submit_bio(READ, bio); + submit_bio(bio); } status = 0; @@ -484,7 +486,7 @@ static int o2hb_issue_node_write(struct o2hb_region *reg, slot = o2nm_this_node(); - bio = o2hb_setup_one_bio(reg, write_wc, &slot, slot+1); + bio = o2hb_setup_one_bio(reg, write_wc, &slot, slot+1, WRITE_SYNC); if (IS_ERR(bio)) { status = PTR_ERR(bio); mlog_errno(status); @@ -492,7 +494,7 @@ static int o2hb_issue_node_write(struct o2hb_region *reg, } atomic_inc(&write_wc->wc_num_reqs); - submit_bio(WRITE_SYNC, bio); + submit_bio(bio); status = 0; bail: diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 29e7e5d..a1052d2 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -382,7 +382,8 @@ xfs_submit_ioend_bio( atomic_inc(&ioend->io_remaining); bio->bi_private = ioend; bio->bi_end_io = xfs_end_bio; - submit_bio(wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE, bio); + bio->bi_rw |= (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + submit_bio(bio); } STATIC struct bio * diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 45a8ea7..644e676 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -1163,7 +1163,7 @@ next_chunk: bio->bi_iter.bi_sector = sector; bio->bi_end_io = xfs_buf_bio_end_io; bio->bi_private = bp; - + bio->bi_rw |= rw; for (; size && nr_pages; nr_pages--, page_index++) { int rbytes, nbytes = PAGE_SIZE - offset; @@ -1187,7 +1187,7 @@ next_chunk: flush_kernel_vmap_range(bp->b_addr, xfs_buf_vmap_len(bp)); } - submit_bio(rw, bio); + submit_bio(bio); if (size) goto next_chunk; } else { diff --git a/include/linux/bio.h b/include/linux/bio.h index 5349e68..9f0e2cf 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -441,7 +441,7 @@ static inline void bio_io_error(struct bio *bio) struct request_queue; extern int bio_phys_segments(struct request_queue *, struct bio *); -extern int submit_bio_wait(int rw, struct bio *bio); +extern int submit_bio_wait(struct bio *bio); extern void bio_advance(struct bio *, unsigned); extern void bio_init(struct bio *); diff --git a/include/linux/fs.h b/include/linux/fs.h index fdab768..3b4e751 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2643,7 +2643,7 @@ static inline void remove_inode_hash(struct inode *inode) extern void inode_sb_list_add(struct inode *inode); #ifdef CONFIG_BLOCK -extern blk_qc_t submit_bio(int, struct bio *); +extern blk_qc_t submit_bio(struct bio *); extern int bdev_read_only(struct block_device *); #endif extern int set_blocksize(struct block_device *, int); diff --git a/kernel/power/swap.c b/kernel/power/swap.c index 12cd989..649dfc7 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -260,6 +260,7 @@ static int hib_submit_io(int rw, pgoff_t page_off, void *addr, bio = bio_alloc(__GFP_RECLAIM | __GFP_HIGH, 1); bio->bi_iter.bi_sector = page_off * (PAGE_SIZE >> 9); bio->bi_bdev = hib_resume_bdev; + bio->bi_rw |= rw; if (bio_add_page(bio, page, PAGE_SIZE, 0) < PAGE_SIZE) { printk(KERN_ERR "PM: Adding page to bio failed at %llu\n", @@ -272,9 +273,9 @@ static int hib_submit_io(int rw, pgoff_t page_off, void *addr, bio->bi_end_io = hib_end_io; bio->bi_private = hb; atomic_inc(&hb->count); - submit_bio(rw, bio); + submit_bio(bio); } else { - error = submit_bio_wait(rw, bio); + error = submit_bio_wait(bio); bio_put(bio); } diff --git a/mm/page_io.c b/mm/page_io.c index b995a5b..8410cdb 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -253,7 +253,7 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc, bio_end_io_t end_write_func) { struct bio *bio; - int ret, rw = WRITE; + int ret; struct swap_info_struct *sis = page_swap_info(page); if (sis->flags & SWP_FILE) { @@ -311,12 +311,13 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc, ret = -ENOMEM; goto out; } + bio->bi_rw |= WRITE; if (wbc->sync_mode == WB_SYNC_ALL) - rw |= REQ_SYNC; + bio->bi_rw |= REQ_SYNC; count_vm_event(PSWPOUT); set_page_writeback(page); unlock_page(page); - submit_bio(rw, bio); + submit_bio(bio); out: return ret; } @@ -358,8 +359,9 @@ int swap_readpage(struct page *page) ret = -ENOMEM; goto out; } + bio->bi_rw |= READ; count_vm_event(PSWPIN); - submit_bio(READ, bio); + submit_bio(bio); out: return ret; } -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:53:56 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 9947A7F37 for ; Tue, 5 Jan 2016 14:53:56 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 8C22B8F804C for ; Tue, 5 Jan 2016 12:53:53 -0800 (PST) X-ASG-Debug-ID: 1452027231-04cb6c42e805210001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 24IrPFwapBDHkNeC (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:53:52 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id A276AA3D26; Tue, 5 Jan 2016 20:53:51 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu1024032; Tue, 5 Jan 2016 15:53:49 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 02/35] block: add REQ_OP definitions and bi_op/op fields Date: Tue, 5 Jan 2016 14:53:05 -0600 X-ASG-Orig-Subj: [PATCH 02/35] block: add REQ_OP definitions and bi_op/op fields Message-Id: <1452027218-32303-3-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027232 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie The following patches separate the operation (write, read, discard, etc) from the flags in bi_rw/cmd_flags. This patch adds definitions for request/bio operations, adds fields to the request/bio to set them, and some temporary compat code so the kernel/modules can use either one. In the final patches this compat code will be removed when everything is converted. Also, in this patch the REQ_OPs match the REQ rq_flag_bits ones for compat reasons while all the code is converted in this set. In the last patches that will also be removed. Signed-off-by: Mike Christie --- block/blk-core.c | 19 ++++++++++++++++--- include/linux/blk_types.h | 15 ++++++++++++++- include/linux/blkdev.h | 1 + include/linux/fs.h | 37 +++++++++++++++++++++++++++++++++++-- 4 files changed, 66 insertions(+), 6 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 9b887e3..954a450 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1697,7 +1697,8 @@ void init_request_from_bio(struct request *req, struct bio *bio) { req->cmd_type = REQ_TYPE_FS; - req->cmd_flags |= bio->bi_rw & REQ_COMMON_MASK; + /* tmp compat. Allow users to set bi_op or bi_rw */ + req->cmd_flags |= (bio->bi_rw | bio->bi_op) & REQ_COMMON_MASK; if (bio->bi_rw & REQ_RAHEAD) req->cmd_flags |= REQ_FAILFAST_MASK; @@ -2032,6 +2033,12 @@ blk_qc_t generic_make_request(struct bio *bio) struct bio_list bio_list_on_stack; blk_qc_t ret = BLK_QC_T_NONE; + /* tmp compat. Allow users to set either one or both. + * This will be removed when we have converted + * everyone in the next patches. + */ + bio->bi_rw |= bio->bi_op; + if (!generic_make_request_checks(bio)) goto out; @@ -2101,6 +2108,12 @@ EXPORT_SYMBOL(generic_make_request); */ blk_qc_t submit_bio(struct bio *bio) { + /* tmp compat. Allow users to set either one or both. + * This will be removed when we have converted + * everyone in the next patches. + */ + bio->bi_rw |= bio->bi_op; + /* * If it's a regular read/write or a barrier with data attached, * go through the normal accounting stuff before submission. @@ -2972,8 +2985,8 @@ EXPORT_SYMBOL_GPL(__blk_end_request_err); void blk_rq_bio_prep(struct request_queue *q, struct request *rq, struct bio *bio) { - /* Bit 0 (R/W) is identical in rq->cmd_flags and bio->bi_rw */ - rq->cmd_flags |= bio->bi_rw & REQ_WRITE; + /* tmp compat. Allow users to set bi_op or bi_rw */ + rq->cmd_flags |= bio_data_dir(bio); if (bio_has_data(bio)) rq->nr_phys_segments = bio_phys_segments(q, bio); diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 86a38ea..6e49c91 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -48,9 +48,15 @@ struct bio { struct block_device *bi_bdev; unsigned int bi_flags; /* status, command, etc */ int bi_error; - unsigned long bi_rw; /* bottom bits READ/WRITE, + unsigned long bi_rw; /* bottom bits rq_flags_bits * top bits priority */ + /* + * this will be a u8 in the next patches and bi_rw can be shrunk to + * a u32. For compat in these transistional patches op is a int here. + */ + int bi_op; /* REQ_OP */ + struct bvec_iter bi_iter; @@ -242,6 +248,13 @@ enum rq_flag_bits { #define REQ_HASHED (1ULL << __REQ_HASHED) #define REQ_MQ_INFLIGHT (1ULL << __REQ_MQ_INFLIGHT) +enum req_op { + REQ_OP_READ, + REQ_OP_WRITE = REQ_WRITE, + REQ_OP_DISCARD = REQ_DISCARD, + REQ_OP_WRITE_SAME = REQ_WRITE_SAME, +}; + typedef unsigned int blk_qc_t; #define BLK_QC_T_NONE -1U #define BLK_QC_T_SHIFT 16 diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 29189ae..35b9eb3 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -96,6 +96,7 @@ struct request { struct request_queue *q; struct blk_mq_ctx *mq_ctx; + int op; u64 cmd_flags; unsigned cmd_type; unsigned long atomic_flags; diff --git a/include/linux/fs.h b/include/linux/fs.h index 3b4e751..fb9e516 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2405,15 +2405,48 @@ extern void make_bad_inode(struct inode *); extern bool is_bad_inode(struct inode *); #ifdef CONFIG_BLOCK + +static inline bool op_is_write(int op) +{ + switch (op) { + case REQ_OP_WRITE: + case REQ_OP_WRITE_SAME: + case REQ_OP_DISCARD: + return true; + default: + return false; + } +} + /* * return READ, READA, or WRITE */ -#define bio_rw(bio) ((bio)->bi_rw & (RW_MASK | RWA_MASK)) +static inline int bio_rw(struct bio *bio) +{ + /* + * tmp cpmpat. Allow users to set either op or rw, until + * all code is converted in the next patches. + */ + if (op_is_write(bio->bi_op)) + return WRITE; + + return bio->bi_rw & (RW_MASK | RWA_MASK); +} /* * return data direction, READ or WRITE */ -#define bio_data_dir(bio) ((bio)->bi_rw & 1) +static inline int bio_data_dir(struct bio *bio) +{ + /* + * tmp cpmpat. Allow users to set either op or rw, until + * all code is converted in the next patches. + */ + if (op_is_write(bio->bi_op)) + return WRITE; + + return bio->bi_rw & 1; +} extern void check_disk_size_change(struct gendisk *disk, struct block_device *bdev); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:00 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2BA7229E03 for ; Tue, 5 Jan 2016 14:54:00 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id DE856304039 for ; Tue, 5 Jan 2016 12:53:56 -0800 (PST) X-ASG-Debug-ID: 1452027234-04bdf03afc04db0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id NSFv6EN4VsqD7H8c (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:53:54 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 58ACBC109088; Tue, 5 Jan 2016 20:53:54 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu2024032; Tue, 5 Jan 2016 15:53:51 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 03/35] block, fs, mm, drivers: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:06 -0600 X-ASG-Orig-Subj: [PATCH 03/35] block, fs, mm, drivers: set bi_op to REQ_OP Message-Id: <1452027218-32303-4-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027234 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch converts the simple bi_rw use cases in the block, drivers, mm and fs code to use bi_op for a REQ_OP and bi_rw for rq_flag_bits. These should be simple one liner cases, so I just did them in one patch. The next patches handle the more complicated cases in a module per patch. Signed-off-by: Mike Christie --- block/bio.c | 8 +++++--- block/blk-flush.c | 1 + block/blk-lib.c | 7 ++++--- block/blk-map.c | 2 +- drivers/block/floppy.c | 2 +- drivers/block/pktcdvd.c | 4 ++-- drivers/lightnvm/rrpc.c | 4 ++-- drivers/scsi/osd/osd_initiator.c | 8 ++++---- fs/exofs/ore.c | 2 +- fs/ext4/crypto.c | 2 +- fs/ext4/page-io.c | 7 ++++--- fs/ext4/readpage.c | 2 +- fs/jfs/jfs_logmgr.c | 2 ++ fs/jfs/jfs_metapage.c | 4 ++-- fs/logfs/dev_bdev.c | 12 ++++++------ fs/nfs/blocklayout/blocklayout.c | 2 +- mm/page_io.c | 4 ++-- 17 files changed, 40 insertions(+), 33 deletions(-) diff --git a/block/bio.c b/block/bio.c index 921112b..3b8e970 100644 --- a/block/bio.c +++ b/block/bio.c @@ -581,6 +581,7 @@ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) */ bio->bi_bdev = bio_src->bi_bdev; bio_set_flag(bio, BIO_CLONED); + bio->bi_op = bio_src->bi_op; bio->bi_rw = bio_src->bi_rw; bio->bi_iter = bio_src->bi_iter; bio->bi_io_vec = bio_src->bi_io_vec; @@ -663,6 +664,7 @@ struct bio *bio_clone_bioset(struct bio *bio_src, gfp_t gfp_mask, return NULL; bio->bi_bdev = bio_src->bi_bdev; + bio->bi_op = bio_src->bi_op; bio->bi_rw = bio_src->bi_rw; bio->bi_iter.bi_sector = bio_src->bi_iter.bi_sector; bio->bi_iter.bi_size = bio_src->bi_iter.bi_size; @@ -1168,7 +1170,7 @@ struct bio *bio_copy_user_iov(struct request_queue *q, goto out_bmd; if (iter->type & WRITE) - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; ret = 0; @@ -1338,7 +1340,7 @@ struct bio *bio_map_user_iov(struct request_queue *q, * set data direction, and check if mapped pages need bouncing */ if (iter->type & WRITE) - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; bio_set_flag(bio, BIO_USER_MAPPED); @@ -1531,7 +1533,7 @@ struct bio *bio_copy_kern(struct request_queue *q, void *data, unsigned int len, bio->bi_private = data; } else { bio->bi_end_io = bio_copy_kern_endio; - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; } return bio; diff --git a/block/blk-flush.c b/block/blk-flush.c index e092e13..386f57a 100644 --- a/block/blk-flush.c +++ b/block/blk-flush.c @@ -484,6 +484,7 @@ int blkdev_issue_flush(struct block_device *bdev, gfp_t gfp_mask, bio = bio_alloc(gfp_mask, 0); bio->bi_bdev = bdev; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_FLUSH; ret = submit_bio_wait(bio); diff --git a/block/blk-lib.c b/block/blk-lib.c index 5292e30..5c55817 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -42,7 +42,7 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, { DECLARE_COMPLETION_ONSTACK(wait); struct request_queue *q = bdev_get_queue(bdev); - int type = REQ_WRITE | REQ_DISCARD; + int type = 0; unsigned int granularity; int alignment; struct bio_batch bb; @@ -102,6 +102,7 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, bio->bi_end_io = bio_batch_end_io; bio->bi_bdev = bdev; bio->bi_private = &bb; + bio->bi_op = REQ_OP_DISCARD; bio->bi_rw |= type; bio->bi_iter.bi_size = req_sects << 9; @@ -178,7 +179,7 @@ int blkdev_issue_write_same(struct block_device *bdev, sector_t sector, bio->bi_io_vec->bv_page = page; bio->bi_io_vec->bv_offset = 0; bio->bi_io_vec->bv_len = bdev_logical_block_size(bdev); - bio->bi_rw |= REQ_WRITE | REQ_WRITE_SAME; + bio->bi_op = REQ_OP_WRITE_SAME; if (nr_sects > max_write_same_sectors) { bio->bi_iter.bi_size = max_write_same_sectors << 9; @@ -240,7 +241,7 @@ static int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, bio->bi_bdev = bdev; bio->bi_end_io = bio_batch_end_io; bio->bi_private = &bb; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; while (nr_sects != 0) { sz = min((sector_t) PAGE_SIZE >> 9 , nr_sects); diff --git a/block/blk-map.c b/block/blk-map.c index f565e11..9021a8f 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -224,7 +224,7 @@ int blk_rq_map_kern(struct request_queue *q, struct request *rq, void *kbuf, return PTR_ERR(bio); if (!reading) - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; if (do_copy) rq->cmd_flags |= REQ_COPY_USER; diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c index e4efcc5..3fda035 100644 --- a/drivers/block/floppy.c +++ b/drivers/block/floppy.c @@ -3817,7 +3817,7 @@ static int __floppy_read_block_0(struct block_device *bdev, int drive) bio.bi_flags |= (1 << BIO_QUIET); bio.bi_private = &cbdata; bio.bi_end_io = floppy_rb0_cb; - bio.bi_rw |= READ; + bio.bi_op = REQ_OP_READ; submit_bio(&bio); process_fd_request(); diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c index d06c62e..a4ecbd3 100644 --- a/drivers/block/pktcdvd.c +++ b/drivers/block/pktcdvd.c @@ -1074,7 +1074,7 @@ static void pkt_gather_data(struct pktcdvd_device *pd, struct packet_data *pkt) BUG(); atomic_inc(&pkt->io_wait); - bio->bi_rw = READ; + bio->bi_op = REQ_OP_READ; pkt_queue_bio(pd, bio); frames_read++; } @@ -1336,7 +1336,7 @@ static void pkt_start_write(struct pktcdvd_device *pd, struct packet_data *pkt) /* Start the write request */ atomic_set(&pkt->io_wait, 1); - pkt->w_bio->bi_rw = WRITE; + pkt->w_bio->bi_op = REQ_OP_WRITE; pkt_queue_bio(pd, pkt->w_bio); } diff --git a/drivers/lightnvm/rrpc.c b/drivers/lightnvm/rrpc.c index 134e4fa..c20cb18 100644 --- a/drivers/lightnvm/rrpc.c +++ b/drivers/lightnvm/rrpc.c @@ -315,7 +315,7 @@ try: /* Perform read to do GC */ bio->bi_iter.bi_sector = rrpc_get_sector(rev->addr); - bio->bi_rw = READ; + bio->bi_op = REQ_OP_READ; bio->bi_private = &wait; bio->bi_end_io = rrpc_end_sync_bio; @@ -333,7 +333,7 @@ try: reinit_completion(&wait); bio->bi_iter.bi_sector = rrpc_get_sector(rev->addr); - bio->bi_rw = WRITE; + bio->bi_op = REQ_OP_WRITE; bio->bi_private = &wait; bio->bi_end_io = rrpc_end_sync_bio; diff --git a/drivers/scsi/osd/osd_initiator.c b/drivers/scsi/osd/osd_initiator.c index d8a2b51..4e7b440 100644 --- a/drivers/scsi/osd/osd_initiator.c +++ b/drivers/scsi/osd/osd_initiator.c @@ -726,7 +726,7 @@ static int _osd_req_list_objects(struct osd_request *or, return PTR_ERR(bio); } - bio->bi_rw &= ~REQ_WRITE; + bio->bi_op = REQ_OP_READ; or->in.bio = bio; or->in.total_bytes = bio->bi_iter.bi_size; return 0; @@ -839,7 +839,7 @@ int osd_req_write_kern(struct osd_request *or, if (IS_ERR(bio)) return PTR_ERR(bio); - bio->bi_rw |= REQ_WRITE; /* FIXME: bio_set_dir() */ + bio->bi_op = REQ_OP_WRITE; osd_req_write(or, obj, offset, bio, len); return 0; } @@ -956,7 +956,7 @@ static int _osd_req_finalize_cdb_cont(struct osd_request *or, const u8 *cap_key) if (IS_ERR(bio)) return PTR_ERR(bio); - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; /* integrity check the continuation before the bio is linked * with the other data segments since the continuation @@ -1077,7 +1077,7 @@ int osd_req_write_sg_kern(struct osd_request *or, if (IS_ERR(bio)) return PTR_ERR(bio); - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; osd_req_write_sg(or, obj, bio, sglist, numentries); return 0; diff --git a/fs/exofs/ore.c b/fs/exofs/ore.c index 7bd8ac8..c40ed74 100644 --- a/fs/exofs/ore.c +++ b/fs/exofs/ore.c @@ -878,7 +878,7 @@ static int _write_mirror(struct ore_io_state *ios, int cur_comp) } else { bio = master_dev->bio; /* FIXME: bio_set_dir() */ - bio->bi_rw |= REQ_WRITE; + bio->bi_op = REQ_OP_WRITE; } osd_req_write(or, _ios_obj(ios, cur_comp), diff --git a/fs/ext4/crypto.c b/fs/ext4/crypto.c index a5fc446..cc884e2 100644 --- a/fs/ext4/crypto.c +++ b/fs/ext4/crypto.c @@ -424,7 +424,7 @@ int ext4_encrypted_zeroout(struct inode *inode, ext4_lblk_t lblk, bio->bi_bdev = inode->i_sb->s_bdev; bio->bi_iter.bi_sector = pblk << (inode->i_sb->s_blocksize_bits - 9); - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; ret = bio_add_page(bio, ciphertext_page, inode->i_sb->s_blocksize, 0); if (ret != inode->i_sb->s_blocksize) { diff --git a/fs/ext4/page-io.c b/fs/ext4/page-io.c index 86bb60ba..e36dec4 100644 --- a/fs/ext4/page-io.c +++ b/fs/ext4/page-io.c @@ -354,10 +354,11 @@ void ext4_io_submit(struct ext4_io_submit *io) struct bio *bio = io->io_bio; if (bio) { - int io_op = io->io_wbc->sync_mode == WB_SYNC_ALL ? - WRITE_SYNC : WRITE; + int io_op_flags = io->io_wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : 0; + bio_get(io->io_bio); - io->io_bio->bi_rw |= io_op; + io->io_bio->bi_op = REQ_OP_WRITE; + io->io_bio->bi_rw |= io_op_flags; submit_bio(io->io_bio); bio_put(io->io_bio); } diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c index 772e40b..fb36434 100644 --- a/fs/ext4/readpage.c +++ b/fs/ext4/readpage.c @@ -294,7 +294,7 @@ int ext4_mpage_readpages(struct address_space *mapping, bio->bi_iter.bi_sector = blocks[0] << (blkbits - 9); bio->bi_end_io = mpage_end_io; bio->bi_private = ctx; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; } length = first_hole << blkbits; diff --git a/fs/jfs/jfs_logmgr.c b/fs/jfs/jfs_logmgr.c index a4321a6..00a4d89 100644 --- a/fs/jfs/jfs_logmgr.c +++ b/fs/jfs/jfs_logmgr.c @@ -2004,6 +2004,7 @@ static int lbmRead(struct jfs_log * log, int pn, struct lbuf ** bpp) bio->bi_end_io = lbmIODone; bio->bi_private = bp; + bio->bi_op = REQ_OP_READ; bio->bi_rw |= READ_SYNC; /*check if journaling to disk has been disabled*/ if (log->no_integrity) { @@ -2148,6 +2149,7 @@ static void lbmStartIO(struct lbuf * bp) bio->bi_end_io = lbmIODone; bio->bi_private = bp; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_SYNC; /* check if journaling to disk has been disabled */ diff --git a/fs/jfs/jfs_metapage.c b/fs/jfs/jfs_metapage.c index b5b76a5..115d99e 100644 --- a/fs/jfs/jfs_metapage.c +++ b/fs/jfs/jfs_metapage.c @@ -434,7 +434,7 @@ static int metapage_writepage(struct page *page, struct writeback_control *wbc) bio->bi_iter.bi_sector = pblock << (inode->i_blkbits - 9); bio->bi_end_io = metapage_write_end_io; bio->bi_private = page; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; /* Don't call bio_add_page yet, we may add to this vec */ bio_offset = offset; @@ -515,7 +515,7 @@ static int metapage_readpage(struct file *fp, struct page *page) pblock << (inode->i_blkbits - 9); bio->bi_end_io = metapage_read_end_io; bio->bi_private = page; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; len = xlen << inode->i_blkbits; offset = block_offset << inode->i_blkbits; if (bio_add_page(bio, page, len, offset) < len) diff --git a/fs/logfs/dev_bdev.c b/fs/logfs/dev_bdev.c index 623dd3e..130b7e2 100644 --- a/fs/logfs/dev_bdev.c +++ b/fs/logfs/dev_bdev.c @@ -14,7 +14,7 @@ #define PAGE_OFS(ofs) ((ofs) & (PAGE_SIZE-1)) -static int sync_request(struct page *page, struct block_device *bdev, int rw) +static int sync_request(struct page *page, struct block_device *bdev, int op) { struct bio bio; struct bio_vec bio_vec; @@ -29,7 +29,7 @@ static int sync_request(struct page *page, struct block_device *bdev, int rw) bio.bi_bdev = bdev; bio.bi_iter.bi_sector = page->index * (PAGE_SIZE >> 9); bio.bi_iter.bi_size = PAGE_SIZE; - bio.bi_rw |= rw; + bio.bi_op = op; return submit_bio_wait(&bio); } @@ -96,7 +96,7 @@ static int __bdev_writeseg(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = writeseg_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; atomic_inc(&super->s_pending_writes); submit_bio(bio); @@ -124,7 +124,7 @@ static int __bdev_writeseg(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = writeseg_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; atomic_inc(&super->s_pending_writes); submit_bio(bio); return 0; @@ -188,7 +188,7 @@ static int do_erase(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = erase_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; atomic_inc(&super->s_pending_writes); submit_bio(bio); @@ -210,7 +210,7 @@ static int do_erase(struct super_block *sb, u64 ofs, pgoff_t index, bio->bi_iter.bi_sector = ofs >> 9; bio->bi_private = sb; bio->bi_end_io = erase_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; atomic_inc(&super->s_pending_writes); submit_bio(bio); return 0; diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c index bb8d629..3832909 100644 --- a/fs/nfs/blocklayout/blocklayout.c +++ b/fs/nfs/blocklayout/blocklayout.c @@ -175,7 +175,7 @@ retry: disk_addr >> SECTOR_SHIFT, end_io, par); if (!bio) return ERR_PTR(-ENOMEM); - bio->bi_rw |= rw; + bio->bi_op = rw; } if (bio_add_page(bio, page, *len, offset) < *len) { bio = bl_submit_bio(bio); diff --git a/mm/page_io.c b/mm/page_io.c index 8410cdb..e48b84c 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -311,7 +311,7 @@ int __swap_writepage(struct page *page, struct writeback_control *wbc, ret = -ENOMEM; goto out; } - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; if (wbc->sync_mode == WB_SYNC_ALL) bio->bi_rw |= REQ_SYNC; count_vm_event(PSWPOUT); @@ -359,7 +359,7 @@ int swap_readpage(struct page *page) ret = -ENOMEM; goto out; } - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; count_vm_event(PSWPIN); submit_bio(bio); out: -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:01 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id 9C4B329E03 for ; Tue, 5 Jan 2016 14:54:01 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay3.corp.sgi.com (Postfix) with ESMTP id 14B7EAC001 for ; Tue, 5 Jan 2016 12:54:01 -0800 (PST) X-ASG-Debug-ID: 1452027237-04cb6c42e605210001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id o1tM5cUy66lN4Z3E (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:53:57 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 19D07C109084; Tue, 5 Jan 2016 20:53:57 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu3024032; Tue, 5 Jan 2016 15:53:54 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 04/35] fs: have submit_bh users pass in op and flags separately Date: Tue, 5 Jan 2016 14:53:07 -0600 X-ASG-Orig-Subj: [PATCH 04/35] fs: have submit_bh users pass in op and flags separately Message-Id: <1452027218-32303-5-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027237 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This has submit_bh users pass in the operation and flags separately, so we can setup the bio->bi_op and bio-bi_rw flags. Signed-off-by: Mike Christie --- drivers/md/bitmap.c | 4 ++-- fs/btrfs/check-integrity.c | 24 ++++++++++---------- fs/btrfs/check-integrity.h | 2 +- fs/btrfs/disk-io.c | 4 ++-- fs/buffer.c | 54 +++++++++++++++++++++++---------------------- fs/ext4/balloc.c | 2 +- fs/ext4/ialloc.c | 2 +- fs/ext4/inode.c | 2 +- fs/ext4/mmp.c | 4 ++-- fs/fat/misc.c | 2 +- fs/gfs2/bmap.c | 2 +- fs/gfs2/dir.c | 2 +- fs/gfs2/meta_io.c | 6 ++--- fs/jbd2/commit.c | 6 ++--- fs/jbd2/journal.c | 8 +++---- fs/nilfs2/btnode.c | 6 ++--- fs/nilfs2/btnode.h | 2 +- fs/nilfs2/btree.c | 6 +++-- fs/nilfs2/gcinode.c | 5 +++-- fs/nilfs2/mdt.c | 11 ++++----- fs/ntfs/aops.c | 6 ++--- fs/ntfs/compress.c | 2 +- fs/ntfs/file.c | 2 +- fs/ntfs/logfile.c | 2 +- fs/ntfs/mft.c | 4 ++-- fs/ocfs2/buffer_head_io.c | 8 +++---- fs/reiserfs/inode.c | 4 ++-- fs/reiserfs/journal.c | 6 ++--- fs/ufs/util.c | 2 +- include/linux/buffer_head.h | 9 ++++---- 30 files changed, 103 insertions(+), 96 deletions(-) diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c index 4f22e91..13811fc 100644 --- a/drivers/md/bitmap.c +++ b/drivers/md/bitmap.c @@ -299,7 +299,7 @@ static void write_page(struct bitmap *bitmap, struct page *page, int wait) atomic_inc(&bitmap->pending_writes); set_buffer_locked(bh); set_buffer_mapped(bh); - submit_bh(WRITE | REQ_SYNC, bh); + submit_bh(REQ_OP_WRITE, REQ_SYNC, bh); bh = bh->b_this_page; } @@ -394,7 +394,7 @@ static int read_page(struct file *file, unsigned long index, atomic_inc(&bitmap->pending_writes); set_buffer_locked(bh); set_buffer_mapped(bh); - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); } block++; bh = bh->b_this_page; diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index 7717043..e3fd86b 100644 --- a/fs/btrfs/check-integrity.c +++ b/fs/btrfs/check-integrity.c @@ -2898,12 +2898,12 @@ static struct btrfsic_dev_state *btrfsic_dev_state_lookup( return ds; } -int btrfsic_submit_bh(int rw, struct buffer_head *bh) +int btrfsic_submit_bh(int op, int op_flags, struct buffer_head *bh) { struct btrfsic_dev_state *dev_state; if (!btrfsic_is_initialized) - return submit_bh(rw, bh); + return submit_bh(op, op_flags, bh); mutex_lock(&btrfsic_mutex); /* since btrfsic_submit_bh() might also be called before @@ -2912,26 +2912,26 @@ int btrfsic_submit_bh(int rw, struct buffer_head *bh) /* Only called to write the superblock (incl. FLUSH/FUA) */ if (NULL != dev_state && - (rw & WRITE) && bh->b_size > 0) { + (op == REQ_OP_WRITE) && bh->b_size > 0) { u64 dev_bytenr; dev_bytenr = 4096 * bh->b_blocknr; if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH) printk(KERN_INFO - "submit_bh(rw=0x%x, blocknr=%llu (bytenr %llu)," - " size=%zu, data=%p, bdev=%p)\n", - rw, (unsigned long long)bh->b_blocknr, + "submit_bh(op=0x%x,0x%x, blocknr=%llu " + "(bytenr %llu), size=%zu, data=%p, bdev=%p)\n", + op, op_flags, (unsigned long long)bh->b_blocknr, dev_bytenr, bh->b_size, bh->b_data, bh->b_bdev); btrfsic_process_written_block(dev_state, dev_bytenr, &bh->b_data, 1, NULL, - NULL, bh, rw); - } else if (NULL != dev_state && (rw & REQ_FLUSH)) { + NULL, bh, op_flags); + } else if (NULL != dev_state && (op_flags & REQ_FLUSH)) { if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH) printk(KERN_INFO - "submit_bh(rw=0x%x FLUSH, bdev=%p)\n", - rw, bh->b_bdev); + "submit_bh(op=0x%x,0x%x FLUSH, bdev=%p)\n", + op, op_flags, bh->b_bdev); if (!dev_state->dummy_block_for_bio_bh_flush.is_iodone) { if ((dev_state->state->print_mask & (BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH | @@ -2949,7 +2949,7 @@ int btrfsic_submit_bh(int rw, struct buffer_head *bh) block->never_written = 0; block->iodone_w_error = 0; block->flush_gen = dev_state->last_flush_gen + 1; - block->submit_bio_bh_rw = rw; + block->submit_bio_bh_rw = op_flags; block->orig_bio_bh_private = bh->b_private; block->orig_bio_bh_end_io.bh = bh->b_end_io; block->next_in_same_bio = NULL; @@ -2958,7 +2958,7 @@ int btrfsic_submit_bh(int rw, struct buffer_head *bh) } } mutex_unlock(&btrfsic_mutex); - return submit_bh(rw, bh); + return submit_bh(op, op_flags, bh); } static void __btrfsic_submit_bio(struct bio *bio) diff --git a/fs/btrfs/check-integrity.h b/fs/btrfs/check-integrity.h index c04e249..f78dff1 100644 --- a/fs/btrfs/check-integrity.h +++ b/fs/btrfs/check-integrity.h @@ -20,7 +20,7 @@ #define __BTRFS_CHECK_INTEGRITY__ #ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY -int btrfsic_submit_bh(int rw, struct buffer_head *bh); +int btrfsic_submit_bh(int op, int op_flags, struct buffer_head *bh); void btrfsic_submit_bio(struct bio *bio); int btrfsic_submit_bio_wait(struct bio *bio); #else diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 66dd552..a77d88e 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -3388,9 +3388,9 @@ static int write_dev_supers(struct btrfs_device *device, * to go down lazy. */ if (i == 0) - ret = btrfsic_submit_bh(WRITE_FUA, bh); + ret = btrfsic_submit_bh(REQ_OP_WRITE, WRITE_FUA, bh); else - ret = btrfsic_submit_bh(WRITE_SYNC, bh); + ret = btrfsic_submit_bh(REQ_OP_WRITE, WRITE_SYNC, bh); if (ret) errors++; } diff --git a/fs/buffer.c b/fs/buffer.c index 63d0aed..0843964 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -45,7 +45,7 @@ #include static int fsync_buffers_list(spinlock_t *lock, struct list_head *list); -static int submit_bh_wbc(int rw, struct buffer_head *bh, +static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh, unsigned long bio_flags, struct writeback_control *wbc); @@ -1236,7 +1236,7 @@ static struct buffer_head *__bread_slow(struct buffer_head *bh) } else { get_bh(bh); bh->b_end_io = end_buffer_read_sync; - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); wait_on_buffer(bh); if (buffer_uptodate(bh)) return bh; @@ -1708,7 +1708,7 @@ static int __block_write_full_page(struct inode *inode, struct page *page, struct buffer_head *bh, *head; unsigned int blocksize, bbits; int nr_underway = 0; - int write_op = (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + int write_flags = (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC: 0); head = create_page_buffers(page, inode, (1 << BH_Dirty)|(1 << BH_Uptodate)); @@ -1797,7 +1797,7 @@ static int __block_write_full_page(struct inode *inode, struct page *page, do { struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { - submit_bh_wbc(write_op, bh, 0, wbc); + submit_bh_wbc(REQ_OP_WRITE, write_flags, bh, 0, wbc); nr_underway++; } bh = next; @@ -1851,7 +1851,7 @@ recover: struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { clear_buffer_dirty(bh); - submit_bh_wbc(write_op, bh, 0, wbc); + submit_bh_wbc(REQ_OP_WRITE, write_flags, bh, 0, wbc); nr_underway++; } bh = next; @@ -2259,7 +2259,7 @@ int block_read_full_page(struct page *page, get_block_t *get_block) if (buffer_uptodate(bh)) end_buffer_async_read(bh, 1); else - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); } return 0; } @@ -2593,7 +2593,7 @@ int nobh_write_begin(struct address_space *mapping, if (block_start < from || block_end > to) { lock_buffer(bh); bh->b_end_io = end_buffer_read_nobh; - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); nr_reads++; } } @@ -2960,7 +2960,7 @@ static void end_bio_bh_io_sync(struct bio *bio) * errors, this only handles the "we need to be able to * do IO at the final sector" case. */ -void guard_bio_eod(int rw, struct bio *bio) +void guard_bio_eod(int op, struct bio *bio) { sector_t maxsector; struct bio_vec *bvec = &bio->bi_io_vec[bio->bi_vcnt - 1]; @@ -2990,13 +2990,13 @@ void guard_bio_eod(int rw, struct bio *bio) bvec->bv_len -= truncated_bytes; /* ..and clear the end of the buffer for reads */ - if ((rw & RW_MASK) == READ) { + if (op == REQ_OP_READ) { zero_user(bvec->bv_page, bvec->bv_offset + bvec->bv_len, truncated_bytes); } } -static int submit_bh_wbc(int rw, struct buffer_head *bh, +static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh, unsigned long bio_flags, struct writeback_control *wbc) { struct bio *bio; @@ -3010,7 +3010,7 @@ static int submit_bh_wbc(int rw, struct buffer_head *bh, /* * Only clear out a write error when rewriting */ - if (test_set_buffer_req(bh) && (rw & WRITE)) + if (test_set_buffer_req(bh) && (op == REQ_OP_WRITE)) clear_buffer_write_io_error(bh); /* @@ -3033,29 +3033,31 @@ static int submit_bh_wbc(int rw, struct buffer_head *bh, bio->bi_end_io = end_bio_bh_io_sync; bio->bi_private = bh; bio->bi_flags |= bio_flags; + bio->bi_op = op; /* Take care of bh's that straddle the end of the device */ - guard_bio_eod(rw, bio); + guard_bio_eod(op, bio); if (buffer_meta(bh)) - rw |= REQ_META; + op_flags |= REQ_META; if (buffer_prio(bh)) - rw |= REQ_PRIO; - bio->bi_rw |= rw; + op_flags |= REQ_PRIO; + bio->bi_rw |= op_flags; submit_bio(bio); return 0; } -int _submit_bh(int rw, struct buffer_head *bh, unsigned long bio_flags) +int _submit_bh(int op, int op_flags, struct buffer_head *bh, + unsigned long bio_flags) { - return submit_bh_wbc(rw, bh, bio_flags, NULL); + return submit_bh_wbc(op, op_flags, bh, bio_flags, NULL); } EXPORT_SYMBOL_GPL(_submit_bh); -int submit_bh(int rw, struct buffer_head *bh) +int submit_bh(int op, int op_flags, struct buffer_head *bh) { - return submit_bh_wbc(rw, bh, 0, NULL); + return submit_bh_wbc(op, op_flags, bh, 0, NULL); } EXPORT_SYMBOL(submit_bh); @@ -3097,14 +3099,14 @@ void ll_rw_block(int rw, int nr, struct buffer_head *bhs[]) if (test_clear_buffer_dirty(bh)) { bh->b_end_io = end_buffer_write_sync; get_bh(bh); - submit_bh(WRITE, bh); + submit_bh(rw, 0, bh); continue; } } else { if (!buffer_uptodate(bh)) { bh->b_end_io = end_buffer_read_sync; get_bh(bh); - submit_bh(rw, bh); + submit_bh(rw, 0, bh); continue; } } @@ -3113,7 +3115,7 @@ void ll_rw_block(int rw, int nr, struct buffer_head *bhs[]) } EXPORT_SYMBOL(ll_rw_block); -void write_dirty_buffer(struct buffer_head *bh, int rw) +void write_dirty_buffer(struct buffer_head *bh, int op_flags) { lock_buffer(bh); if (!test_clear_buffer_dirty(bh)) { @@ -3122,7 +3124,7 @@ void write_dirty_buffer(struct buffer_head *bh, int rw) } bh->b_end_io = end_buffer_write_sync; get_bh(bh); - submit_bh(rw, bh); + submit_bh(REQ_OP_WRITE, op_flags, bh); } EXPORT_SYMBOL(write_dirty_buffer); @@ -3131,7 +3133,7 @@ EXPORT_SYMBOL(write_dirty_buffer); * and then start new I/O and then wait upon it. The caller must have a ref on * the buffer_head. */ -int __sync_dirty_buffer(struct buffer_head *bh, int rw) +int __sync_dirty_buffer(struct buffer_head *bh, int op_flags) { int ret = 0; @@ -3140,7 +3142,7 @@ int __sync_dirty_buffer(struct buffer_head *bh, int rw) if (test_clear_buffer_dirty(bh)) { get_bh(bh); bh->b_end_io = end_buffer_write_sync; - ret = submit_bh(rw, bh); + ret = submit_bh(REQ_OP_WRITE, op_flags, bh); wait_on_buffer(bh); if (!ret && !buffer_uptodate(bh)) ret = -EIO; @@ -3403,7 +3405,7 @@ int bh_submit_read(struct buffer_head *bh) get_bh(bh); bh->b_end_io = end_buffer_read_sync; - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); wait_on_buffer(bh); if (buffer_uptodate(bh)) return 0; diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c index ec0668a..b03bfd8 100644 --- a/fs/ext4/balloc.c +++ b/fs/ext4/balloc.c @@ -469,7 +469,7 @@ ext4_read_block_bitmap_nowait(struct super_block *sb, ext4_group_t block_group) trace_ext4_read_block_bitmap_load(sb, block_group); bh->b_end_io = ext4_end_bitmap_read; get_bh(bh); - submit_bh(READ | REQ_META | REQ_PRIO, bh); + submit_bh(REQ_OP_READ, REQ_META | REQ_PRIO, bh); return bh; verify: err = ext4_validate_block_bitmap(sb, desc, block_group, bh); diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c index e86e33c..0001b19 100644 --- a/fs/ext4/ialloc.c +++ b/fs/ext4/ialloc.c @@ -212,7 +212,7 @@ ext4_read_inode_bitmap(struct super_block *sb, ext4_group_t block_group) trace_ext4_load_inode_bitmap(sb, block_group); bh->b_end_io = ext4_end_bitmap_read; get_bh(bh); - submit_bh(READ | REQ_META | REQ_PRIO, bh); + submit_bh(REQ_OP_READ, REQ_META | REQ_PRIO, bh); wait_on_buffer(bh); if (!buffer_uptodate(bh)) { put_bh(bh); diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 032e6f2..4fc178a 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -4094,7 +4094,7 @@ make_io: trace_ext4_load_inode(inode); get_bh(bh); bh->b_end_io = end_buffer_read_sync; - submit_bh(READ | REQ_META | REQ_PRIO, bh); + submit_bh(REQ_OP_READ, REQ_META | REQ_PRIO, bh); wait_on_buffer(bh); if (!buffer_uptodate(bh)) { EXT4_ERROR_INODE_BLOCK(inode, block, diff --git a/fs/ext4/mmp.c b/fs/ext4/mmp.c index 0a512aa..bd90abf 100644 --- a/fs/ext4/mmp.c +++ b/fs/ext4/mmp.c @@ -52,7 +52,7 @@ static int write_mmp_block(struct super_block *sb, struct buffer_head *bh) lock_buffer(bh); bh->b_end_io = end_buffer_write_sync; get_bh(bh); - submit_bh(WRITE_SYNC | REQ_META | REQ_PRIO, bh); + submit_bh(REQ_OP_WRITE, WRITE_SYNC | REQ_META | REQ_PRIO, bh); wait_on_buffer(bh); sb_end_write(sb); if (unlikely(!buffer_uptodate(bh))) @@ -88,7 +88,7 @@ static int read_mmp_block(struct super_block *sb, struct buffer_head **bh, get_bh(*bh); lock_buffer(*bh); (*bh)->b_end_io = end_buffer_read_sync; - submit_bh(READ_SYNC | REQ_META | REQ_PRIO, *bh); + submit_bh(REQ_OP_READ, READ_SYNC | REQ_META | REQ_PRIO, *bh); wait_on_buffer(*bh); if (!buffer_uptodate(*bh)) { brelse(*bh); diff --git a/fs/fat/misc.c b/fs/fat/misc.c index c4589e9..8a86981 100644 --- a/fs/fat/misc.c +++ b/fs/fat/misc.c @@ -267,7 +267,7 @@ int fat_sync_bhs(struct buffer_head **bhs, int nr_bhs) int i, err = 0; for (i = 0; i < nr_bhs; i++) - write_dirty_buffer(bhs[i], WRITE); + write_dirty_buffer(bhs[i], 0); for (i = 0; i < nr_bhs; i++) { wait_on_buffer(bhs[i]); diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c index 0860f0b..29faf72 100644 --- a/fs/gfs2/bmap.c +++ b/fs/gfs2/bmap.c @@ -285,7 +285,7 @@ static void gfs2_metapath_ra(struct gfs2_glock *gl, if (trylock_buffer(rabh)) { if (!buffer_uptodate(rabh)) { rabh->b_end_io = end_buffer_read_sync; - submit_bh(READA | REQ_META, rabh); + submit_bh(REQ_OP_READ, READA | REQ_META, rabh); continue; } unlock_buffer(rabh); diff --git a/fs/gfs2/dir.c b/fs/gfs2/dir.c index 6a92592..2b81841 100644 --- a/fs/gfs2/dir.c +++ b/fs/gfs2/dir.c @@ -1510,7 +1510,7 @@ static void gfs2_dir_readahead(struct inode *inode, unsigned hsize, u32 index, continue; } bh->b_end_io = end_buffer_read_sync; - submit_bh(READA | REQ_META, bh); + submit_bh(REQ_OP_READ, READA | REQ_META, bh); continue; } brelse(bh); diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c index 013886d..b54a6fa 100644 --- a/fs/gfs2/meta_io.c +++ b/fs/gfs2/meta_io.c @@ -37,8 +37,8 @@ static int gfs2_aspace_writepage(struct page *page, struct writeback_control *wb { struct buffer_head *bh, *head; int nr_underway = 0; - int write_op = REQ_META | REQ_PRIO | - (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + int write_flags = REQ_META | REQ_PRIO | + (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : 0); BUG_ON(!PageLocked(page)); BUG_ON(!page_has_buffers(page)); @@ -79,7 +79,7 @@ static int gfs2_aspace_writepage(struct page *page, struct writeback_control *wb do { struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { - submit_bh(write_op, bh); + submit_bh(REQ_OP_WRITE, write_flags, bh); nr_underway++; } bh = next; diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c index 36345fe..7390e3a 100644 --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c @@ -157,9 +157,9 @@ static int journal_submit_commit_record(journal_t *journal, if (journal->j_flags & JBD2_BARRIER && !jbd2_has_feature_async_commit(journal)) - ret = submit_bh(WRITE_SYNC | WRITE_FLUSH_FUA, bh); + ret = submit_bh(REQ_OP_WRITE, WRITE_SYNC | WRITE_FLUSH_FUA, bh); else - ret = submit_bh(WRITE_SYNC, bh); + ret = submit_bh(REQ_OP_WRITE, WRITE_SYNC, bh); *cbh = bh; return ret; @@ -737,7 +737,7 @@ start_journal_io: clear_buffer_dirty(bh); set_buffer_uptodate(bh); bh->b_end_io = journal_end_buffer_io_sync; - submit_bh(WRITE_SYNC, bh); + submit_bh(REQ_OP_WRITE, WRITE_SYNC, bh); } cond_resched(); stats.run.rs_blocks_logged += bufs; diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c index 81e6226..3b7db0b 100644 --- a/fs/jbd2/journal.c +++ b/fs/jbd2/journal.c @@ -1323,15 +1323,15 @@ static int journal_reset(journal_t *journal) return jbd2_journal_start_thread(journal); } -static int jbd2_write_superblock(journal_t *journal, int write_op) +static int jbd2_write_superblock(journal_t *journal, int write_flags) { struct buffer_head *bh = journal->j_sb_buffer; journal_superblock_t *sb = journal->j_superblock; int ret; - trace_jbd2_write_superblock(journal, write_op); + trace_jbd2_write_superblock(journal, write_flags); if (!(journal->j_flags & JBD2_BARRIER)) - write_op &= ~(REQ_FUA | REQ_FLUSH); + write_flags &= ~(REQ_FUA | REQ_FLUSH); lock_buffer(bh); if (buffer_write_io_error(bh)) { /* @@ -1351,7 +1351,7 @@ static int jbd2_write_superblock(journal_t *journal, int write_op) jbd2_superblock_csum_set(journal, sb); get_bh(bh); bh->b_end_io = end_buffer_write_sync; - ret = submit_bh(write_op, bh); + ret = submit_bh(REQ_OP_WRITE, write_flags, bh); wait_on_buffer(bh); if (buffer_write_io_error(bh)) { clear_buffer_write_io_error(bh); diff --git a/fs/nilfs2/btnode.c b/fs/nilfs2/btnode.c index a35ae35..07fe874 100644 --- a/fs/nilfs2/btnode.c +++ b/fs/nilfs2/btnode.c @@ -67,7 +67,7 @@ nilfs_btnode_create_block(struct address_space *btnc, __u64 blocknr) } int nilfs_btnode_submit_block(struct address_space *btnc, __u64 blocknr, - sector_t pblocknr, int mode, + sector_t pblocknr, int mode, int mode_flags, struct buffer_head **pbh, sector_t *submit_ptr) { struct buffer_head *bh; @@ -100,7 +100,7 @@ int nilfs_btnode_submit_block(struct address_space *btnc, __u64 blocknr, } } - if (mode == READA) { + if (mode_flags & REQ_RAHEAD) { if (pblocknr != *submit_ptr + 1 || !trylock_buffer(bh)) { err = -EBUSY; /* internal code */ brelse(bh); @@ -119,7 +119,7 @@ int nilfs_btnode_submit_block(struct address_space *btnc, __u64 blocknr, bh->b_blocknr = pblocknr; /* set block address for read */ bh->b_end_io = end_buffer_read_sync; get_bh(bh); - submit_bh(mode, bh); + submit_bh(mode, mode_flags, bh); bh->b_blocknr = blocknr; /* set back to the given block address */ *submit_ptr = pblocknr; err = 0; diff --git a/fs/nilfs2/btnode.h b/fs/nilfs2/btnode.h index d876b56..3f93197 100644 --- a/fs/nilfs2/btnode.h +++ b/fs/nilfs2/btnode.h @@ -47,7 +47,7 @@ void nilfs_btnode_cache_clear(struct address_space *); struct buffer_head *nilfs_btnode_create_block(struct address_space *btnc, __u64 blocknr); int nilfs_btnode_submit_block(struct address_space *, __u64, sector_t, int, - struct buffer_head **, sector_t *); + int, struct buffer_head **, sector_t *); void nilfs_btnode_delete(struct buffer_head *); int nilfs_btnode_prepare_change_key(struct address_space *, struct nilfs_btnode_chkey_ctxt *); diff --git a/fs/nilfs2/btree.c b/fs/nilfs2/btree.c index 3a3821b..5d6a2c6 100644 --- a/fs/nilfs2/btree.c +++ b/fs/nilfs2/btree.c @@ -480,7 +480,8 @@ static int __nilfs_btree_get_block(const struct nilfs_bmap *btree, __u64 ptr, sector_t submit_ptr = 0; int ret; - ret = nilfs_btnode_submit_block(btnc, ptr, 0, READ, &bh, &submit_ptr); + ret = nilfs_btnode_submit_block(btnc, ptr, 0, REQ_OP_READ, 0, &bh, + &submit_ptr); if (ret) { if (ret != -EEXIST) return ret; @@ -496,7 +497,8 @@ static int __nilfs_btree_get_block(const struct nilfs_bmap *btree, __u64 ptr, n > 0 && i < ra->ncmax; n--, i++) { ptr2 = nilfs_btree_node_get_ptr(ra->node, i, ra->ncmax); - ret = nilfs_btnode_submit_block(btnc, ptr2, 0, READA, + ret = nilfs_btnode_submit_block(btnc, ptr2, 0, + REQ_OP_READ, REQ_RAHEAD, &ra_bh, &submit_ptr); if (likely(!ret || ret == -EEXIST)) brelse(ra_bh); diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c index 748ca23..1f18ffc 100644 --- a/fs/nilfs2/gcinode.c +++ b/fs/nilfs2/gcinode.c @@ -106,7 +106,7 @@ int nilfs_gccache_submit_read_data(struct inode *inode, sector_t blkoff, bh->b_blocknr = pbn; bh->b_end_io = end_buffer_read_sync; get_bh(bh); - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); if (vbn) bh->b_blocknr = vbn; out: @@ -143,7 +143,8 @@ int nilfs_gccache_submit_read_node(struct inode *inode, sector_t pbn, int ret; ret = nilfs_btnode_submit_block(&NILFS_I(inode)->i_btnode_cache, - vbn ? : pbn, pbn, READ, out_bh, &pbn); + vbn ? : pbn, pbn, REQ_OP_READ, 0, + out_bh, &pbn); if (ret == -EEXIST) /* internal code (cache hit) */ ret = 0; return ret; diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c index 1125f40..92e627d 100644 --- a/fs/nilfs2/mdt.c +++ b/fs/nilfs2/mdt.c @@ -124,7 +124,7 @@ static int nilfs_mdt_create_block(struct inode *inode, unsigned long block, static int nilfs_mdt_submit_block(struct inode *inode, unsigned long blkoff, - int mode, struct buffer_head **out_bh) + int mode, int mode_flags, struct buffer_head **out_bh) { struct buffer_head *bh; __u64 blknum = 0; @@ -138,7 +138,7 @@ nilfs_mdt_submit_block(struct inode *inode, unsigned long blkoff, if (buffer_uptodate(bh)) goto out; - if (mode == READA) { + if (mode_flags & REQ_RAHEAD) { if (!trylock_buffer(bh)) { ret = -EBUSY; goto failed_bh; @@ -160,7 +160,7 @@ nilfs_mdt_submit_block(struct inode *inode, unsigned long blkoff, bh->b_end_io = end_buffer_read_sync; get_bh(bh); - submit_bh(mode, bh); + submit_bh(mode, mode_flags, bh); ret = 0; trace_nilfs2_mdt_submit_block(inode, inode->i_ino, blkoff, mode); @@ -184,7 +184,7 @@ static int nilfs_mdt_read_block(struct inode *inode, unsigned long block, int i, nr_ra_blocks = NILFS_MDT_MAX_RA_BLOCKS; int err; - err = nilfs_mdt_submit_block(inode, block, READ, &first_bh); + err = nilfs_mdt_submit_block(inode, block, REQ_OP_READ, 0, &first_bh); if (err == -EEXIST) /* internal code */ goto out; @@ -194,7 +194,8 @@ static int nilfs_mdt_read_block(struct inode *inode, unsigned long block, if (readahead) { blkoff = block + 1; for (i = 0; i < nr_ra_blocks; i++, blkoff++) { - err = nilfs_mdt_submit_block(inode, blkoff, READA, &bh); + err = nilfs_mdt_submit_block(inode, blkoff, REQ_OP_READ, + REQ_RAHEAD, &bh); if (likely(!err || err == -EEXIST)) brelse(bh); else if (err != -EBUSY) diff --git a/fs/ntfs/aops.c b/fs/ntfs/aops.c index 7521e11..57c64bd 100644 --- a/fs/ntfs/aops.c +++ b/fs/ntfs/aops.c @@ -362,7 +362,7 @@ handle_zblock: for (i = 0; i < nr; i++) { tbh = arr[i]; if (likely(!buffer_uptodate(tbh))) - submit_bh(READ, tbh); + submit_bh(REQ_OP_READ, 0, tbh); else ntfs_end_buffer_async_read(tbh, 1); } @@ -877,7 +877,7 @@ lock_retry_remap: do { struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); need_end_writeback = false; } bh = next; @@ -1202,7 +1202,7 @@ lock_retry_remap: BUG_ON(!buffer_mapped(tbh)); get_bh(tbh); tbh->b_end_io = end_buffer_write_sync; - submit_bh(WRITE, tbh); + submit_bh(REQ_OP_WRITE, 0, tbh); } /* Synchronize the mft mirror now if not @sync. */ if (is_mft && !sync) diff --git a/fs/ntfs/compress.c b/fs/ntfs/compress.c index f82498c..1fd482c 100644 --- a/fs/ntfs/compress.c +++ b/fs/ntfs/compress.c @@ -674,7 +674,7 @@ lock_retry_remap: } get_bh(tbh); tbh->b_end_io = end_buffer_read_sync; - submit_bh(READ, tbh); + submit_bh(REQ_OP_READ, 0, tbh); } /* Wait for io completion on all buffer heads. */ diff --git a/fs/ntfs/file.c b/fs/ntfs/file.c index 9d383e5..03a1a32 100644 --- a/fs/ntfs/file.c +++ b/fs/ntfs/file.c @@ -553,7 +553,7 @@ static inline int ntfs_submit_bh_for_read(struct buffer_head *bh) lock_buffer(bh); get_bh(bh); bh->b_end_io = end_buffer_read_sync; - return submit_bh(READ, bh); + return submit_bh(REQ_OP_READ, 0, bh); } /** diff --git a/fs/ntfs/logfile.c b/fs/ntfs/logfile.c index c71de29..1c95c41 100644 --- a/fs/ntfs/logfile.c +++ b/fs/ntfs/logfile.c @@ -821,7 +821,7 @@ map_vcn: * completed ignore errors afterwards as we can assume * that if one buffer worked all of them will work. */ - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); if (should_wait) { should_wait = false; wait_on_buffer(bh); diff --git a/fs/ntfs/mft.c b/fs/ntfs/mft.c index 3014a36..38c6f7a 100644 --- a/fs/ntfs/mft.c +++ b/fs/ntfs/mft.c @@ -592,7 +592,7 @@ int ntfs_sync_mft_mirror(ntfs_volume *vol, const unsigned long mft_no, clear_buffer_dirty(tbh); get_bh(tbh); tbh->b_end_io = end_buffer_write_sync; - submit_bh(WRITE, tbh); + submit_bh(REQ_OP_WRITE, 0, tbh); } /* Wait on i/o completion of buffers. */ for (i_bhs = 0; i_bhs < nr_bhs; i_bhs++) { @@ -785,7 +785,7 @@ int write_mft_record_nolock(ntfs_inode *ni, MFT_RECORD *m, int sync) clear_buffer_dirty(tbh); get_bh(tbh); tbh->b_end_io = end_buffer_write_sync; - submit_bh(WRITE, tbh); + submit_bh(REQ_OP_WRITE, 0, tbh); } /* Synchronize the mft mirror now if not @sync. */ if (!sync && ni->mft_no < vol->mftmirr_size) diff --git a/fs/ocfs2/buffer_head_io.c b/fs/ocfs2/buffer_head_io.c index fe50ded..fb775c9 100644 --- a/fs/ocfs2/buffer_head_io.c +++ b/fs/ocfs2/buffer_head_io.c @@ -79,7 +79,7 @@ int ocfs2_write_block(struct ocfs2_super *osb, struct buffer_head *bh, get_bh(bh); /* for end_buffer_write_sync() */ bh->b_end_io = end_buffer_write_sync; - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); wait_on_buffer(bh); @@ -149,7 +149,7 @@ int ocfs2_read_blocks_sync(struct ocfs2_super *osb, u64 block, clear_buffer_uptodate(bh); get_bh(bh); /* for end_buffer_read_sync() */ bh->b_end_io = end_buffer_read_sync; - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); } for (i = nr; i > 0; i--) { @@ -305,7 +305,7 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr, if (validate) set_buffer_needs_validate(bh); bh->b_end_io = end_buffer_read_sync; - submit_bh(READ, bh); + submit_bh(REQ_OP_READ, 0, bh); continue; } } @@ -419,7 +419,7 @@ int ocfs2_write_super_or_backup(struct ocfs2_super *osb, get_bh(bh); /* for end_buffer_write_sync() */ bh->b_end_io = end_buffer_write_sync; ocfs2_compute_meta_ecc(osb->sb, bh->b_data, &di->i_check); - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); wait_on_buffer(bh); diff --git a/fs/reiserfs/inode.c b/fs/reiserfs/inode.c index ae9e5b3..91155ed 100644 --- a/fs/reiserfs/inode.c +++ b/fs/reiserfs/inode.c @@ -2668,7 +2668,7 @@ static int reiserfs_write_full_page(struct page *page, do { struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); nr++; } put_bh(bh); @@ -2728,7 +2728,7 @@ fail: struct buffer_head *next = bh->b_this_page; if (buffer_async_write(bh)) { clear_buffer_dirty(bh); - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); nr++; } put_bh(bh); diff --git a/fs/reiserfs/journal.c b/fs/reiserfs/journal.c index 9d6486d..9e63bc2 100644 --- a/fs/reiserfs/journal.c +++ b/fs/reiserfs/journal.c @@ -654,7 +654,7 @@ static void submit_logged_buffer(struct buffer_head *bh) BUG(); if (!buffer_uptodate(bh)) BUG(); - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); } static void submit_ordered_buffer(struct buffer_head *bh) @@ -664,7 +664,7 @@ static void submit_ordered_buffer(struct buffer_head *bh) clear_buffer_dirty(bh); if (!buffer_uptodate(bh)) BUG(); - submit_bh(WRITE, bh); + submit_bh(REQ_OP_WRITE, 0, bh); } #define CHUNK_SIZE 32 @@ -2271,7 +2271,7 @@ abort_replay: /* flush out the real blocks */ for (i = 0; i < get_desc_trans_len(desc); i++) { set_buffer_dirty(real_blocks[i]); - write_dirty_buffer(real_blocks[i], WRITE); + write_dirty_buffer(real_blocks[i], 0); } for (i = 0; i < get_desc_trans_len(desc); i++) { wait_on_buffer(real_blocks[i]); diff --git a/fs/ufs/util.c b/fs/ufs/util.c index b6c2f94..5f68b1a 100644 --- a/fs/ufs/util.c +++ b/fs/ufs/util.c @@ -118,7 +118,7 @@ void ubh_sync_block(struct ufs_buffer_head *ubh) unsigned i; for (i = 0; i < ubh->count; i++) - write_dirty_buffer(ubh->bh[i], WRITE); + write_dirty_buffer(ubh->bh[i], 0); for (i = 0; i < ubh->count; i++) wait_on_buffer(ubh->bh[i]); diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 89d9aa9..f7cc163 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -189,10 +189,11 @@ void unlock_buffer(struct buffer_head *bh); void __lock_buffer(struct buffer_head *bh); void ll_rw_block(int, int, struct buffer_head * bh[]); int sync_dirty_buffer(struct buffer_head *bh); -int __sync_dirty_buffer(struct buffer_head *bh, int rw); -void write_dirty_buffer(struct buffer_head *bh, int rw); -int _submit_bh(int rw, struct buffer_head *bh, unsigned long bio_flags); -int submit_bh(int, struct buffer_head *); +int __sync_dirty_buffer(struct buffer_head *bh, int op_flags); +void write_dirty_buffer(struct buffer_head *bh, int op_flags); +int _submit_bh(int op, int op_flags, struct buffer_head *bh, + unsigned long bio_flags); +int submit_bh(int, int, struct buffer_head *); void write_boundary_block(struct block_device *bdev, sector_t bblock, unsigned blocksize); int bh_uptodate_or_lock(struct buffer_head *bh); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:02 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 966C229E0C for ; Tue, 5 Jan 2016 14:54:02 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 4DBEE8F804C for ; Tue, 5 Jan 2016 12:54:02 -0800 (PST) X-ASG-Debug-ID: 1452027240-04bdf03af904db0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id XQ20vroXhr2mxC6c (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:00 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id D7B9D1317; Tue, 5 Jan 2016 20:53:59 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu4024032; Tue, 5 Jan 2016 15:53:57 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 05/35] fs: have ll_rw_block users pass in op and flags separately Date: Tue, 5 Jan 2016 14:53:08 -0600 X-ASG-Orig-Subj: [PATCH 05/35] fs: have ll_rw_block users pass in op and flags separately Message-Id: <1452027218-32303-6-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027240 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This has ll_rw_block users pass in the operation and flags separately, so we can setup the bio->bi_op and bio-bi_rw flags. Signed-off-by: Mike Christie --- fs/buffer.c | 19 ++++++++++--------- fs/ext4/inode.c | 6 +++--- fs/ext4/namei.c | 2 +- fs/ext4/super.c | 2 +- fs/gfs2/bmap.c | 2 +- fs/gfs2/meta_io.c | 4 ++-- fs/gfs2/quota.c | 2 +- fs/isofs/compress.c | 2 +- fs/jbd2/journal.c | 2 +- fs/jbd2/recovery.c | 4 ++-- fs/ocfs2/aops.c | 2 +- fs/ocfs2/super.c | 2 +- fs/reiserfs/journal.c | 8 ++++---- fs/reiserfs/stree.c | 4 ++-- fs/reiserfs/super.c | 2 +- fs/squashfs/block.c | 4 ++-- fs/udf/dir.c | 2 +- fs/udf/directory.c | 2 +- fs/udf/inode.c | 2 +- fs/ufs/balloc.c | 2 +- include/linux/buffer_head.h | 2 +- 21 files changed, 39 insertions(+), 38 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 0843964..1a14bf2 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -595,7 +595,7 @@ void write_boundary_block(struct block_device *bdev, struct buffer_head *bh = __find_get_block(bdev, bblock + 1, blocksize); if (bh) { if (buffer_dirty(bh)) - ll_rw_block(WRITE, 1, &bh); + ll_rw_block(REQ_OP_WRITE, 0, 1, &bh); put_bh(bh); } } @@ -1406,7 +1406,7 @@ void __breadahead(struct block_device *bdev, sector_t block, unsigned size) { struct buffer_head *bh = __getblk(bdev, block, size); if (likely(bh)) { - ll_rw_block(READA, 1, &bh); + ll_rw_block(REQ_OP_READ, READA, 1, &bh); brelse(bh); } } @@ -1966,7 +1966,7 @@ int __block_write_begin(struct page *page, loff_t pos, unsigned len, if (!buffer_uptodate(bh) && !buffer_delay(bh) && !buffer_unwritten(bh) && (block_start < from || block_end > to)) { - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); *wait_bh++=bh; } } @@ -2863,7 +2863,7 @@ int block_truncate_page(struct address_space *mapping, if (!buffer_uptodate(bh) && !buffer_delay(bh) && !buffer_unwritten(bh)) { err = -EIO; - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); /* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate(bh)) @@ -3063,7 +3063,8 @@ EXPORT_SYMBOL(submit_bh); /** * ll_rw_block: low-level access to block devices (DEPRECATED) - * @rw: whether to %READ or %WRITE or maybe %READA (readahead) + * @op: whether to %READ or %WRITE + * op_flags: rq_flag_bits or %READA (readahead) * @nr: number of &struct buffer_heads in the array * @bhs: array of pointers to &struct buffer_head * @@ -3086,7 +3087,7 @@ EXPORT_SYMBOL(submit_bh); * All of the buffers must be for the same device, and must also be a * multiple of the current approved size for the device. */ -void ll_rw_block(int rw, int nr, struct buffer_head *bhs[]) +void ll_rw_block(int op, int op_flags, int nr, struct buffer_head *bhs[]) { int i; @@ -3095,18 +3096,18 @@ void ll_rw_block(int rw, int nr, struct buffer_head *bhs[]) if (!trylock_buffer(bh)) continue; - if (rw == WRITE) { + if (op == WRITE) { if (test_clear_buffer_dirty(bh)) { bh->b_end_io = end_buffer_write_sync; get_bh(bh); - submit_bh(rw, 0, bh); + submit_bh(op, op_flags, bh); continue; } } else { if (!buffer_uptodate(bh)) { bh->b_end_io = end_buffer_read_sync; get_bh(bh); - submit_bh(rw, 0, bh); + submit_bh(op, op_flags, bh); continue; } } diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 4fc178a..26a07cb 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -810,7 +810,7 @@ struct buffer_head *ext4_bread(handle_t *handle, struct inode *inode, return bh; if (!bh || buffer_uptodate(bh)) return bh; - ll_rw_block(READ | REQ_META | REQ_PRIO, 1, &bh); + ll_rw_block(REQ_OP_READ, REQ_META | REQ_PRIO, 1, &bh); wait_on_buffer(bh); if (buffer_uptodate(bh)) return bh; @@ -964,7 +964,7 @@ static int ext4_block_write_begin(struct page *page, loff_t pos, unsigned len, if (!buffer_uptodate(bh) && !buffer_delay(bh) && !buffer_unwritten(bh) && (block_start < from || block_end > to)) { - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); *wait_bh++ = bh; decrypt = ext4_encrypted_inode(inode) && S_ISREG(inode->i_mode); @@ -3511,7 +3511,7 @@ static int __ext4_block_zero_page_range(handle_t *handle, if (!buffer_uptodate(bh)) { err = -EIO; - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); /* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate(bh)) diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 71d1021..714bf29 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -1440,7 +1440,7 @@ restart: } bh_use[ra_max] = bh; if (bh) - ll_rw_block(READ | REQ_META | REQ_PRIO, + ll_rw_block(REQ_OP_READ, REQ_META | REQ_PRIO, 1, &bh); } } diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 50b2207..380aadc 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -4204,7 +4204,7 @@ static journal_t *ext4_get_dev_journal(struct super_block *sb, goto out_bdev; } journal->j_private = sb; - ll_rw_block(READ | REQ_META | REQ_PRIO, 1, &journal->j_sb_buffer); + ll_rw_block(REQ_OP_READ, REQ_META | REQ_PRIO, 1, &journal->j_sb_buffer); wait_on_buffer(journal->j_sb_buffer); if (!buffer_uptodate(journal->j_sb_buffer)) { ext4_msg(sb, KERN_ERR, "I/O error on journal device"); diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c index 29faf72..628d4a4 100644 --- a/fs/gfs2/bmap.c +++ b/fs/gfs2/bmap.c @@ -974,7 +974,7 @@ static int gfs2_block_truncate_page(struct address_space *mapping, loff_t from) if (!buffer_uptodate(bh)) { err = -EIO; - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); /* Uhhuh. Read error. Complain and punt. */ if (!buffer_uptodate(bh)) diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c index b54a6fa..e2fc2b9 100644 --- a/fs/gfs2/meta_io.c +++ b/fs/gfs2/meta_io.c @@ -448,7 +448,7 @@ struct buffer_head *gfs2_meta_ra(struct gfs2_glock *gl, u64 dblock, u32 extlen) if (buffer_uptodate(first_bh)) goto out; if (!buffer_locked(first_bh)) - ll_rw_block(READ_SYNC | REQ_META, 1, &first_bh); + ll_rw_block(REQ_OP_READ, READ_SYNC | REQ_META, 1, &first_bh); dblock++; extlen--; @@ -457,7 +457,7 @@ struct buffer_head *gfs2_meta_ra(struct gfs2_glock *gl, u64 dblock, u32 extlen) bh = gfs2_getbuf(gl, dblock, CREATE); if (!buffer_uptodate(bh) && !buffer_locked(bh)) - ll_rw_block(READA | REQ_META, 1, &bh); + ll_rw_block(REQ_OP_READ, READA | REQ_META, 1, &bh); brelse(bh); dblock++; extlen--; diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c index be6d9c4..13be922 100644 --- a/fs/gfs2/quota.c +++ b/fs/gfs2/quota.c @@ -730,7 +730,7 @@ static int gfs2_write_buf_to_page(struct gfs2_inode *ip, unsigned long index, if (PageUptodate(page)) set_buffer_uptodate(bh); if (!buffer_uptodate(bh)) { - ll_rw_block(READ | REQ_META, 1, &bh); + ll_rw_block(REQ_OP_READ, REQ_META, 1, &bh); wait_on_buffer(bh); if (!buffer_uptodate(bh)) goto unlock_out; diff --git a/fs/isofs/compress.c b/fs/isofs/compress.c index f311bf0..f9b5fc8 100644 --- a/fs/isofs/compress.c +++ b/fs/isofs/compress.c @@ -81,7 +81,7 @@ static loff_t zisofs_uncompress_block(struct inode *inode, loff_t block_start, blocknum = block_start >> bufshift; memset(bhs, 0, (needblocks + 1) * sizeof(struct buffer_head *)); haveblocks = isofs_get_blocks(inode, blocknum, bhs, needblocks); - ll_rw_block(READ, haveblocks, bhs); + ll_rw_block(REQ_OP_READ, 0, haveblocks, bhs); curbh = 0; curpage = 0; diff --git a/fs/jbd2/journal.c b/fs/jbd2/journal.c index 3b7db0b..7104d76 100644 --- a/fs/jbd2/journal.c +++ b/fs/jbd2/journal.c @@ -1474,7 +1474,7 @@ static int journal_get_superblock(journal_t *journal) J_ASSERT(bh != NULL); if (!buffer_uptodate(bh)) { - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); if (!buffer_uptodate(bh)) { printk(KERN_ERR diff --git a/fs/jbd2/recovery.c b/fs/jbd2/recovery.c index 7f277e4..8d66ea9 100644 --- a/fs/jbd2/recovery.c +++ b/fs/jbd2/recovery.c @@ -104,7 +104,7 @@ static int do_readahead(journal_t *journal, unsigned int start) if (!buffer_uptodate(bh) && !buffer_locked(bh)) { bufs[nbufs++] = bh; if (nbufs == MAXBUF) { - ll_rw_block(READ, nbufs, bufs); + ll_rw_block(REQ_OP_READ, 0, nbufs, bufs); journal_brelse_array(bufs, nbufs); nbufs = 0; } @@ -113,7 +113,7 @@ static int do_readahead(journal_t *journal, unsigned int start) } if (nbufs) - ll_rw_block(READ, nbufs, bufs); + ll_rw_block(REQ_OP_READ, 0, nbufs, bufs); err = 0; failed: diff --git a/fs/ocfs2/aops.c b/fs/ocfs2/aops.c index 182f754..2e761fa 100644 --- a/fs/ocfs2/aops.c +++ b/fs/ocfs2/aops.c @@ -640,7 +640,7 @@ int ocfs2_map_page_blocks(struct page *page, u64 *p_blkno, !buffer_new(bh) && ocfs2_should_read_blk(inode, page, block_start) && (block_start < from || block_end > to)) { - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); *wait_bh++=bh; } diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c index 45b6139..9550593 100644 --- a/fs/ocfs2/super.c +++ b/fs/ocfs2/super.c @@ -1818,7 +1818,7 @@ static int ocfs2_get_sector(struct super_block *sb, if (!buffer_dirty(*bh)) clear_buffer_uptodate(*bh); unlock_buffer(*bh); - ll_rw_block(READ, 1, bh); + ll_rw_block(REQ_OP_READ, 0, 1, bh); wait_on_buffer(*bh); if (!buffer_uptodate(*bh)) { mlog_errno(-EIO); diff --git a/fs/reiserfs/journal.c b/fs/reiserfs/journal.c index 9e63bc2..5c14c90 100644 --- a/fs/reiserfs/journal.c +++ b/fs/reiserfs/journal.c @@ -872,7 +872,7 @@ loop_next: */ if (buffer_dirty(bh) && unlikely(bh->b_page->mapping == NULL)) { spin_unlock(lock); - ll_rw_block(WRITE, 1, &bh); + ll_rw_block(REQ_OP_WRITE, 0, 1, &bh); spin_lock(lock); } put_bh(bh); @@ -1059,7 +1059,7 @@ static int flush_commit_list(struct super_block *s, if (tbh) { if (buffer_dirty(tbh)) { depth = reiserfs_write_unlock_nested(s); - ll_rw_block(WRITE, 1, &tbh); + ll_rw_block(REQ_OP_WRITE, 0, 1, &tbh); reiserfs_write_lock_nested(s, depth); } put_bh(tbh) ; @@ -2246,7 +2246,7 @@ abort_replay: } } /* read in the log blocks, memcpy to the corresponding real block */ - ll_rw_block(READ, get_desc_trans_len(desc), log_blocks); + ll_rw_block(REQ_OP_READ, 0, get_desc_trans_len(desc), log_blocks); for (i = 0; i < get_desc_trans_len(desc); i++) { wait_on_buffer(log_blocks[i]); @@ -2348,7 +2348,7 @@ static struct buffer_head *reiserfs_breada(struct block_device *dev, } else bhlist[j++] = bh; } - ll_rw_block(READ, j, bhlist); + ll_rw_block(REQ_OP_READ, 0, j, bhlist); for (i = 1; i < j; i++) brelse(bhlist[i]); bh = bhlist[0]; diff --git a/fs/reiserfs/stree.c b/fs/reiserfs/stree.c index 24cbe01..5f9b6df 100644 --- a/fs/reiserfs/stree.c +++ b/fs/reiserfs/stree.c @@ -551,7 +551,7 @@ static int search_by_key_reada(struct super_block *s, if (!buffer_uptodate(bh[j])) { if (depth == -1) depth = reiserfs_write_unlock_nested(s); - ll_rw_block(READA, 1, bh + j); + ll_rw_block(REQ_OP_READ, READA, 1, bh + j); } brelse(bh[j]); } @@ -660,7 +660,7 @@ int search_by_key(struct super_block *sb, const struct cpu_key *key, if (!buffer_uptodate(bh) && depth == -1) depth = reiserfs_write_unlock_nested(sb); - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); if (depth != -1) diff --git a/fs/reiserfs/super.c b/fs/reiserfs/super.c index 05db747..bd04525 100644 --- a/fs/reiserfs/super.c +++ b/fs/reiserfs/super.c @@ -1660,7 +1660,7 @@ static int read_super_block(struct super_block *s, int offset) /* after journal replay, reread all bitmap and super blocks */ static int reread_meta_blocks(struct super_block *s) { - ll_rw_block(READ, 1, &SB_BUFFER_WITH_SB(s)); + ll_rw_block(REQ_OP_READ, 0, 1, &SB_BUFFER_WITH_SB(s)); wait_on_buffer(SB_BUFFER_WITH_SB(s)); if (!buffer_uptodate(SB_BUFFER_WITH_SB(s))) { reiserfs_warning(s, "reiserfs-2504", "error reading the super"); diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c index 0cea9b9..59375be 100644 --- a/fs/squashfs/block.c +++ b/fs/squashfs/block.c @@ -124,7 +124,7 @@ int squashfs_read_data(struct super_block *sb, u64 index, int length, goto block_release; bytes += msblk->devblksize; } - ll_rw_block(READ, b, bh); + ll_rw_block(REQ_OP_READ, 0, b, bh); } else { /* * Metadata block. @@ -156,7 +156,7 @@ int squashfs_read_data(struct super_block *sb, u64 index, int length, goto block_release; bytes += msblk->devblksize; } - ll_rw_block(READ, b - 1, bh + 1); + ll_rw_block(REQ_OP_READ, 0, b - 1, bh + 1); } for (i = 0; i < b; i++) { diff --git a/fs/udf/dir.c b/fs/udf/dir.c index 541d9c6..91b9f02 100644 --- a/fs/udf/dir.c +++ b/fs/udf/dir.c @@ -113,7 +113,7 @@ static int udf_readdir(struct file *file, struct dir_context *ctx) brelse(tmp); } if (num) { - ll_rw_block(READA, num, bha); + ll_rw_block(REQ_OP_READ, READA, num, bha); for (i = 0; i < num; i++) brelse(bha[i]); } diff --git a/fs/udf/directory.c b/fs/udf/directory.c index c763fda..71f3e0b 100644 --- a/fs/udf/directory.c +++ b/fs/udf/directory.c @@ -87,7 +87,7 @@ struct fileIdentDesc *udf_fileident_read(struct inode *dir, loff_t *nf_pos, brelse(tmp); } if (num) { - ll_rw_block(READA, num, bha); + ll_rw_block(REQ_OP_READ, READA, num, bha); for (i = 0; i < num; i++) brelse(bha[i]); } diff --git a/fs/udf/inode.c b/fs/udf/inode.c index 87dc16d..e49036b 100644 --- a/fs/udf/inode.c +++ b/fs/udf/inode.c @@ -1200,7 +1200,7 @@ struct buffer_head *udf_bread(struct inode *inode, int block, if (buffer_uptodate(bh)) return bh; - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); if (buffer_uptodate(bh)) diff --git a/fs/ufs/balloc.c b/fs/ufs/balloc.c index dc5fae6..26a3bb1 100644 --- a/fs/ufs/balloc.c +++ b/fs/ufs/balloc.c @@ -292,7 +292,7 @@ static void ufs_change_blocknr(struct inode *inode, sector_t beg, if (!buffer_mapped(bh)) map_bh(bh, inode->i_sb, oldb + pos); if (!buffer_uptodate(bh)) { - ll_rw_block(READ, 1, &bh); + ll_rw_block(REQ_OP_READ, 0, 1, &bh); wait_on_buffer(bh); if (!buffer_uptodate(bh)) { ufs_error(inode->i_sb, __func__, diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index f7cc163..15af19f 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -187,7 +187,7 @@ struct buffer_head *alloc_buffer_head(gfp_t gfp_flags); void free_buffer_head(struct buffer_head * bh); void unlock_buffer(struct buffer_head *bh); void __lock_buffer(struct buffer_head *bh); -void ll_rw_block(int, int, struct buffer_head * bh[]); +void ll_rw_block(int, int, int, struct buffer_head * bh[]); int sync_dirty_buffer(struct buffer_head *bh); int __sync_dirty_buffer(struct buffer_head *bh, int op_flags); void write_dirty_buffer(struct buffer_head *bh, int op_flags); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:05 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id CBDFC29DFC for ; Tue, 5 Jan 2016 14:54:04 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 6406CAC001 for ; Tue, 5 Jan 2016 12:54:04 -0800 (PST) X-ASG-Debug-ID: 1452027243-04bdf03afb04dc0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id GQCoKs7NlzpNdlu0 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:03 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id D2A72263C; Tue, 5 Jan 2016 20:54:02 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu5024032; Tue, 5 Jan 2016 15:53:59 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 06/35] direct-io: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:09 -0600 X-ASG-Orig-Subj: [PATCH 06/35] direct-io: set bi_op to REQ_OP Message-Id: <1452027218-32303-7-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027243 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has the dio code set the bio bi_op to a REQ_OP. It also begins to convert btrfs's dio_submit_t related code, because of the submit_io callout use. In the btrfs_submit_direct change, I OR'd the op and flag back together. It is only temporary. The next patch will completely convert all the btrfs code paths. Signed-off-by: Mike Christie --- fs/btrfs/inode.c | 9 +++++---- fs/direct-io.c | 35 +++++++++++++++++++++-------------- include/linux/fs.h | 2 +- 3 files changed, 27 insertions(+), 19 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 00f27eb..06f88bf 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -8300,14 +8300,14 @@ out_err: return 0; } -static void btrfs_submit_direct(int rw, struct bio *dio_bio, - struct inode *inode, loff_t file_offset) +static void btrfs_submit_direct(struct bio *dio_bio, struct inode *inode, + loff_t file_offset) { struct btrfs_dio_private *dip = NULL; struct bio *io_bio = NULL; struct btrfs_io_bio *btrfs_bio; int skip_sum; - int write = rw & REQ_WRITE; + bool write = (dio_bio->bi_op == REQ_OP_WRITE); int ret = 0; skip_sum = BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM; @@ -8358,7 +8358,8 @@ static void btrfs_submit_direct(int rw, struct bio *dio_bio, dio_data->unsubmitted_oe_range_end; } - ret = btrfs_submit_direct_hook(rw, dip, skip_sum); + ret = btrfs_submit_direct_hook(dio_bio->bi_op | dio_bio->bi_rw, dip, + skip_sum); if (!ret) return; diff --git a/fs/direct-io.c b/fs/direct-io.c index 66b1d3eb..aa12742 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -108,7 +108,8 @@ struct dio_submit { /* dio_state communicated between submission path and end_io */ struct dio { int flags; /* doesn't change */ - int rw; + int op; + int op_flags; blk_qc_t bio_cookie; struct block_device *bio_bdev; struct inode *inode; @@ -163,7 +164,7 @@ static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio) ret = iov_iter_get_pages(sdio->iter, dio->pages, LONG_MAX, DIO_PAGES, &sdio->from); - if (ret < 0 && sdio->blocks_available && (dio->rw & WRITE)) { + if (ret < 0 && sdio->blocks_available && (dio->op == REQ_OP_WRITE)) { struct page *page = ZERO_PAGE(0); /* * A memory fault, but the filesystem has some outstanding @@ -242,7 +243,8 @@ static ssize_t dio_complete(struct dio *dio, loff_t offset, ssize_t ret, transferred = dio->result; /* Check for short read case */ - if ((dio->rw == READ) && ((offset + transferred) > dio->i_size)) + if ((dio->op == REQ_OP_READ) && + ((offset + transferred) > dio->i_size)) transferred = dio->i_size - offset; } @@ -260,7 +262,7 @@ static ssize_t dio_complete(struct dio *dio, loff_t offset, ssize_t ret, inode_dio_end(dio->inode); if (is_async) { - if (dio->rw & WRITE) { + if (dio->op == REQ_OP_WRITE) { int err; err = generic_write_sync(dio->iocb->ki_filp, offset, @@ -369,7 +371,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, bio->bi_bdev = bdev; bio->bi_iter.bi_sector = first_sector; - bio->bi_rw |= dio->rw; + bio->bi_op = dio->op; + bio->bi_rw |= dio->op_flags; if (dio->is_async) bio->bi_end_io = dio_bio_end_aio; else @@ -397,14 +400,13 @@ static inline void dio_bio_submit(struct dio *dio, struct dio_submit *sdio) dio->refcount++; spin_unlock_irqrestore(&dio->bio_lock, flags); - if (dio->is_async && dio->rw == READ && dio->should_dirty) + if (dio->is_async && dio->op == REQ_OP_READ && dio->should_dirty) bio_set_pages_dirty(bio); dio->bio_bdev = bio->bi_bdev; if (sdio->submit_io) { - sdio->submit_io(dio->rw, bio, dio->inode, - sdio->logical_offset_in_bio); + sdio->submit_io(bio, dio->inode, sdio->logical_offset_in_bio); dio->bio_cookie = BLK_QC_T_NONE; } else dio->bio_cookie = submit_bio(bio); @@ -472,14 +474,14 @@ static int dio_bio_complete(struct dio *dio, struct bio *bio) if (bio->bi_error) dio->io_error = -EIO; - if (dio->is_async && dio->rw == READ && dio->should_dirty) { + if (dio->is_async && dio->op == REQ_OP_READ && dio->should_dirty) { bio_check_pages_dirty(bio); /* transfers ownership */ err = bio->bi_error; } else { bio_for_each_segment_all(bvec, bio, i) { struct page *page = bvec->bv_page; - if (dio->rw == READ && !PageCompound(page) && + if (dio->op == REQ_OP_READ && !PageCompound(page) && dio->should_dirty) set_page_dirty_lock(page); page_cache_release(page); @@ -632,7 +634,7 @@ static int get_more_blocks(struct dio *dio, struct dio_submit *sdio, * which may decide to handle it or also return an unmapped * buffer head. */ - create = dio->rw & WRITE; + create = dio->op == REQ_OP_WRITE; if (dio->flags & DIO_SKIP_HOLES) { if (sdio->block_in_file < (i_size_read(dio->inode) >> sdio->blkbits)) @@ -782,7 +784,7 @@ submit_page_section(struct dio *dio, struct dio_submit *sdio, struct page *page, { int ret = 0; - if (dio->rw & WRITE) { + if (dio->op == REQ_OP_WRITE) { /* * Read accounting is performed in submit_bio() */ @@ -982,7 +984,7 @@ do_holes: loff_t i_size_aligned; /* AKPM: eargh, -ENOTBLK is a hack */ - if (dio->rw & WRITE) { + if (dio->op == REQ_OP_WRITE) { page_cache_release(page); return -ENOTBLK; } @@ -1195,7 +1197,12 @@ do_blockdev_direct_IO(struct kiocb *iocb, struct inode *inode, dio->is_async = true; dio->inode = inode; - dio->rw = iov_iter_rw(iter) == WRITE ? WRITE_ODIRECT : READ; + if (iov_iter_rw(iter) == WRITE) { + dio->op = REQ_OP_WRITE; + dio->op_flags = WRITE_ODIRECT; + } else { + dio->op = REQ_OP_READ; + } /* * For AIO O_(D)SYNC writes we need to defer completions to a workqueue diff --git a/include/linux/fs.h b/include/linux/fs.h index fb9e516..3ab5342 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2731,7 +2731,7 @@ extern int generic_file_open(struct inode * inode, struct file * filp); extern int nonseekable_open(struct inode * inode, struct file * filp); #ifdef CONFIG_BLOCK -typedef void (dio_submit_t)(int rw, struct bio *bio, struct inode *inode, +typedef void (dio_submit_t)(struct bio *bio, struct inode *inode, loff_t file_offset); enum { -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:08 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 2726429E06 for ; Tue, 5 Jan 2016 14:54:08 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id CBCB4304039 for ; Tue, 5 Jan 2016 12:54:07 -0800 (PST) X-ASG-Debug-ID: 1452027246-04bdf03af904dc0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id IzxQnHqXsV9kdYyJ (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:06 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id D2250A203B; Tue, 5 Jan 2016 20:54:05 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu6024032; Tue, 5 Jan 2016 15:54:02 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 07/35] btrfs: have submit_one_bio users setup bio bi_op Date: Tue, 5 Jan 2016 14:53:10 -0600 X-ASG-Orig-Subj: [PATCH 07/35] btrfs: have submit_one_bio users setup bio bi_op Message-Id: <1452027218-32303-8-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027246 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has btrfs's submit_one_bio callers set the bio->bi_op to a REQ_OP and the bi_rw to rq_flag_bits. The next patches will continue to convert btrfs, so submit_bio_hook and merge_bio_hook related code will be modified to take only the bio. I did not do it in this patch to try and keep it smaller. Note: I have run xfs tests on these btrfs patches. There were some failures with and without the patches. I have not had time to track down why xfstest fails without the patches. Signed-off-by: Mike Christie --- fs/btrfs/extent_io.c | 92 +++++++++++++++++++++++++++------------------------- 1 file changed, 47 insertions(+), 45 deletions(-) diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index 7bcc729..b6c281a 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -2382,7 +2382,7 @@ static int bio_readpage_error(struct bio *failed_bio, u64 phy_offset, int read_mode; int ret; - BUG_ON(failed_bio->bi_rw & REQ_WRITE); + BUG_ON(failed_bio->bi_op == REQ_OP_WRITE); ret = btrfs_get_io_failure_record(inode, start, end, &failrec); if (ret) @@ -2408,6 +2408,8 @@ static int bio_readpage_error(struct bio *failed_bio, u64 phy_offset, free_io_failure(inode, failrec); return -EIO; } + bio->bi_op = REQ_OP_READ; + bio->bi_rw |= read_mode; pr_debug("Repair Read Error: submitting new read[%#x] to this_mirror=%d, in_validation=%d\n", read_mode, failrec->this_mirror, failrec->in_validation); @@ -2719,8 +2721,8 @@ struct bio *btrfs_io_bio_alloc(gfp_t gfp_mask, unsigned int nr_iovecs) } -static int __must_check submit_one_bio(int rw, struct bio *bio, - int mirror_num, unsigned long bio_flags) +static int __must_check submit_one_bio(struct bio *bio, int mirror_num, + unsigned long bio_flags) { int ret = 0; struct bio_vec *bvec = bio->bi_io_vec + bio->bi_vcnt - 1; @@ -2731,12 +2733,12 @@ static int __must_check submit_one_bio(int rw, struct bio *bio, start = page_offset(page) + bvec->bv_offset; bio->bi_private = NULL; - bio->bi_rw |= rw; bio_get(bio); if (tree->ops && tree->ops->submit_bio_hook) - ret = tree->ops->submit_bio_hook(page->mapping->host, rw, bio, - mirror_num, bio_flags, start); + ret = tree->ops->submit_bio_hook(page->mapping->host, + bio->bi_rw, bio, mirror_num, + bio_flags, start); else btrfsic_submit_bio(bio); @@ -2744,20 +2746,20 @@ static int __must_check submit_one_bio(int rw, struct bio *bio, return ret; } -static int merge_bio(int rw, struct extent_io_tree *tree, struct page *page, +static int merge_bio(struct extent_io_tree *tree, struct page *page, unsigned long offset, size_t size, struct bio *bio, unsigned long bio_flags) { int ret = 0; if (tree->ops && tree->ops->merge_bio_hook) - ret = tree->ops->merge_bio_hook(rw, page, offset, size, bio, - bio_flags); + ret = tree->ops->merge_bio_hook(bio->bi_op, page, offset, size, + bio, bio_flags); BUG_ON(ret < 0); return ret; } -static int submit_extent_page(int rw, struct extent_io_tree *tree, +static int submit_extent_page(int op, int op_flags, struct extent_io_tree *tree, struct writeback_control *wbc, struct page *page, sector_t sector, size_t size, unsigned long offset, @@ -2785,10 +2787,9 @@ static int submit_extent_page(int rw, struct extent_io_tree *tree, if (prev_bio_flags != bio_flags || !contig || force_bio_submit || - merge_bio(rw, tree, page, offset, page_size, bio, bio_flags) || + merge_bio(tree, page, offset, page_size, bio, bio_flags) || bio_add_page(bio, page, page_size, offset) < page_size) { - ret = submit_one_bio(rw, bio, mirror_num, - prev_bio_flags); + ret = submit_one_bio(bio, mirror_num, prev_bio_flags); if (ret < 0) { *bio_ret = NULL; return ret; @@ -2809,6 +2810,8 @@ static int submit_extent_page(int rw, struct extent_io_tree *tree, bio_add_page(bio, page, page_size, offset); bio->bi_end_io = end_io_func; bio->bi_private = tree; + bio->bi_op = op; + bio->bi_rw |= op_flags; if (wbc) { wbc_init_bio(wbc, bio); wbc_account_io(wbc, page, page_size); @@ -2817,7 +2820,7 @@ static int submit_extent_page(int rw, struct extent_io_tree *tree, if (bio_ret) *bio_ret = bio; else - ret = submit_one_bio(rw, bio, mirror_num, bio_flags); + ret = submit_one_bio(bio, mirror_num, bio_flags); return ret; } @@ -2881,7 +2884,7 @@ static int __do_readpage(struct extent_io_tree *tree, get_extent_t *get_extent, struct extent_map **em_cached, struct bio **bio, int mirror_num, - unsigned long *bio_flags, int rw, + unsigned long *bio_flags, int read_flags, u64 *prev_em_start) { struct inode *inode = page->mapping->host; @@ -3072,8 +3075,8 @@ static int __do_readpage(struct extent_io_tree *tree, } pnr -= page->index; - ret = submit_extent_page(rw, tree, NULL, page, - sector, disk_io_size, pg_offset, + ret = submit_extent_page(REQ_OP_READ, read_flags, tree, NULL, + page, sector, disk_io_size, pg_offset, bdev, bio, pnr, end_bio_extent_readpage, mirror_num, *bio_flags, @@ -3105,7 +3108,7 @@ static inline void __do_contiguous_readpages(struct extent_io_tree *tree, get_extent_t *get_extent, struct extent_map **em_cached, struct bio **bio, int mirror_num, - unsigned long *bio_flags, int rw, + unsigned long *bio_flags, u64 *prev_em_start) { struct inode *inode; @@ -3126,7 +3129,7 @@ static inline void __do_contiguous_readpages(struct extent_io_tree *tree, for (index = 0; index < nr_pages; index++) { __do_readpage(tree, pages[index], get_extent, em_cached, bio, - mirror_num, bio_flags, rw, prev_em_start); + mirror_num, bio_flags, 0, prev_em_start); page_cache_release(pages[index]); } } @@ -3136,7 +3139,7 @@ static void __extent_readpages(struct extent_io_tree *tree, int nr_pages, get_extent_t *get_extent, struct extent_map **em_cached, struct bio **bio, int mirror_num, - unsigned long *bio_flags, int rw, + unsigned long *bio_flags, u64 *prev_em_start) { u64 start = 0; @@ -3158,7 +3161,7 @@ static void __extent_readpages(struct extent_io_tree *tree, index - first_index, start, end, get_extent, em_cached, bio, mirror_num, bio_flags, - rw, prev_em_start); + prev_em_start); start = page_start; end = start + PAGE_CACHE_SIZE - 1; first_index = index; @@ -3169,7 +3172,7 @@ static void __extent_readpages(struct extent_io_tree *tree, __do_contiguous_readpages(tree, &pages[first_index], index - first_index, start, end, get_extent, em_cached, bio, - mirror_num, bio_flags, rw, + mirror_num, bio_flags, prev_em_start); } @@ -3177,7 +3180,7 @@ static int __extent_read_full_page(struct extent_io_tree *tree, struct page *page, get_extent_t *get_extent, struct bio **bio, int mirror_num, - unsigned long *bio_flags, int rw) + unsigned long *bio_flags, int read_flags) { struct inode *inode = page->mapping->host; struct btrfs_ordered_extent *ordered; @@ -3196,7 +3199,7 @@ static int __extent_read_full_page(struct extent_io_tree *tree, } ret = __do_readpage(tree, page, get_extent, NULL, bio, mirror_num, - bio_flags, rw, NULL); + bio_flags, read_flags, NULL); return ret; } @@ -3208,9 +3211,9 @@ int extent_read_full_page(struct extent_io_tree *tree, struct page *page, int ret; ret = __extent_read_full_page(tree, page, get_extent, &bio, mirror_num, - &bio_flags, READ); + &bio_flags, 0); if (bio) - ret = submit_one_bio(READ, bio, mirror_num, bio_flags); + ret = submit_one_bio(bio, mirror_num, bio_flags); return ret; } @@ -3222,9 +3225,9 @@ int extent_read_full_page_nolock(struct extent_io_tree *tree, struct page *page, int ret; ret = __do_readpage(tree, page, get_extent, NULL, &bio, mirror_num, - &bio_flags, READ, NULL); + &bio_flags, 0, NULL); if (bio) - ret = submit_one_bio(READ, bio, mirror_num, bio_flags); + ret = submit_one_bio(bio, mirror_num, bio_flags); return ret; } @@ -3471,7 +3474,8 @@ static noinline_for_stack int __extent_writepage_io(struct inode *inode, page->index, cur, end); } - ret = submit_extent_page(write_flags, tree, wbc, page, + ret = submit_extent_page(REQ_OP_WRITE, write_flags, + tree, wbc, page, sector, iosize, pg_offset, bdev, &epd->bio, max_nr, end_bio_extent_writepage, @@ -3511,13 +3515,11 @@ static int __extent_writepage(struct page *page, struct writeback_control *wbc, size_t pg_offset = 0; loff_t i_size = i_size_read(inode); unsigned long end_index = i_size >> PAGE_CACHE_SHIFT; - int write_flags; + int write_flags = 0; unsigned long nr_written = 0; if (wbc->sync_mode == WB_SYNC_ALL) write_flags = WRITE_SYNC; - else - write_flags = WRITE; trace___extent_writepage(page, inode, wbc); @@ -3761,7 +3763,7 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb, u64 offset = eb->start; unsigned long i, num_pages; unsigned long bio_flags = 0; - int rw = (epd->sync_io ? WRITE_SYNC : WRITE) | REQ_META; + int write_flags = (epd->sync_io ? WRITE_SYNC : 0) | REQ_META; int ret = 0; clear_bit(EXTENT_BUFFER_WRITE_ERR, &eb->bflags); @@ -3775,9 +3777,10 @@ static noinline_for_stack int write_one_eb(struct extent_buffer *eb, clear_page_dirty_for_io(p); set_page_writeback(p); - ret = submit_extent_page(rw, tree, wbc, p, offset >> 9, - PAGE_CACHE_SIZE, 0, bdev, &epd->bio, - -1, end_bio_extent_buffer_writepage, + ret = submit_extent_page(REQ_OP_WRITE, write_flags, tree, wbc, + p, offset >> 9, PAGE_CACHE_SIZE, 0, + bdev, &epd->bio, -1, + end_bio_extent_buffer_writepage, 0, epd->bio_flags, bio_flags, false); epd->bio_flags = bio_flags; if (ret) { @@ -4066,13 +4069,13 @@ retry: static void flush_epd_write_bio(struct extent_page_data *epd) { if (epd->bio) { - int rw = WRITE; int ret; + epd->bio->bi_op = REQ_OP_WRITE; if (epd->sync_io) - rw = WRITE_SYNC; + epd->bio->bi_rw = WRITE_SYNC; - ret = submit_one_bio(rw, epd->bio, 0, epd->bio_flags); + ret = submit_one_bio(epd->bio, 0, epd->bio_flags); BUG_ON(ret < 0); /* -ENOMEM */ epd->bio = NULL; } @@ -4199,19 +4202,19 @@ int extent_readpages(struct extent_io_tree *tree, if (nr < ARRAY_SIZE(pagepool)) continue; __extent_readpages(tree, pagepool, nr, get_extent, &em_cached, - &bio, 0, &bio_flags, READ, &prev_em_start); + &bio, 0, &bio_flags, &prev_em_start); nr = 0; } if (nr) __extent_readpages(tree, pagepool, nr, get_extent, &em_cached, - &bio, 0, &bio_flags, READ, &prev_em_start); + &bio, 0, &bio_flags, &prev_em_start); if (em_cached) free_extent_map(em_cached); BUG_ON(!list_empty(pages)); if (bio) - return submit_one_bio(READ, bio, 0, bio_flags); + return submit_one_bio(bio, 0, bio_flags); return 0; } @@ -5233,7 +5236,7 @@ int read_extent_buffer_pages(struct extent_io_tree *tree, err = __extent_read_full_page(tree, page, get_extent, &bio, mirror_num, &bio_flags, - READ | REQ_META); + REQ_META); if (err) ret = err; } else { @@ -5242,8 +5245,7 @@ int read_extent_buffer_pages(struct extent_io_tree *tree, } if (bio) { - err = submit_one_bio(READ | REQ_META, bio, mirror_num, - bio_flags); + err = submit_one_bio(bio, mirror_num, bio_flags); if (err) return err; } -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:10 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 9170F29E1C for ; Tue, 5 Jan 2016 14:54:10 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 574F1304066 for ; Tue, 5 Jan 2016 12:54:10 -0800 (PST) X-ASG-Debug-ID: 1452027248-04bdf03afa04dd0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id W1C8wj4VU82R7jKq (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:08 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 16A27461C4; Tue, 5 Jan 2016 20:54:08 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu7024032; Tue, 5 Jan 2016 15:54:05 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 08/35] btrfs: set bi_op tp REQ_OP Date: Tue, 5 Jan 2016 14:53:11 -0600 X-ASG-Orig-Subj: [PATCH 08/35] btrfs: set bi_op tp REQ_OP Message-Id: <1452027218-32303-9-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027248 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has btrfs set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. Signed-off-by: Mike Christie --- fs/btrfs/check-integrity.c | 19 +++++++++---------- fs/btrfs/compression.c | 4 ++++ fs/btrfs/disk-io.c | 7 ++++--- fs/btrfs/inode.c | 20 +++++++++++++------- fs/btrfs/raid56.c | 10 +++++----- fs/btrfs/scrub.c | 9 +++++---- fs/btrfs/volumes.c | 18 +++++++++--------- 7 files changed, 49 insertions(+), 38 deletions(-) diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-integrity.c index e3fd86b..e409d1f 100644 --- a/fs/btrfs/check-integrity.c +++ b/fs/btrfs/check-integrity.c @@ -1683,7 +1683,7 @@ static int btrfsic_read_block(struct btrfsic_state *state, } bio->bi_bdev = block_ctx->dev->bdev; bio->bi_iter.bi_sector = dev_bytenr >> 9; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; for (j = i; j < num_pages; j++) { ret = bio_add_page(bio, block_ctx->pagev[j], @@ -2964,7 +2964,6 @@ int btrfsic_submit_bh(int op, int op_flags, struct buffer_head *bh) static void __btrfsic_submit_bio(struct bio *bio) { struct btrfsic_dev_state *dev_state; - int rw = bio->bi_rw; if (!btrfsic_is_initialized) return; @@ -2974,7 +2973,7 @@ static void __btrfsic_submit_bio(struct bio *bio) * btrfsic_mount(), this might return NULL */ dev_state = btrfsic_dev_state_lookup(bio->bi_bdev); if (NULL != dev_state && - (rw & WRITE) && NULL != bio->bi_io_vec) { + (bio->bi_op == REQ_OP_WRITE) && NULL != bio->bi_io_vec) { unsigned int i; u64 dev_bytenr; u64 cur_bytenr; @@ -2986,9 +2985,9 @@ static void __btrfsic_submit_bio(struct bio *bio) if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH) printk(KERN_INFO - "submit_bio(rw=0x%x, bi_vcnt=%u," + "submit_bio(rw=%d,0x%lx, bi_vcnt=%u," " bi_sector=%llu (bytenr %llu), bi_bdev=%p)\n", - rw, bio->bi_vcnt, + bio->bi_op, bio->bi_rw, bio->bi_vcnt, (unsigned long long)bio->bi_iter.bi_sector, dev_bytenr, bio->bi_bdev); @@ -3019,18 +3018,18 @@ static void __btrfsic_submit_bio(struct bio *bio) btrfsic_process_written_block(dev_state, dev_bytenr, mapped_datav, bio->bi_vcnt, bio, &bio_is_patched, - NULL, rw); + NULL, bio->bi_rw); while (i > 0) { i--; kunmap(bio->bi_io_vec[i].bv_page); } kfree(mapped_datav); - } else if (NULL != dev_state && (rw & REQ_FLUSH)) { + } else if (NULL != dev_state && (bio->bi_rw & REQ_FLUSH)) { if (dev_state->state->print_mask & BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH) printk(KERN_INFO - "submit_bio(rw=0x%x FLUSH, bdev=%p)\n", - rw, bio->bi_bdev); + "submit_bio(rw=%d,0x%lx FLUSH, bdev=%p)\n", + bio->bi_op, bio->bi_rw, bio->bi_bdev); if (!dev_state->dummy_block_for_bio_bh_flush.is_iodone) { if ((dev_state->state->print_mask & (BTRFSIC_PRINT_MASK_SUBMIT_BIO_BH | @@ -3048,7 +3047,7 @@ static void __btrfsic_submit_bio(struct bio *bio) block->never_written = 0; block->iodone_w_error = 0; block->flush_gen = dev_state->last_flush_gen + 1; - block->submit_bio_bh_rw = rw; + block->submit_bio_bh_rw = bio->bi_rw; block->orig_bio_bh_private = bio->bi_private; block->orig_bio_bh_end_io.bio = bio->bi_end_io; block->next_in_same_bio = NULL; diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index c473c42..25bf179 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -363,6 +363,7 @@ int btrfs_submit_compressed_write(struct inode *inode, u64 start, kfree(cb); return -ENOMEM; } + bio->bi_op = REQ_OP_WRITE; bio->bi_private = cb; bio->bi_end_io = end_compressed_bio_write; atomic_inc(&cb->pending_bios); @@ -408,6 +409,7 @@ int btrfs_submit_compressed_write(struct inode *inode, u64 start, bio = compressed_bio_alloc(bdev, first_byte, GFP_NOFS); BUG_ON(!bio); + bio->bi_op = REQ_OP_WRITE; bio->bi_private = cb; bio->bi_end_io = end_compressed_bio_write; bio_add_page(bio, page, PAGE_CACHE_SIZE, 0); @@ -650,6 +652,7 @@ int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, comp_bio = compressed_bio_alloc(bdev, cur_disk_byte, GFP_NOFS); if (!comp_bio) goto fail2; + comp_bio->bi_op = REQ_OP_READ; comp_bio->bi_private = cb; comp_bio->bi_end_io = end_compressed_bio_read; atomic_inc(&cb->pending_bios); @@ -703,6 +706,7 @@ int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, comp_bio = compressed_bio_alloc(bdev, cur_disk_byte, GFP_NOFS); BUG_ON(!comp_bio); + comp_bio->bi_op = REQ_OP_READ; comp_bio->bi_private = cb; comp_bio->bi_end_io = end_compressed_bio_read; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index a77d88e..cd152e2 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -713,7 +713,7 @@ static void end_workqueue_bio(struct bio *bio) fs_info = end_io_wq->info; end_io_wq->error = bio->bi_error; - if (bio->bi_rw & REQ_WRITE) { + if (bio->bi_op == REQ_OP_WRITE) { if (end_io_wq->metadata == BTRFS_WQ_ENDIO_METADATA) { wq = fs_info->endio_meta_write_workers; func = btrfs_endio_meta_write_helper; @@ -859,7 +859,7 @@ int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode, atomic_inc(&fs_info->nr_async_submits); - if (rw & REQ_SYNC) + if (bio->bi_rw & REQ_SYNC) btrfs_set_work_high_priority(&async->work); btrfs_queue_work(fs_info->workers, &async->work); @@ -937,7 +937,7 @@ static int btree_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, int async = check_async_write(inode, bio_flags); int ret; - if (!(rw & REQ_WRITE)) { + if (bio->bi_op != REQ_OP_WRITE) { /* * called for a read, do the setup so that checksum validation * can happen in the async kernel threads @@ -3454,6 +3454,7 @@ static int write_dev_flush(struct btrfs_device *device, int wait) bio->bi_end_io = btrfs_end_empty_barrier; bio->bi_bdev = device->bdev; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_FLUSH; init_completion(&device->flush_wait); bio->bi_private = &device->flush_wait; diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 06f88bf..7a830c7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1895,7 +1895,7 @@ static int btrfs_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, if (btrfs_is_free_space_inode(inode)) metadata = BTRFS_WQ_ENDIO_FREE_SPACE; - if (!(rw & REQ_WRITE)) { + if (bio->bi_op != REQ_OP_WRITE) { ret = btrfs_bio_wq_end_io(root->fs_info, bio, metadata); if (ret) goto out; @@ -7713,7 +7713,7 @@ static inline int submit_dio_repair_bio(struct inode *inode, struct bio *bio, struct btrfs_root *root = BTRFS_I(inode)->root; int ret; - BUG_ON(rw & REQ_WRITE); + BUG_ON(bio->bi_op == REQ_OP_WRITE); bio_get(bio); @@ -7773,7 +7773,7 @@ static int dio_read_error(struct inode *inode, struct bio *failed_bio, int read_mode; int ret; - BUG_ON(failed_bio->bi_rw & REQ_WRITE); + BUG_ON(failed_bio->bi_op == REQ_OP_WRITE); ret = btrfs_get_io_failure_record(inode, start, end, &failrec); if (ret) @@ -7799,6 +7799,8 @@ static int dio_read_error(struct inode *inode, struct bio *failed_bio, free_io_failure(inode, failrec); return -EIO; } + bio->bi_op = REQ_OP_READ; + bio->bi_rw |= read_mode; btrfs_debug(BTRFS_I(inode)->root->fs_info, "Repair DIO Read Error: submitting new dio read[%#x] to this_mirror=%d, in_validation=%d\n", @@ -8061,8 +8063,8 @@ static void btrfs_end_dio_bio(struct bio *bio) if (err) btrfs_warn(BTRFS_I(dip->inode)->root->fs_info, - "direct IO failed ino %llu rw %lu sector %#Lx len %u err no %d", - btrfs_ino(dip->inode), bio->bi_rw, + "direct IO failed ino %llu rw %d,%lu sector %#Lx len %u err no %d", + btrfs_ino(dip->inode), bio->bi_op, bio->bi_rw, (unsigned long long)bio->bi_iter.bi_sector, bio->bi_iter.bi_size, err); @@ -8140,7 +8142,7 @@ static inline int __btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, int async_submit) { struct btrfs_dio_private *dip = bio->bi_private; - int write = rw & REQ_WRITE; + bool write = bio->bi_op == REQ_OP_WRITE; struct btrfs_root *root = BTRFS_I(inode)->root; int ret; @@ -8225,6 +8227,8 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, if (!bio) return -ENOMEM; + bio->bi_op = orig_bio->bi_op; + bio->bi_rw = orig_bio->bi_rw; bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio; btrfs_io_bio(bio)->logical = file_offset; @@ -8260,12 +8264,14 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, start_sector, GFP_NOFS); if (!bio) goto out_err; + bio->bi_op = orig_bio->bi_op; + bio->bi_rw = orig_bio->bi_rw; bio->bi_private = dip; bio->bi_end_io = btrfs_end_dio_bio; btrfs_io_bio(bio)->logical = file_offset; map_length = orig_bio->bi_iter.bi_size; - ret = btrfs_map_block(root->fs_info, rw, + ret = btrfs_map_block(root->fs_info, orig_bio->bi_op, start_sector << 9, &map_length, NULL, 0); if (ret) { diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 532ceb0..54e4941 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1323,7 +1323,7 @@ write_data: bio->bi_private = rbio; bio->bi_end_io = raid_write_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; submit_bio(bio); } @@ -1579,7 +1579,7 @@ static int raid56_rmw_stripe(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid_rmw_end_io; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); @@ -2106,7 +2106,7 @@ static int __raid56_parity_recover(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid_recover_end_io; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); @@ -2469,7 +2469,7 @@ submit_write: bio->bi_private = rbio; bio->bi_end_io = raid_write_parity_end_io; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; submit_bio(bio); } return; @@ -2647,7 +2647,7 @@ static void raid56_parity_scrub_stripe(struct btrfs_raid_bio *rbio) bio->bi_private = rbio; bio->bi_end_io = raid56_parity_scrub_end_io; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; btrfs_bio_wq_end_io(rbio->fs_info, bio, BTRFS_WQ_ENDIO_RAID56); diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 944959f..cc8f29c 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c @@ -1504,7 +1504,7 @@ static void scrub_recheck_block(struct btrfs_fs_info *fs_info, sblock->no_io_error_seen = 0; } else { bio->bi_iter.bi_sector = page->physical >> 9; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; if (btrfsic_submit_bio_wait(bio)) sblock->no_io_error_seen = 0; @@ -1586,7 +1586,7 @@ static int scrub_repair_page_from_good_copy(struct scrub_block *sblock_bad, return -EIO; bio->bi_bdev = page_bad->dev->bdev; bio->bi_iter.bi_sector = page_bad->physical >> 9; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; ret = bio_add_page(bio, page_good->page, PAGE_SIZE, 0); if (PAGE_SIZE != ret) { @@ -1687,7 +1687,7 @@ again: bio->bi_end_io = scrub_wr_bio_end_io; bio->bi_bdev = sbio->dev->bdev; bio->bi_iter.bi_sector = sbio->physical >> 9; - bio->bi_rw |= WRITE; + bio->bi_op = REQ_OP_WRITE; sbio->err = 0; } else if (sbio->physical + sbio->page_count * PAGE_SIZE != spage->physical_for_dev_replace || @@ -2091,7 +2091,7 @@ again: bio->bi_end_io = scrub_bio_end_io; bio->bi_bdev = sbio->dev->bdev; bio->bi_iter.bi_sector = sbio->physical >> 9; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; sbio->err = 0; } else if (sbio->physical + sbio->page_count * PAGE_SIZE != spage->physical || @@ -4394,6 +4394,7 @@ static int write_page_nocow(struct scrub_ctx *sctx, bio->bi_iter.bi_size = 0; bio->bi_iter.bi_sector = physical_for_dev_replace >> 9; bio->bi_bdev = dev->bdev; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_SYNC; ret = bio_add_page(bio, page, PAGE_CACHE_SIZE, 0); diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 2463761..0da1d32 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5888,7 +5888,7 @@ static void btrfs_end_bio(struct bio *bio) BUG_ON(stripe_index >= bbio->num_stripes); dev = bbio->stripes[stripe_index].dev; if (dev->bdev) { - if (bio->bi_rw & WRITE) + if (bio->bi_op == REQ_OP_WRITE) btrfs_dev_stat_inc(dev, BTRFS_DEV_STAT_WRITE_ERRS); else @@ -5953,7 +5953,7 @@ static noinline void btrfs_schedule_bio(struct btrfs_root *root, } /* don't bother with additional async steps for reads, right now */ - if (!(bio->bi_rw & REQ_WRITE)) { + if (bio->bi_op == REQ_OP_READ) { bio_get(bio); btrfsic_submit_bio(bio); bio_put(bio); @@ -6002,15 +6002,14 @@ static void submit_stripe_bio(struct btrfs_root *root, struct btrfs_bio *bbio, btrfs_io_bio(bio)->stripe_index = dev_nr; bio->bi_end_io = btrfs_end_bio; bio->bi_iter.bi_sector = physical >> 9; - bio->bi_rw |= rw; #ifdef DEBUG { struct rcu_string *name; rcu_read_lock(); name = rcu_dereference(dev->name); - pr_debug("btrfs_map_bio: rw %d, sector=%llu, dev=%lu " - "(%s id %llu), size=%u\n", rw, + pr_debug("btrfs_map_bio: rw %d 0x%x, sector=%llu, dev=%lu " + "(%s id %llu), size=%u\n", bio->bi_op, bio->bi_rw, (u64)bio->bi_iter.bi_sector, (u_long)dev->bdev->bd_dev, name->str, dev->devid, bio->bi_iter.bi_size); rcu_read_unlock(); @@ -6057,8 +6056,8 @@ int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, map_length = length; btrfs_bio_counter_inc_blocked(root->fs_info); - ret = __btrfs_map_block(root->fs_info, rw, logical, &map_length, &bbio, - mirror_num, 1); + ret = __btrfs_map_block(root->fs_info, bio->bi_op, logical, + &map_length, &bbio, mirror_num, 1); if (ret) { btrfs_bio_counter_dec(root->fs_info); return ret; @@ -6074,7 +6073,7 @@ int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, if (bbio->raid_map) { /* In this case, map_length has been set to the length of a single stripe; not the whole write */ - if (rw & WRITE) { + if (bio->bi_op == REQ_OP_WRITE) { ret = raid56_parity_write(root, bio, bbio, map_length); } else { ret = raid56_parity_recover(root, bio, bbio, map_length, @@ -6093,7 +6092,8 @@ int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, for (dev_nr = 0; dev_nr < total_devs; dev_nr++) { dev = bbio->stripes[dev_nr].dev; - if (!dev || !dev->bdev || (rw & WRITE && !dev->writeable)) { + if (!dev || !dev->bdev || + (bio->bi_op == REQ_OP_WRITE && !dev->writeable)) { bbio_error(bbio, first_bio, logical); continue; } -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:12 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id E2ECD29E14 for ; Tue, 5 Jan 2016 14:54:12 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id D4CAA8F804C for ; Tue, 5 Jan 2016 12:54:12 -0800 (PST) X-ASG-Debug-ID: 1452027251-04cb6c42e705230001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id naO0RGZRi4zMj1KN (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:11 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id D36B136209A; Tue, 5 Jan 2016 20:54:10 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu8024032; Tue, 5 Jan 2016 15:54:08 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 09/35] btrfs: update __btrfs_map_block for bi_op transition Date: Tue, 5 Jan 2016 14:53:12 -0600 X-ASG-Orig-Subj: [PATCH 09/35] btrfs: update __btrfs_map_block for bi_op transition Message-Id: <1452027218-32303-10-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027251 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie We no longer pass in a bitmap of rq_flag_bits bits to __btrfs_map_block. It will always be a REQ_OP, or the btrfs specific REQ_GET_READ_MIRRORS, so this drops the bit tests. Signed-off-by: Mike Christie --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/inode.c | 2 +- fs/btrfs/volumes.c | 55 +++++++++++++++++++++++++++----------------------- fs/btrfs/volumes.h | 4 ++-- 4 files changed, 34 insertions(+), 29 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index add4af6..4d503d0 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -2043,7 +2043,7 @@ int btrfs_discard_extent(struct btrfs_root *root, u64 bytenr, /* Tell the block device(s) that the sectors can be discarded */ - ret = btrfs_map_block(root->fs_info, REQ_DISCARD, + ret = btrfs_map_block(root->fs_info, REQ_OP_DISCARD, bytenr, &num_bytes, &bbio, 0); /* Error condition is -ENOMEM */ if (!ret) { diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 7a830c7..b1e88ec 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -8206,7 +8206,7 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, int async_submit = 0; map_length = orig_bio->bi_iter.bi_size; - ret = btrfs_map_block(root->fs_info, rw, start_sector << 9, + ret = btrfs_map_block(root->fs_info, orig_bio->bi_op, start_sector << 9, &map_length, NULL, 0); if (ret) return -EIO; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 0da1d32..bf1e9af 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5218,7 +5218,7 @@ void btrfs_put_bbio(struct btrfs_bio *bbio) kfree(bbio); } -static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, +static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int op, u64 logical, u64 *length, struct btrfs_bio **bbio_ret, int mirror_num, int need_raid_map) @@ -5296,7 +5296,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, raid56_full_stripe_start *= full_stripe_len; } - if (rw & REQ_DISCARD) { + if (op == REQ_OP_DISCARD) { /* we don't discard raid56 yet */ if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) { ret = -EOPNOTSUPP; @@ -5309,7 +5309,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, For other RAID types and for RAID[56] reads, just allow a single stripe (on a single disk). */ if ((map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) && - (rw & REQ_WRITE)) { + (op == REQ_OP_WRITE)) { max_len = stripe_len * nr_data_stripes(map) - (offset - raid56_full_stripe_start); } else { @@ -5332,8 +5332,8 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, btrfs_dev_replace_unlock(dev_replace); if (dev_replace_is_ongoing && mirror_num == map->num_stripes + 1 && - !(rw & (REQ_WRITE | REQ_DISCARD | REQ_GET_READ_MIRRORS)) && - dev_replace->tgtdev != NULL) { + op != REQ_OP_WRITE && op != REQ_OP_DISCARD && + op != REQ_GET_READ_MIRRORS && dev_replace->tgtdev != NULL) { /* * in dev-replace case, for repair case (that's the only * case where the mirror is selected explicitly when @@ -5422,15 +5422,17 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, (offset + *length); if (map->type & BTRFS_BLOCK_GROUP_RAID0) { - if (rw & REQ_DISCARD) + if (op == REQ_OP_DISCARD) num_stripes = min_t(u64, map->num_stripes, stripe_nr_end - stripe_nr_orig); stripe_nr = div_u64_rem(stripe_nr, map->num_stripes, &stripe_index); - if (!(rw & (REQ_WRITE | REQ_DISCARD | REQ_GET_READ_MIRRORS))) + if (op != REQ_OP_WRITE && op != REQ_OP_DISCARD && + op != REQ_GET_READ_MIRRORS) mirror_num = 1; } else if (map->type & BTRFS_BLOCK_GROUP_RAID1) { - if (rw & (REQ_WRITE | REQ_DISCARD | REQ_GET_READ_MIRRORS)) + if (op == REQ_OP_WRITE || op == REQ_OP_DISCARD || + op == REQ_GET_READ_MIRRORS) num_stripes = map->num_stripes; else if (mirror_num) stripe_index = mirror_num - 1; @@ -5443,7 +5445,8 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, } } else if (map->type & BTRFS_BLOCK_GROUP_DUP) { - if (rw & (REQ_WRITE | REQ_DISCARD | REQ_GET_READ_MIRRORS)) { + if (op == REQ_OP_WRITE || REQ_OP_DISCARD || + op == REQ_GET_READ_MIRRORS) { num_stripes = map->num_stripes; } else if (mirror_num) { stripe_index = mirror_num - 1; @@ -5457,9 +5460,9 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, stripe_nr = div_u64_rem(stripe_nr, factor, &stripe_index); stripe_index *= map->sub_stripes; - if (rw & (REQ_WRITE | REQ_GET_READ_MIRRORS)) + if (op == REQ_OP_WRITE || op == REQ_GET_READ_MIRRORS) num_stripes = map->sub_stripes; - else if (rw & REQ_DISCARD) + else if (op == REQ_OP_DISCARD) num_stripes = min_t(u64, map->sub_stripes * (stripe_nr_end - stripe_nr_orig), map->num_stripes); @@ -5477,7 +5480,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, } else if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) { if (need_raid_map && - ((rw & (REQ_WRITE | REQ_GET_READ_MIRRORS)) || + (op == REQ_OP_WRITE || op == REQ_GET_READ_MIRRORS || mirror_num > 1)) { /* push stripe_nr back to the start of the full stripe */ stripe_nr = div_u64(raid56_full_stripe_start, @@ -5505,8 +5508,8 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, /* We distribute the parity blocks across stripes */ div_u64_rem(stripe_nr + stripe_index, map->num_stripes, &stripe_index); - if (!(rw & (REQ_WRITE | REQ_DISCARD | - REQ_GET_READ_MIRRORS)) && mirror_num <= 1) + if ((op != REQ_OP_WRITE && op != REQ_OP_DISCARD && + op != REQ_GET_READ_MIRRORS) && mirror_num <= 1) mirror_num = 1; } } else { @@ -5523,9 +5526,9 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, num_alloc_stripes = num_stripes; if (dev_replace_is_ongoing) { - if (rw & (REQ_WRITE | REQ_DISCARD)) + if (op == REQ_OP_WRITE || op == REQ_OP_DISCARD) num_alloc_stripes <<= 1; - if (rw & REQ_GET_READ_MIRRORS) + if (op == REQ_GET_READ_MIRRORS) num_alloc_stripes++; tgtdev_indexes = num_stripes; } @@ -5540,7 +5543,8 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, /* build raid_map */ if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK && - need_raid_map && ((rw & (REQ_WRITE | REQ_GET_READ_MIRRORS)) || + need_raid_map && + ((op == REQ_OP_WRITE || op == REQ_GET_READ_MIRRORS) || mirror_num > 1)) { u64 tmp; unsigned rot; @@ -5565,7 +5569,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, RAID6_Q_STRIPE; } - if (rw & REQ_DISCARD) { + if (op == REQ_OP_DISCARD) { u32 factor = 0; u32 sub_stripes = 0; u64 stripes_per_dev = 0; @@ -5645,14 +5649,15 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, } } - if (rw & (REQ_WRITE | REQ_GET_READ_MIRRORS)) + if (op == REQ_OP_WRITE || op == REQ_GET_READ_MIRRORS) max_errors = btrfs_chunk_max_errors(map); if (bbio->raid_map) sort_parity_stripes(bbio, num_stripes); tgtdev_indexes = 0; - if (dev_replace_is_ongoing && (rw & (REQ_WRITE | REQ_DISCARD)) && + if (dev_replace_is_ongoing && + (op == REQ_OP_WRITE || op == REQ_OP_DISCARD) && dev_replace->tgtdev != NULL) { int index_where_to_add; u64 srcdev_devid = dev_replace->srcdev->devid; @@ -5687,7 +5692,7 @@ static int __btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, } } num_stripes = index_where_to_add; - } else if (dev_replace_is_ongoing && (rw & REQ_GET_READ_MIRRORS) && + } else if (dev_replace_is_ongoing && (op == REQ_GET_READ_MIRRORS) && dev_replace->tgtdev != NULL) { u64 srcdev_devid = dev_replace->srcdev->devid; int index_srcdev = 0; @@ -5760,21 +5765,21 @@ out: return ret; } -int btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, +int btrfs_map_block(struct btrfs_fs_info *fs_info, int op, u64 logical, u64 *length, struct btrfs_bio **bbio_ret, int mirror_num) { - return __btrfs_map_block(fs_info, rw, logical, length, bbio_ret, + return __btrfs_map_block(fs_info, op, logical, length, bbio_ret, mirror_num, 0); } /* For Scrub/replace */ -int btrfs_map_sblock(struct btrfs_fs_info *fs_info, int rw, +int btrfs_map_sblock(struct btrfs_fs_info *fs_info, int op, u64 logical, u64 *length, struct btrfs_bio **bbio_ret, int mirror_num, int need_raid_map) { - return __btrfs_map_block(fs_info, rw, logical, length, bbio_ret, + return __btrfs_map_block(fs_info, op, logical, length, bbio_ret, mirror_num, need_raid_map); } diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 6a4375a..8ef796a 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -421,10 +421,10 @@ int btrfs_account_dev_extents_size(struct btrfs_device *device, u64 start, u64 end, u64 *length); void btrfs_get_bbio(struct btrfs_bio *bbio); void btrfs_put_bbio(struct btrfs_bio *bbio); -int btrfs_map_block(struct btrfs_fs_info *fs_info, int rw, +int btrfs_map_block(struct btrfs_fs_info *fs_info, int op, u64 logical, u64 *length, struct btrfs_bio **bbio_ret, int mirror_num); -int btrfs_map_sblock(struct btrfs_fs_info *fs_info, int rw, +int btrfs_map_sblock(struct btrfs_fs_info *fs_info, int op, u64 logical, u64 *length, struct btrfs_bio **bbio_ret, int mirror_num, int need_raid_map); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:16 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 426E229E0C for ; Tue, 5 Jan 2016 14:54:16 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 25EB18F8050 for ; Tue, 5 Jan 2016 12:54:16 -0800 (PST) X-ASG-Debug-ID: 1452027253-04cb6c42e805240001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 2txJtNXVvTWQNVFv (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:14 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 920A0A2C16; Tue, 5 Jan 2016 20:54:13 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05Krfu9024032; Tue, 5 Jan 2016 15:54:10 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 10/35] btrfs: don't pass rq_flag_bits if there is a bio Date: Tue, 5 Jan 2016 14:53:13 -0600 X-ASG-Orig-Subj: [PATCH 10/35] btrfs: don't pass rq_flag_bits if there is a bio Message-Id: <1452027218-32303-11-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027254 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie The bio bi_op and bi_rw is now setup, so there is no need to pass around the rq_flag_bits bits too. Signed-off-by: Mike Christie --- fs/btrfs/compression.c | 9 ++++----- fs/btrfs/disk-io.c | 30 ++++++++++++------------------ fs/btrfs/disk-io.h | 2 +- fs/btrfs/extent_io.c | 16 +++++++--------- fs/btrfs/extent_io.h | 6 +++--- fs/btrfs/inode.c | 40 ++++++++++++++++++---------------------- fs/btrfs/volumes.c | 6 +++--- fs/btrfs/volumes.h | 2 +- 8 files changed, 49 insertions(+), 62 deletions(-) diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c index 25bf179..3112cc3 100644 --- a/fs/btrfs/compression.c +++ b/fs/btrfs/compression.c @@ -402,7 +402,7 @@ int btrfs_submit_compressed_write(struct inode *inode, u64 start, BUG_ON(ret); /* -ENOMEM */ } - ret = btrfs_map_bio(root, WRITE, bio, 0, 1); + ret = btrfs_map_bio(root, bio, 0, 1); BUG_ON(ret); /* -ENOMEM */ bio_put(bio); @@ -433,7 +433,7 @@ int btrfs_submit_compressed_write(struct inode *inode, u64 start, BUG_ON(ret); /* -ENOMEM */ } - ret = btrfs_map_bio(root, WRITE, bio, 0, 1); + ret = btrfs_map_bio(root, bio, 0, 1); BUG_ON(ret); /* -ENOMEM */ bio_put(bio); @@ -694,8 +694,7 @@ int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, sums += DIV_ROUND_UP(comp_bio->bi_iter.bi_size, root->sectorsize); - ret = btrfs_map_bio(root, READ, comp_bio, - mirror_num, 0); + ret = btrfs_map_bio(root, comp_bio, mirror_num, 0); if (ret) { bio->bi_error = ret; bio_endio(comp_bio); @@ -725,7 +724,7 @@ int btrfs_submit_compressed_read(struct inode *inode, struct bio *bio, BUG_ON(ret); /* -ENOMEM */ } - ret = btrfs_map_bio(root, READ, comp_bio, mirror_num, 0); + ret = btrfs_map_bio(root, comp_bio, mirror_num, 0); if (ret) { bio->bi_error = ret; bio_endio(comp_bio); diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index cd152e2..d344231 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -119,7 +119,6 @@ struct async_submit_bio { struct list_head list; extent_submit_bio_hook_t *submit_bio_start; extent_submit_bio_hook_t *submit_bio_done; - int rw; int mirror_num; unsigned long bio_flags; /* @@ -783,7 +782,7 @@ static void run_one_async_start(struct btrfs_work *work) int ret; async = container_of(work, struct async_submit_bio, work); - ret = async->submit_bio_start(async->inode, async->rw, async->bio, + ret = async->submit_bio_start(async->inode, async->bio, async->mirror_num, async->bio_flags, async->bio_offset); if (ret) @@ -816,9 +815,8 @@ static void run_one_async_done(struct btrfs_work *work) return; } - async->submit_bio_done(async->inode, async->rw, async->bio, - async->mirror_num, async->bio_flags, - async->bio_offset); + async->submit_bio_done(async->inode, async->bio, async->mirror_num, + async->bio_flags, async->bio_offset); } static void run_one_async_free(struct btrfs_work *work) @@ -830,7 +828,7 @@ static void run_one_async_free(struct btrfs_work *work) } int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode, - int rw, struct bio *bio, int mirror_num, + struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset, extent_submit_bio_hook_t *submit_bio_start, @@ -843,7 +841,6 @@ int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode, return -ENOMEM; async->inode = inode; - async->rw = rw; async->bio = bio; async->mirror_num = mirror_num; async->submit_bio_start = submit_bio_start; @@ -889,9 +886,8 @@ static int btree_csum_one_bio(struct bio *bio) return ret; } -static int __btree_submit_bio_start(struct inode *inode, int rw, - struct bio *bio, int mirror_num, - unsigned long bio_flags, +static int __btree_submit_bio_start(struct inode *inode, struct bio *bio, + int mirror_num, unsigned long bio_flags, u64 bio_offset) { /* @@ -901,7 +897,7 @@ static int __btree_submit_bio_start(struct inode *inode, int rw, return btree_csum_one_bio(bio); } -static int __btree_submit_bio_done(struct inode *inode, int rw, struct bio *bio, +static int __btree_submit_bio_done(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset) { @@ -911,7 +907,7 @@ static int __btree_submit_bio_done(struct inode *inode, int rw, struct bio *bio, * when we're called for a write, we're already in the async * submission context. Just jump into btrfs_map_bio */ - ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio, mirror_num, 1); + ret = btrfs_map_bio(BTRFS_I(inode)->root, bio, mirror_num, 1); if (ret) { bio->bi_error = ret; bio_endio(bio); @@ -930,7 +926,7 @@ static int check_async_write(struct inode *inode, unsigned long bio_flags) return 1; } -static int btree_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, +static int btree_submit_bio_hook(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset) { @@ -946,21 +942,19 @@ static int btree_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, bio, BTRFS_WQ_ENDIO_METADATA); if (ret) goto out_w_error; - ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio, - mirror_num, 0); + ret = btrfs_map_bio(BTRFS_I(inode)->root, bio, mirror_num, 0); } else if (!async) { ret = btree_csum_one_bio(bio); if (ret) goto out_w_error; - ret = btrfs_map_bio(BTRFS_I(inode)->root, rw, bio, - mirror_num, 0); + ret = btrfs_map_bio(BTRFS_I(inode)->root, bio, mirror_num, 0); } else { /* * kthread helpers are used to submit writes so that * checksumming can happen in parallel across all CPUs */ ret = btrfs_wq_submit_bio(BTRFS_I(inode)->root->fs_info, - inode, rw, bio, mirror_num, 0, + inode, bio, mirror_num, 0, bio_offset, __btree_submit_bio_start, __btree_submit_bio_done); diff --git a/fs/btrfs/disk-io.h b/fs/btrfs/disk-io.h index 7c52e29..4a5d5df 100644 --- a/fs/btrfs/disk-io.h +++ b/fs/btrfs/disk-io.h @@ -122,7 +122,7 @@ void btrfs_csum_final(u32 crc, char *result); int btrfs_bio_wq_end_io(struct btrfs_fs_info *info, struct bio *bio, enum btrfs_wq_endio_type metadata); int btrfs_wq_submit_bio(struct btrfs_fs_info *fs_info, struct inode *inode, - int rw, struct bio *bio, int mirror_num, + struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset, extent_submit_bio_hook_t *submit_bio_start, extent_submit_bio_hook_t *submit_bio_done); diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index b6c281a..25a6da0 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -2414,8 +2414,7 @@ static int bio_readpage_error(struct bio *failed_bio, u64 phy_offset, pr_debug("Repair Read Error: submitting new read[%#x] to this_mirror=%d, in_validation=%d\n", read_mode, failrec->this_mirror, failrec->in_validation); - ret = tree->ops->submit_bio_hook(inode, read_mode, bio, - failrec->this_mirror, + ret = tree->ops->submit_bio_hook(inode, bio, failrec->this_mirror, failrec->bio_flags, 0); if (ret) { free_io_failure(inode, failrec); @@ -2736,9 +2735,8 @@ static int __must_check submit_one_bio(struct bio *bio, int mirror_num, bio_get(bio); if (tree->ops && tree->ops->submit_bio_hook) - ret = tree->ops->submit_bio_hook(page->mapping->host, - bio->bi_rw, bio, mirror_num, - bio_flags, start); + ret = tree->ops->submit_bio_hook(page->mapping->host, bio, + mirror_num, bio_flags, start); else btrfsic_submit_bio(bio); @@ -2746,14 +2744,14 @@ static int __must_check submit_one_bio(struct bio *bio, int mirror_num, return ret; } -static int merge_bio(struct extent_io_tree *tree, struct page *page, +static int merge_bio(int rw, struct extent_io_tree *tree, struct page *page, unsigned long offset, size_t size, struct bio *bio, unsigned long bio_flags) { int ret = 0; if (tree->ops && tree->ops->merge_bio_hook) - ret = tree->ops->merge_bio_hook(bio->bi_op, page, offset, size, - bio, bio_flags); + ret = tree->ops->merge_bio_hook(rw, page, offset, size, bio, + bio_flags); BUG_ON(ret < 0); return ret; @@ -2787,7 +2785,7 @@ static int submit_extent_page(int op, int op_flags, struct extent_io_tree *tree, if (prev_bio_flags != bio_flags || !contig || force_bio_submit || - merge_bio(tree, page, offset, page_size, bio, bio_flags) || + merge_bio(op, tree, page, offset, page_size, bio, bio_flags) || bio_add_page(bio, page, page_size, offset) < page_size) { ret = submit_one_bio(bio, mirror_num, prev_bio_flags); if (ret < 0) { diff --git a/fs/btrfs/extent_io.h b/fs/btrfs/extent_io.h index 0377413..89f1b75 100644 --- a/fs/btrfs/extent_io.h +++ b/fs/btrfs/extent_io.h @@ -63,9 +63,9 @@ struct extent_state; struct btrfs_root; struct btrfs_io_bio; -typedef int (extent_submit_bio_hook_t)(struct inode *inode, int rw, - struct bio *bio, int mirror_num, - unsigned long bio_flags, u64 bio_offset); +typedef int (extent_submit_bio_hook_t)(struct inode *inode, struct bio *bio, + int mirror_num, unsigned long bio_flags, + u64 bio_offset); struct extent_io_ops { int (*fill_delalloc)(struct inode *inode, struct page *locked_page, u64 start, u64 end, int *page_started, diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b1e88ec..a96984f 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1840,9 +1840,8 @@ int btrfs_merge_bio_hook(int rw, struct page *page, unsigned long offset, * At IO completion time the cums attached on the ordered extent record * are inserted into the btree */ -static int __btrfs_submit_bio_start(struct inode *inode, int rw, - struct bio *bio, int mirror_num, - unsigned long bio_flags, +static int __btrfs_submit_bio_start(struct inode *inode, struct bio *bio, + int mirror_num, unsigned long bio_flags, u64 bio_offset) { struct btrfs_root *root = BTRFS_I(inode)->root; @@ -1861,14 +1860,14 @@ static int __btrfs_submit_bio_start(struct inode *inode, int rw, * At IO completion time the cums attached on the ordered extent record * are inserted into the btree */ -static int __btrfs_submit_bio_done(struct inode *inode, int rw, struct bio *bio, +static int __btrfs_submit_bio_done(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset) { struct btrfs_root *root = BTRFS_I(inode)->root; int ret; - ret = btrfs_map_bio(root, rw, bio, mirror_num, 1); + ret = btrfs_map_bio(root, bio, mirror_num, 1); if (ret) { bio->bi_error = ret; bio_endio(bio); @@ -1880,7 +1879,7 @@ static int __btrfs_submit_bio_done(struct inode *inode, int rw, struct bio *bio, * extent_io.c submission hook. This does the right thing for csum calculation * on write, or reading the csums from the tree before a read */ -static int btrfs_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, +static int btrfs_submit_bio_hook(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 bio_offset) { @@ -1917,7 +1916,7 @@ static int btrfs_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, goto mapit; /* we're doing a write, do the async checksumming */ ret = btrfs_wq_submit_bio(BTRFS_I(inode)->root->fs_info, - inode, rw, bio, mirror_num, + inode, bio, mirror_num, bio_flags, bio_offset, __btrfs_submit_bio_start, __btrfs_submit_bio_done); @@ -1929,7 +1928,7 @@ static int btrfs_submit_bio_hook(struct inode *inode, int rw, struct bio *bio, } mapit: - ret = btrfs_map_bio(root, rw, bio, mirror_num, 0); + ret = btrfs_map_bio(root, bio, mirror_num, 0); out: if (ret < 0) { @@ -7708,7 +7707,7 @@ err: } static inline int submit_dio_repair_bio(struct inode *inode, struct bio *bio, - int rw, int mirror_num) + int mirror_num) { struct btrfs_root *root = BTRFS_I(inode)->root; int ret; @@ -7722,7 +7721,7 @@ static inline int submit_dio_repair_bio(struct inode *inode, struct bio *bio, if (ret) goto err; - ret = btrfs_map_bio(root, rw, bio, mirror_num, 0); + ret = btrfs_map_bio(root, bio, mirror_num, 0); err: bio_put(bio); return ret; @@ -7806,8 +7805,7 @@ static int dio_read_error(struct inode *inode, struct bio *failed_bio, "Repair DIO Read Error: submitting new dio read[%#x] to this_mirror=%d, in_validation=%d\n", read_mode, failrec->this_mirror, failrec->in_validation); - ret = submit_dio_repair_bio(inode, bio, read_mode, - failrec->this_mirror); + ret = submit_dio_repair_bio(inode, bio, failrec->this_mirror); if (ret) { free_io_failure(inode, failrec); bio_put(bio); @@ -8045,7 +8043,7 @@ static void btrfs_endio_direct_write(struct bio *bio) bio_put(bio); } -static int __btrfs_submit_bio_start_direct_io(struct inode *inode, int rw, +static int __btrfs_submit_bio_start_direct_io(struct inode *inode, struct bio *bio, int mirror_num, unsigned long bio_flags, u64 offset) { @@ -8138,7 +8136,7 @@ static inline int btrfs_lookup_and_bind_dio_csum(struct btrfs_root *root, } static inline int __btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, - int rw, u64 file_offset, int skip_sum, + u64 file_offset, int skip_sum, int async_submit) { struct btrfs_dio_private *dip = bio->bi_private; @@ -8163,8 +8161,7 @@ static inline int __btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, if (write && async_submit) { ret = btrfs_wq_submit_bio(root->fs_info, - inode, rw, bio, 0, 0, - file_offset, + inode, bio, 0, 0, file_offset, __btrfs_submit_bio_start_direct_io, __btrfs_submit_bio_done); goto err; @@ -8183,13 +8180,13 @@ static inline int __btrfs_submit_dio_bio(struct bio *bio, struct inode *inode, goto err; } map: - ret = btrfs_map_bio(root, rw, bio, 0, async_submit); + ret = btrfs_map_bio(root, bio, 0, async_submit); err: bio_put(bio); return ret; } -static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, +static int btrfs_submit_direct_hook(struct btrfs_dio_private *dip, int skip_sum) { struct inode *inode = dip->inode; @@ -8245,7 +8242,7 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, * before we're done setting it up */ atomic_inc(&dip->pending_bios); - ret = __btrfs_submit_dio_bio(bio, inode, rw, + ret = __btrfs_submit_dio_bio(bio, inode, file_offset, skip_sum, async_submit); if (ret) { @@ -8286,7 +8283,7 @@ static int btrfs_submit_direct_hook(int rw, struct btrfs_dio_private *dip, } submit: - ret = __btrfs_submit_dio_bio(bio, inode, rw, file_offset, skip_sum, + ret = __btrfs_submit_dio_bio(bio, inode, file_offset, skip_sum, async_submit); if (!ret) return 0; @@ -8364,8 +8361,7 @@ static void btrfs_submit_direct(struct bio *dio_bio, struct inode *inode, dio_data->unsubmitted_oe_range_end; } - ret = btrfs_submit_direct_hook(dio_bio->bi_op | dio_bio->bi_rw, dip, - skip_sum); + ret = btrfs_submit_direct_hook(dip, skip_sum); if (!ret) return; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index bf1e9af..c281874 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -5999,7 +5999,7 @@ static noinline void btrfs_schedule_bio(struct btrfs_root *root, static void submit_stripe_bio(struct btrfs_root *root, struct btrfs_bio *bbio, struct bio *bio, u64 physical, int dev_nr, - int rw, int async) + int async) { struct btrfs_device *dev = bbio->stripes[dev_nr].dev; @@ -6044,7 +6044,7 @@ static void bbio_error(struct btrfs_bio *bbio, struct bio *bio, u64 logical) } } -int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, +int btrfs_map_bio(struct btrfs_root *root,struct bio *bio, int mirror_num, int async_submit) { struct btrfs_device *dev; @@ -6110,7 +6110,7 @@ int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, bio = first_bio; submit_stripe_bio(root, bbio, bio, - bbio->stripes[dev_nr].physical, dev_nr, rw, + bbio->stripes[dev_nr].physical, dev_nr, async_submit); } btrfs_bio_counter_dec(root->fs_info); diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 8ef796a..5a2da8a 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -437,7 +437,7 @@ int btrfs_alloc_chunk(struct btrfs_trans_handle *trans, struct btrfs_root *extent_root, u64 type); void btrfs_mapping_init(struct btrfs_mapping_tree *tree); void btrfs_mapping_tree_free(struct btrfs_mapping_tree *tree); -int btrfs_map_bio(struct btrfs_root *root, int rw, struct bio *bio, +int btrfs_map_bio(struct btrfs_root *root, struct bio *bio, int mirror_num, int async_submit); int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, fmode_t flags, void *holder); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:18 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id BA43129E26 for ; Tue, 5 Jan 2016 14:54:18 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay1.corp.sgi.com (Postfix) with ESMTP id 844448F804C for ; Tue, 5 Jan 2016 12:54:18 -0800 (PST) X-ASG-Debug-ID: 1452027256-04bdf03afa04de0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 3YWG30UvP1OamL74 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:16 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 4BA5A42E5D9; Tue, 5 Jan 2016 20:54:16 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuA024032; Tue, 5 Jan 2016 15:54:13 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 11/35] f2fs: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:14 -0600 X-ASG-Orig-Subj: [PATCH 11/35] f2fs: set bi_op to REQ_OP Message-Id: <1452027218-32303-12-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027256 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has f2fs set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. This patch is compile tested only. Signed-off-by: Mike Christie --- fs/f2fs/checkpoint.c | 10 ++++++---- fs/f2fs/data.c | 33 ++++++++++++++++++++------------- fs/f2fs/f2fs.h | 5 +++-- fs/f2fs/gc.c | 9 ++++++--- fs/f2fs/inline.c | 3 ++- fs/f2fs/node.c | 8 +++++--- fs/f2fs/segment.c | 10 +++++++--- fs/f2fs/trace.c | 8 +++++--- include/trace/events/f2fs.h | 34 +++++++++++++++++++++------------- 9 files changed, 75 insertions(+), 45 deletions(-) diff --git a/fs/f2fs/checkpoint.c b/fs/f2fs/checkpoint.c index fdd43f7..92d05d8 100644 --- a/fs/f2fs/checkpoint.c +++ b/fs/f2fs/checkpoint.c @@ -55,13 +55,14 @@ static struct page *__get_meta_page(struct f2fs_sb_info *sbi, pgoff_t index, struct f2fs_io_info fio = { .sbi = sbi, .type = META, - .rw = READ_SYNC | REQ_META | REQ_PRIO, + .op = REQ_OP_READ, + .op_flags = READ_SYNC | REQ_META | REQ_PRIO, .blk_addr = index, .encrypted_page = NULL, }; if (unlikely(!is_meta)) - fio.rw &= ~REQ_META; + fio.op_flags &= ~REQ_META; repeat: page = grab_cache_page(mapping, index); if (!page) { @@ -149,12 +150,13 @@ int ra_meta_pages(struct f2fs_sb_info *sbi, block_t start, int nrpages, struct f2fs_io_info fio = { .sbi = sbi, .type = META, - .rw = sync ? (READ_SYNC | REQ_META | REQ_PRIO) : READA, + .op = REQ_OP_READ, + .op_flags = sync ? (READ_SYNC | REQ_META | REQ_PRIO) : READA, .encrypted_page = NULL, }; if (unlikely(type == META_POR)) - fio.rw &= ~REQ_META; + fio.op_flags &= ~REQ_META; for (; nrpages-- > 0; blkno++) { diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 5325408..14757cb 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -107,11 +107,12 @@ static void __submit_merged_bio(struct f2fs_bio_info *io) if (!io->bio) return; - if (is_read_io(fio->rw)) + if (is_read_io(fio->op)) trace_f2fs_submit_read_bio(io->sbi->sb, fio, io->bio); else trace_f2fs_submit_write_bio(io->sbi->sb, fio, io->bio); - io->bio->bi_rw |= fio->rw; + io->bio->bi_op = fio->op; + io->bio->bi_rw |= fio->op_flags; submit_bio(io->bio); io->bio = NULL; @@ -130,10 +131,12 @@ void f2fs_submit_merged_bio(struct f2fs_sb_info *sbi, /* change META to META_FLUSH in the checkpoint procedure */ if (type >= META_FLUSH) { io->fio.type = META_FLUSH; + io->fio.op = REQ_OP_WRITE; if (test_opt(sbi, NOBARRIER)) - io->fio.rw = WRITE_FLUSH | REQ_META | REQ_PRIO; + io->fio.op_flags = WRITE_FLUSH | REQ_META | REQ_PRIO; else - io->fio.rw = WRITE_FLUSH_FUA | REQ_META | REQ_PRIO; + io->fio.op_flags = WRITE_FLUSH_FUA | REQ_META | + REQ_PRIO; } __submit_merged_bio(io); up_write(&io->io_rwsem); @@ -152,13 +155,14 @@ int f2fs_submit_page_bio(struct f2fs_io_info *fio) f2fs_trace_ios(fio, 0); /* Allocate a new bio */ - bio = __bio_alloc(fio->sbi, fio->blk_addr, 1, is_read_io(fio->rw)); + bio = __bio_alloc(fio->sbi, fio->blk_addr, 1, is_read_io(fio->op)); if (bio_add_page(bio, page, PAGE_CACHE_SIZE, 0) < PAGE_CACHE_SIZE) { bio_put(bio); return -EFAULT; } - bio->bi_rw |= fio->rw; + bio->bi_op = fio->op; + bio->bi_rw |= fio->op_flags; submit_bio(bio); return 0; @@ -169,7 +173,7 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) struct f2fs_sb_info *sbi = fio->sbi; enum page_type btype = PAGE_TYPE_OF_BIO(fio->type); struct f2fs_bio_info *io; - bool is_read = is_read_io(fio->rw); + bool is_read = is_read_io(fio->op); struct page *bio_page; io = is_read ? &sbi->read_io : &sbi->write_io[btype]; @@ -182,7 +186,7 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) inc_page_count(sbi, F2FS_WRITEBACK); if (io->bio && (io->last_block_in_bio != fio->blk_addr - 1 || - io->fio.rw != fio->rw)) + (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags))) __submit_merged_bio(io); alloc_new: if (io->bio == NULL) { @@ -278,7 +282,7 @@ int f2fs_get_block(struct dnode_of_data *dn, pgoff_t index) } struct page *get_read_data_page(struct inode *inode, pgoff_t index, - int rw, bool for_write) + int op_flags, bool for_write) { struct address_space *mapping = inode->i_mapping; struct dnode_of_data dn; @@ -288,7 +292,8 @@ struct page *get_read_data_page(struct inode *inode, pgoff_t index, struct f2fs_io_info fio = { .sbi = F2FS_I_SB(inode), .type = DATA, - .rw = rw, + .op = REQ_OP_READ, + .op_flags = op_flags, .encrypted_page = NULL, }; @@ -984,7 +989,7 @@ submit_and_realloc: bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(block_nr); bio->bi_end_io = f2fs_read_end_io; bio->bi_private = ctx; - bio->bi_rw |= READ; + bio->bi_op = REQ_OP_READ; } if (bio_add_page(bio, page, blocksize, 0) < blocksize) @@ -1118,7 +1123,8 @@ static int f2fs_write_data_page(struct page *page, struct f2fs_io_info fio = { .sbi = sbi, .type = DATA, - .rw = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : WRITE, + .op = REQ_OP_WRITE, + .op_flags = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : 0, .page = page, .encrypted_page = NULL, }; @@ -1489,7 +1495,8 @@ put_next: struct f2fs_io_info fio = { .sbi = sbi, .type = DATA, - .rw = READ_SYNC, + .op = REQ_OP_READ, + .op_flags = READ_SYNC, .blk_addr = dn.data_blkaddr, .page = page, .encrypted_page = NULL, diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h index fda0831..6e8227b 100644 --- a/fs/f2fs/f2fs.h +++ b/fs/f2fs/f2fs.h @@ -683,13 +683,14 @@ enum page_type { struct f2fs_io_info { struct f2fs_sb_info *sbi; /* f2fs_sb_info pointer */ enum page_type type; /* contains DATA/NODE/META/META_FLUSH */ - int rw; /* contains R/RS/W/WS with REQ_META/REQ_PRIO */ + int op; /* contains REQ_OP_ */ + int op_flags; /* rq_flag_bits */ block_t blk_addr; /* block address to be written */ struct page *page; /* page to be written */ struct page *encrypted_page; /* encrypted page */ }; -#define is_read_io(rw) (((rw) & 1) == READ) +#define is_read_io(rw) (rw == READ) struct f2fs_bio_info { struct f2fs_sb_info *sbi; /* f2fs superblock */ struct bio *bio; /* bios to merge */ diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c index ce350c4..8561cf5 100644 --- a/fs/f2fs/gc.c +++ b/fs/f2fs/gc.c @@ -540,7 +540,8 @@ static void move_encrypted_block(struct inode *inode, block_t bidx) struct f2fs_io_info fio = { .sbi = F2FS_I_SB(inode), .type = DATA, - .rw = READ_SYNC, + .op = REQ_OP_READ, + .op_flags = READ_SYNC, .encrypted_page = NULL, }; struct dnode_of_data dn; @@ -607,7 +608,8 @@ static void move_encrypted_block(struct inode *inode, block_t bidx) f2fs_wait_on_page_writeback(dn.node_page, NODE); allocate_data_block(fio.sbi, NULL, fio.blk_addr, &fio.blk_addr, &sum, CURSEG_COLD_DATA); - fio.rw = WRITE_SYNC; + fio.op = REQ_OP_WRITE; + fio.op_flags = WRITE_SYNC; f2fs_submit_page_mbio(&fio); dn.data_blkaddr = fio.blk_addr; @@ -641,7 +643,8 @@ static void move_data_page(struct inode *inode, block_t bidx, int gc_type) struct f2fs_io_info fio = { .sbi = F2FS_I_SB(inode), .type = DATA, - .rw = WRITE_SYNC, + .op = REQ_OP_WRITE, + .op_flags = WRITE_SYNC, .page = page, .encrypted_page = NULL, }; diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c index bda7126..d8c89f8 100644 --- a/fs/f2fs/inline.c +++ b/fs/f2fs/inline.c @@ -112,7 +112,8 @@ int f2fs_convert_inline_page(struct dnode_of_data *dn, struct page *page) struct f2fs_io_info fio = { .sbi = F2FS_I_SB(dn->inode), .type = DATA, - .rw = WRITE_SYNC | REQ_PRIO, + .op = REQ_OP_WRITE, + .op_flags = WRITE_SYNC | REQ_PRIO, .page = page, .encrypted_page = NULL, }; diff --git a/fs/f2fs/node.c b/fs/f2fs/node.c index d842b19..7261a29 100644 --- a/fs/f2fs/node.c +++ b/fs/f2fs/node.c @@ -1001,14 +1001,15 @@ fail: * 0: f2fs_put_page(page, 0) * LOCKED_PAGE or error: f2fs_put_page(page, 1) */ -static int read_node_page(struct page *page, int rw) +static int read_node_page(struct page *page, int op_flags) { struct f2fs_sb_info *sbi = F2FS_P_SB(page); struct node_info ni; struct f2fs_io_info fio = { .sbi = sbi, .type = NODE, - .rw = rw, + .op = REQ_OP_READ, + .op_flags = op_flags, .page = page, .encrypted_page = NULL, }; @@ -1309,7 +1310,8 @@ static int f2fs_write_node_page(struct page *page, struct f2fs_io_info fio = { .sbi = sbi, .type = NODE, - .rw = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : WRITE, + .op = REQ_OP_WRITE, + .op_flags = (wbc->sync_mode == WB_SYNC_ALL) ? WRITE_SYNC : 0, .page = page, .encrypted_page = NULL, }; diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index 6ebdbb3..f2ceb8a 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -200,7 +200,8 @@ int commit_inmem_pages(struct inode *inode, bool abort) struct f2fs_io_info fio = { .sbi = sbi, .type = DATA, - .rw = WRITE_SYNC | REQ_PRIO, + .op = REQ_OP_WRITE, + .op_flags = WRITE_SYNC | REQ_PRIO, .encrypted_page = NULL, }; int err = 0; @@ -318,6 +319,7 @@ repeat: fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); bio->bi_bdev = sbi->sb->s_bdev; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_FLUSH; ret = submit_bio_wait(bio); @@ -351,6 +353,7 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) int ret; bio->bi_bdev = sbi->sb->s_bdev; + bio->bi_op = REQ_OP_WRITE; bio->bi_rw |= WRITE_FLUSH; ret = submit_bio_wait(bio); bio_put(bio); @@ -1290,14 +1293,15 @@ void write_meta_page(struct f2fs_sb_info *sbi, struct page *page) struct f2fs_io_info fio = { .sbi = sbi, .type = META, - .rw = WRITE_SYNC | REQ_META | REQ_PRIO, + .op = REQ_OP_WRITE, + .op_flags = WRITE_SYNC | REQ_META | REQ_PRIO, .blk_addr = page->index, .page = page, .encrypted_page = NULL, }; if (unlikely(page->index >= MAIN_BLKADDR(sbi))) - fio.rw &= ~REQ_META; + fio.op_flags &= ~REQ_META; set_page_writeback(page); f2fs_submit_page_mbio(&fio); diff --git a/fs/f2fs/trace.c b/fs/f2fs/trace.c index 145fb65..5b7edca 100644 --- a/fs/f2fs/trace.c +++ b/fs/f2fs/trace.c @@ -25,11 +25,12 @@ static inline void __print_last_io(void) if (!last_io.len) return; - trace_printk("%3x:%3x %4x %-16s %2x %5x %12x %4x\n", + trace_printk("%3x:%3x %4x %-16s %2x %5x %5x %12x %4x\n", last_io.major, last_io.minor, last_io.pid, "----------------", last_io.type, - last_io.fio.rw, last_io.fio.blk_addr, + last_io.fio.op, last_io.fio.op_flags, + last_io.fio.blk_addr, last_io.len); memset(&last_io, 0, sizeof(last_io)); } @@ -100,7 +101,8 @@ void f2fs_trace_ios(struct f2fs_io_info *fio, int flush) if (last_io.major == major && last_io.minor == minor && last_io.pid == pid && last_io.type == __file_type(inode, pid) && - last_io.fio.rw == fio->rw && + last_io.fio.op == fio->op && + last_io.fio.op_flags == fio->op_flags && last_io.fio.blk_addr + last_io.len == fio->blk_addr) { last_io.len++; return; diff --git a/include/trace/events/f2fs.h b/include/trace/events/f2fs.h index a1b4888..ddf4144 100644 --- a/include/trace/events/f2fs.h +++ b/include/trace/events/f2fs.h @@ -55,17 +55,21 @@ TRACE_DEFINE_ENUM(CP_DISCARD); { IPU, "IN-PLACE" }, \ { OPU, "OUT-OF-PLACE" }) -#define F2FS_BIO_MASK(t) (t & (READA | WRITE_FLUSH_FUA)) +#define F2FS_BIO_FLAG_MASK(t) (t & (READA | WRITE_FLUSH_FUA)) #define F2FS_BIO_EXTRA_MASK(t) (t & (REQ_META | REQ_PRIO)) -#define show_bio_type(type) show_bio_base(type), show_bio_extra(type) +#define show_bio_type(op, op_flags) show_bio_op(op), \ + show_bio_op_flags(op_flags), show_bio_extra(op_flags) -#define show_bio_base(type) \ - __print_symbolic(F2FS_BIO_MASK(type), \ +#define show_bio_op(op) \ + __print_symbolic(op, \ { READ, "READ" }, \ + { WRITE, "WRITE" }) + +#define show_bio_op_flags(flags) \ + __print_symbolic(F2FS_BIO_FLAG_MASK(flags), \ { READA, "READAHEAD" }, \ { READ_SYNC, "READ_SYNC" }, \ - { WRITE, "WRITE" }, \ { WRITE_SYNC, "WRITE_SYNC" }, \ { WRITE_FLUSH, "WRITE_FLUSH" }, \ { WRITE_FUA, "WRITE_FUA" }, \ @@ -728,7 +732,8 @@ DECLARE_EVENT_CLASS(f2fs__submit_page_bio, __field(ino_t, ino) __field(pgoff_t, index) __field(block_t, blkaddr) - __field(int, rw) + __field(int, op) + __field(int, op_flags) __field(int, type) ), @@ -737,16 +742,17 @@ DECLARE_EVENT_CLASS(f2fs__submit_page_bio, __entry->ino = page->mapping->host->i_ino; __entry->index = page->index; __entry->blkaddr = fio->blk_addr; - __entry->rw = fio->rw; + __entry->op = fio->op; + __entry->op_flags = fio->op_flags; __entry->type = fio->type; ), TP_printk("dev = (%d,%d), ino = %lu, page_index = 0x%lx, " - "blkaddr = 0x%llx, rw = %s%s, type = %s", + "blkaddr = 0x%llx, rw = %s%s%s, type = %s", show_dev_ino(__entry), (unsigned long)__entry->index, (unsigned long long)__entry->blkaddr, - show_bio_type(__entry->rw), + show_bio_type(__entry->op, __entry->op_flags), show_block_type(__entry->type)) ); @@ -777,7 +783,8 @@ DECLARE_EVENT_CLASS(f2fs__submit_bio, TP_STRUCT__entry( __field(dev_t, dev) - __field(int, rw) + __field(int, op) + __field(int, op_flags) __field(int, type) __field(sector_t, sector) __field(unsigned int, size) @@ -785,15 +792,16 @@ DECLARE_EVENT_CLASS(f2fs__submit_bio, TP_fast_assign( __entry->dev = sb->s_dev; - __entry->rw = fio->rw; + __entry->op = fio->op; + __entry->op_flags = fio->op_flags; __entry->type = fio->type; __entry->sector = bio->bi_iter.bi_sector; __entry->size = bio->bi_iter.bi_size; ), - TP_printk("dev = (%d,%d), %s%s, %s, sector = %lld, size = %u", + TP_printk("dev = (%d,%d), %s%s%s, %s, sector = %lld, size = %u", show_dev(__entry), - show_bio_type(__entry->rw), + show_bio_type(__entry->op, __entry->op_flags), show_block_type(__entry->type), (unsigned long long)__entry->sector, __entry->size) -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:20 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id CAAEE29E30 for ; Tue, 5 Jan 2016 14:54:20 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 931C5304066 for ; Tue, 5 Jan 2016 12:54:20 -0800 (PST) X-ASG-Debug-ID: 1452027259-04bdf03af904de0001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id CHWJ1jCmyz6V9ZDj (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:19 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id EF1456E797; Tue, 5 Jan 2016 20:54:18 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuB024032; Tue, 5 Jan 2016 15:54:16 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 12/35] gfs2: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:15 -0600 X-ASG-Orig-Subj: [PATCH 12/35] gfs2: set bi_op to REQ_OP Message-Id: <1452027218-32303-13-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027259 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has gfs2 set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. This patch is compile tested only. v2: Bob, I did not add your signed off, because there was the gfs2_submit_bhs changes since last time you reviewed it. Signed-off-by: Mike Christie --- fs/gfs2/log.c | 8 ++++---- fs/gfs2/lops.c | 12 +++++++----- fs/gfs2/lops.h | 2 +- fs/gfs2/meta_io.c | 8 +++++--- fs/gfs2/ops_fstype.c | 1 + 5 files changed, 18 insertions(+), 13 deletions(-) diff --git a/fs/gfs2/log.c b/fs/gfs2/log.c index 0ff028c..e58ccef0 100644 --- a/fs/gfs2/log.c +++ b/fs/gfs2/log.c @@ -657,7 +657,7 @@ static void log_write_header(struct gfs2_sbd *sdp, u32 flags) struct gfs2_log_header *lh; unsigned int tail; u32 hash; - int rw = WRITE_FLUSH_FUA | REQ_META; + int op_flags = WRITE_FLUSH_FUA | REQ_META; struct page *page = mempool_alloc(gfs2_page_pool, GFP_NOIO); enum gfs2_freeze_state state = atomic_read(&sdp->sd_freeze_state); lh = page_address(page); @@ -682,12 +682,12 @@ static void log_write_header(struct gfs2_sbd *sdp, u32 flags) if (test_bit(SDF_NOBARRIERS, &sdp->sd_flags)) { gfs2_ordered_wait(sdp); log_flush_wait(sdp); - rw = WRITE_SYNC | REQ_META | REQ_PRIO; + op_flags = WRITE_SYNC | REQ_META | REQ_PRIO; } sdp->sd_log_idle = (tail == sdp->sd_log_flush_head); gfs2_log_write_page(sdp, page); - gfs2_log_flush_bio(sdp, rw); + gfs2_log_flush_bio(sdp, REQ_OP_WRITE, op_flags); log_flush_wait(sdp); if (sdp->sd_log_tail != tail) @@ -738,7 +738,7 @@ void gfs2_log_flush(struct gfs2_sbd *sdp, struct gfs2_glock *gl, gfs2_ordered_write(sdp); lops_before_commit(sdp, tr); - gfs2_log_flush_bio(sdp, WRITE); + gfs2_log_flush_bio(sdp, REQ_OP_WRITE, 0); if (sdp->sd_log_head != sdp->sd_log_flush_head) { log_flush_wait(sdp); diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c index acc5ccb..11980f6 100644 --- a/fs/gfs2/lops.c +++ b/fs/gfs2/lops.c @@ -230,17 +230,19 @@ static void gfs2_end_log_write(struct bio *bio) /** * gfs2_log_flush_bio - Submit any pending log bio * @sdp: The superblock - * @rw: The rw flags + * @op: REQ_OP + * @op_flags: rq_flag_bits * * Submit any pending part-built or full bio to the block device. If * there is no pending bio, then this is a no-op. */ -void gfs2_log_flush_bio(struct gfs2_sbd *sdp, int rw) +void gfs2_log_flush_bio(struct gfs2_sbd *sdp, int op, int op_flags) { if (sdp->sd_log_bio) { atomic_inc(&sdp->sd_log_in_flight); - sdp->sd_log_bio->bi_rw |= rw; + sdp->sd_log_bio->bi_op = op; + sdp->sd_log_bio->bi_rw |= op_flags; submit_bio(sdp->sd_log_bio); sdp->sd_log_bio = NULL; } @@ -300,7 +302,7 @@ static struct bio *gfs2_log_get_bio(struct gfs2_sbd *sdp, u64 blkno) nblk >>= sdp->sd_fsb2bb_shift; if (blkno == nblk) return bio; - gfs2_log_flush_bio(sdp, WRITE); + gfs2_log_flush_bio(sdp, REQ_OP_WRITE, 0); } return gfs2_log_alloc_bio(sdp, blkno); @@ -329,7 +331,7 @@ static void gfs2_log_write(struct gfs2_sbd *sdp, struct page *page, bio = gfs2_log_get_bio(sdp, blkno); ret = bio_add_page(bio, page, size, offset); if (ret == 0) { - gfs2_log_flush_bio(sdp, WRITE); + gfs2_log_flush_bio(sdp, REQ_OP_WRITE, 0); bio = gfs2_log_alloc_bio(sdp, blkno); ret = bio_add_page(bio, page, size, offset); WARN_ON(ret == 0); diff --git a/fs/gfs2/lops.h b/fs/gfs2/lops.h index a65a7ba..e529f53 100644 --- a/fs/gfs2/lops.h +++ b/fs/gfs2/lops.h @@ -27,7 +27,7 @@ extern const struct gfs2_log_operations gfs2_databuf_lops; extern const struct gfs2_log_operations *gfs2_log_ops[]; extern void gfs2_log_write_page(struct gfs2_sbd *sdp, struct page *page); -extern void gfs2_log_flush_bio(struct gfs2_sbd *sdp, int rw); +extern void gfs2_log_flush_bio(struct gfs2_sbd *sdp, int op, int op_flags); extern void gfs2_pin(struct gfs2_sbd *sdp, struct buffer_head *bh); static inline unsigned int buf_limit(struct gfs2_sbd *sdp) diff --git a/fs/gfs2/meta_io.c b/fs/gfs2/meta_io.c index e2fc2b9..3996c1d 100644 --- a/fs/gfs2/meta_io.c +++ b/fs/gfs2/meta_io.c @@ -213,7 +213,8 @@ static void gfs2_meta_read_endio(struct bio *bio) * Submit several consecutive buffer head I/O requests as a single bio I/O * request. (See submit_bh_wbc.) */ -static void gfs2_submit_bhs(int rw, struct buffer_head *bhs[], int num) +static void gfs2_submit_bhs(int op, int op_flags, struct buffer_head *bhs[], + int num) { struct buffer_head *bh = bhs[0]; struct bio *bio; @@ -230,7 +231,8 @@ static void gfs2_submit_bhs(int rw, struct buffer_head *bhs[], int num) bio_add_page(bio, bh->b_page, bh->b_size, bh_offset(bh)); } bio->bi_end_io = gfs2_meta_read_endio; - bio->bi_rw |= rw; + bio->bi_op = op; + bio->bi_rw |= op_flags; submit_bio(bio); } @@ -281,7 +283,7 @@ int gfs2_meta_read(struct gfs2_glock *gl, u64 blkno, int flags, } } - gfs2_submit_bhs(READ_SYNC | REQ_META | REQ_PRIO, bhs, num); + gfs2_submit_bhs(REQ_OP_READ, READ_SYNC | REQ_META | REQ_PRIO, bhs, num); if (!(flags & DIO_WAIT)) return 0; diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c index 15a4f86..cc8600a 100644 --- a/fs/gfs2/ops_fstype.c +++ b/fs/gfs2/ops_fstype.c @@ -246,6 +246,7 @@ static int gfs2_read_super(struct gfs2_sbd *sdp, sector_t sector, int silent) bio->bi_end_io = end_bio_io_page; bio->bi_private = page; + bio->bi_op = REQ_OP_READ; bio->bi_rw |= READ_SYNC | REQ_META; submit_bio(bio); wait_on_page_locked(page); -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:23 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111]) by oss.sgi.com (Postfix) with ESMTP id 662FE29E17 for ; Tue, 5 Jan 2016 14:54:23 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay1.corp.sgi.com (Postfix) with ESMTP id 4F1508F8040 for ; Tue, 5 Jan 2016 12:54:23 -0800 (PST) X-ASG-Debug-ID: 1452027262-04cb6c42e505250001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id EA2B2EcBFxB1xsu7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:22 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id CD1043B757; Tue, 5 Jan 2016 20:54:21 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuC024032; Tue, 5 Jan 2016 15:54:19 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 13/35] xfs: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:16 -0600 X-ASG-Orig-Subj: [PATCH 13/35] xfs: set bi_op to REQ_OP Message-Id: <1452027218-32303-14-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027262 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has xfs set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. Note: I have run xfs tests on these btrfs patches. There were some failures with and without the patches. I have not had time to track down why xfstest fails without the patches. Signed-off-by: Mike Christie --- fs/xfs/xfs_aops.c | 3 ++- fs/xfs/xfs_buf.c | 27 +++++++++++++++------------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index a1052d2..3a00935 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -382,7 +382,8 @@ xfs_submit_ioend_bio( atomic_inc(&ioend->io_remaining); bio->bi_private = ioend; bio->bi_end_io = xfs_end_bio; - bio->bi_rw |= (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + bio->bi_op = REQ_OP_WRITE; + bio->bi_rw |= WB_SYNC_ALL ? WRITE_SYNC : 0; submit_bio(bio); } diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 644e676..4cfba72 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -1124,7 +1124,8 @@ xfs_buf_ioapply_map( int map, int *buf_offset, int *count, - int rw) + int op, + int op_flags) { int page_index; int total_nr_pages = bp->b_page_count; @@ -1163,7 +1164,8 @@ next_chunk: bio->bi_iter.bi_sector = sector; bio->bi_end_io = xfs_buf_bio_end_io; bio->bi_private = bp; - bio->bi_rw |= rw; + bio->bi_op = op; + bio->bi_rw |= op_flags; for (; size && nr_pages; nr_pages--, page_index++) { int rbytes, nbytes = PAGE_SIZE - offset; @@ -1207,7 +1209,8 @@ _xfs_buf_ioapply( struct xfs_buf *bp) { struct blk_plug plug; - int rw; + int op; + int op_flags = 0; int offset; int size; int i; @@ -1226,14 +1229,13 @@ _xfs_buf_ioapply( bp->b_ioend_wq = bp->b_target->bt_mount->m_buf_workqueue; if (bp->b_flags & XBF_WRITE) { + op = REQ_OP_WRITE; if (bp->b_flags & XBF_SYNCIO) - rw = WRITE_SYNC; - else - rw = WRITE; + op_flags = WRITE_SYNC; if (bp->b_flags & XBF_FUA) - rw |= REQ_FUA; + op_flags |= REQ_FUA; if (bp->b_flags & XBF_FLUSH) - rw |= REQ_FLUSH; + op_flags |= REQ_FLUSH; /* * Run the write verifier callback function if it exists. If @@ -1263,13 +1265,14 @@ _xfs_buf_ioapply( } } } else if (bp->b_flags & XBF_READ_AHEAD) { - rw = READA; + op = REQ_OP_READ; + op_flags = REQ_RAHEAD; } else { - rw = READ; + op = REQ_OP_READ; } /* we only use the buffer cache for meta-data */ - rw |= REQ_META; + op_flags |= REQ_META; /* * Walk all the vectors issuing IO on them. Set up the initial offset @@ -1281,7 +1284,7 @@ _xfs_buf_ioapply( size = BBTOB(bp->b_io_length); blk_start_plug(&plug); for (i = 0; i < bp->b_map_count; i++) { - xfs_buf_ioapply_map(bp, i, &offset, &size, rw); + xfs_buf_ioapply_map(bp, i, &offset, &size, op, op_flags); if (bp->b_error) break; if (size <= 0) -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:26 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 491C529E06 for ; Tue, 5 Jan 2016 14:54:26 -0600 (CST) Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by relay2.corp.sgi.com (Postfix) with ESMTP id 3B888304039 for ; Tue, 5 Jan 2016 12:54:26 -0800 (PST) X-ASG-Debug-ID: 1452027264-04cb6c42e705260001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id Km4TJCLr9QqGPMUN (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:25 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 75C4B8E931; Tue, 5 Jan 2016 20:54:24 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuD024032; Tue, 5 Jan 2016 15:54:21 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 14/35] hfsplus: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:17 -0600 X-ASG-Orig-Subj: [PATCH 14/35] hfsplus: set bi_op to REQ_OP Message-Id: <1452027218-32303-15-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027265 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has hfsplus set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. This patch is compile tested only. Signed-off-by: Mike Christie --- fs/hfsplus/hfsplus_fs.h | 2 +- fs/hfsplus/part_tbl.c | 5 +++-- fs/hfsplus/super.c | 6 ++++-- fs/hfsplus/wrapper.c | 15 +++++++++------ 4 files changed, 17 insertions(+), 11 deletions(-) diff --git a/fs/hfsplus/hfsplus_fs.h b/fs/hfsplus/hfsplus_fs.h index f91a1fa..80154aa 100644 --- a/fs/hfsplus/hfsplus_fs.h +++ b/fs/hfsplus/hfsplus_fs.h @@ -525,7 +525,7 @@ int hfsplus_compare_dentry(const struct dentry *parent, /* wrapper.c */ int hfsplus_submit_bio(struct super_block *sb, sector_t sector, void *buf, - void **data, int rw); + void **data, int op, int op_flags); int hfsplus_read_wrapper(struct super_block *sb); /* time macros */ diff --git a/fs/hfsplus/part_tbl.c b/fs/hfsplus/part_tbl.c index eb355d8..63164eb 100644 --- a/fs/hfsplus/part_tbl.c +++ b/fs/hfsplus/part_tbl.c @@ -112,7 +112,8 @@ static int hfs_parse_new_pmap(struct super_block *sb, void *buf, if ((u8 *)pm - (u8 *)buf >= buf_size) { res = hfsplus_submit_bio(sb, *part_start + HFS_PMAP_BLK + i, - buf, (void **)&pm, READ); + buf, (void **)&pm, REQ_OP_READ, + 0); if (res) return res; } @@ -136,7 +137,7 @@ int hfs_part_find(struct super_block *sb, return -ENOMEM; res = hfsplus_submit_bio(sb, *part_start + HFS_PMAP_BLK, - buf, &data, READ); + buf, &data, REQ_OP_READ, 0); if (res) goto out; diff --git a/fs/hfsplus/super.c b/fs/hfsplus/super.c index 5d54490..01cf313 100644 --- a/fs/hfsplus/super.c +++ b/fs/hfsplus/super.c @@ -219,7 +219,8 @@ static int hfsplus_sync_fs(struct super_block *sb, int wait) error2 = hfsplus_submit_bio(sb, sbi->part_start + HFSPLUS_VOLHEAD_SECTOR, - sbi->s_vhdr_buf, NULL, WRITE_SYNC); + sbi->s_vhdr_buf, NULL, REQ_OP_WRITE, + WRITE_SYNC); if (!error) error = error2; if (!write_backup) @@ -227,7 +228,8 @@ static int hfsplus_sync_fs(struct super_block *sb, int wait) error2 = hfsplus_submit_bio(sb, sbi->part_start + sbi->sect_count - 2, - sbi->s_backup_vhdr_buf, NULL, WRITE_SYNC); + sbi->s_backup_vhdr_buf, NULL, REQ_OP_WRITE, + WRITE_SYNC); if (!error) error2 = error; out: diff --git a/fs/hfsplus/wrapper.c b/fs/hfsplus/wrapper.c index 7e605b5..d09b726 100644 --- a/fs/hfsplus/wrapper.c +++ b/fs/hfsplus/wrapper.c @@ -30,7 +30,8 @@ struct hfsplus_wd { * @sector: block to read or write, for blocks of HFSPLUS_SECTOR_SIZE bytes * @buf: buffer for I/O * @data: output pointer for location of requested data - * @rw: direction of I/O + * @op: direction of I/O + * @op_flags: request op flags * * The unit of I/O is hfsplus_min_io_size(sb), which may be bigger than * HFSPLUS_SECTOR_SIZE, and @buf must be sized accordingly. On reads @@ -44,7 +45,7 @@ struct hfsplus_wd { * will work correctly. */ int hfsplus_submit_bio(struct super_block *sb, sector_t sector, - void *buf, void **data, int rw) + void *buf, void **data, int op, int op_flags) { struct bio *bio; int ret = 0; @@ -65,9 +66,10 @@ int hfsplus_submit_bio(struct super_block *sb, sector_t sector, bio = bio_alloc(GFP_NOIO, 1); bio->bi_iter.bi_sector = sector; bio->bi_bdev = sb->s_bdev; - bio->bi_rw |= rw; + bio->bi_op = op; + bio->bi_rw |= op_flags; - if (!(rw & WRITE) && data) + if (op != WRITE && data) *data = (u8 *)buf + offset; while (io_size > 0) { @@ -182,7 +184,7 @@ int hfsplus_read_wrapper(struct super_block *sb) reread: error = hfsplus_submit_bio(sb, part_start + HFSPLUS_VOLHEAD_SECTOR, sbi->s_vhdr_buf, (void **)&sbi->s_vhdr, - READ); + REQ_OP_READ, 0); if (error) goto out_free_backup_vhdr; @@ -214,7 +216,8 @@ reread: error = hfsplus_submit_bio(sb, part_start + part_size - 2, sbi->s_backup_vhdr_buf, - (void **)&sbi->s_backup_vhdr, READ); + (void **)&sbi->s_backup_vhdr, REQ_OP_READ, + 0); if (error) goto out_free_backup_vhdr; -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:29 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D859F29E37 for ; Tue, 5 Jan 2016 14:54:28 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay3.corp.sgi.com (Postfix) with ESMTP id 7B548AC001 for ; Tue, 5 Jan 2016 12:54:28 -0800 (PST) X-ASG-Debug-ID: 1452027267-04bdf03afb04e00001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id oBzpPnxb61nBx7id (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:27 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 1B37914CAC6; Tue, 5 Jan 2016 20:54:27 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuE024032; Tue, 5 Jan 2016 15:54:24 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 15/35] mpage: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:18 -0600 X-ASG-Orig-Subj: [PATCH 15/35] mpage: set bi_op to REQ_OP Message-Id: <1452027218-32303-16-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027267 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has the mpage.c code set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. I have run xfstest with xfs, but I am not sure if I have stressed these code paths well. Signed-off-by: Mike Christie --- fs/mpage.c | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/fs/mpage.c b/fs/mpage.c index 9fec67f..3f7d221 100644 --- a/fs/mpage.c +++ b/fs/mpage.c @@ -55,11 +55,12 @@ static void mpage_end_io(struct bio *bio) bio_put(bio); } -static struct bio *mpage_bio_submit(int rw, struct bio *bio) +static struct bio *mpage_bio_submit(int op, int op_flags, struct bio *bio) { bio->bi_end_io = mpage_end_io; - bio->bi_rw |= rw; - guard_bio_eod(rw, bio); + bio->bi_op = op; + bio->bi_rw |= op_flags; + guard_bio_eod(op, bio); submit_bio(bio); return NULL; } @@ -269,7 +270,7 @@ do_mpage_readpage(struct bio *bio, struct page *page, unsigned nr_pages, * This page will go to BIO. Do we need to send this BIO off first? */ if (bio && (*last_block_in_bio != blocks[0] - 1)) - bio = mpage_bio_submit(READ, bio); + bio = mpage_bio_submit(REQ_OP_READ, 0, bio); alloc_new: if (bio == NULL) { @@ -286,7 +287,7 @@ alloc_new: length = first_hole << blkbits; if (bio_add_page(bio, page, length, 0) < length) { - bio = mpage_bio_submit(READ, bio); + bio = mpage_bio_submit(REQ_OP_READ, 0, bio); goto alloc_new; } @@ -294,7 +295,7 @@ alloc_new: nblocks = map_bh->b_size >> blkbits; if ((buffer_boundary(map_bh) && relative_block == nblocks) || (first_hole != blocks_per_page)) - bio = mpage_bio_submit(READ, bio); + bio = mpage_bio_submit(REQ_OP_READ, 0, bio); else *last_block_in_bio = blocks[blocks_per_page - 1]; out: @@ -302,7 +303,7 @@ out: confused: if (bio) - bio = mpage_bio_submit(READ, bio); + bio = mpage_bio_submit(REQ_OP_READ, 0, bio); if (!PageUptodate(page)) block_read_full_page(page, get_block); else @@ -384,7 +385,7 @@ mpage_readpages(struct address_space *mapping, struct list_head *pages, } BUG_ON(!list_empty(pages)); if (bio) - mpage_bio_submit(READ, bio); + mpage_bio_submit(REQ_OP_READ, 0, bio); return 0; } EXPORT_SYMBOL(mpage_readpages); @@ -405,7 +406,7 @@ int mpage_readpage(struct page *page, get_block_t get_block) bio = do_mpage_readpage(bio, page, 1, &last_block_in_bio, &map_bh, &first_logical_block, get_block, gfp); if (bio) - mpage_bio_submit(READ, bio); + mpage_bio_submit(REQ_OP_READ, 0, bio); return 0; } EXPORT_SYMBOL(mpage_readpage); @@ -486,7 +487,7 @@ static int __mpage_writepage(struct page *page, struct writeback_control *wbc, struct buffer_head map_bh; loff_t i_size = i_size_read(inode); int ret = 0; - int wr = (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE); + int op_flags = (wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : 0); if (page_has_buffers(page)) { struct buffer_head *head = page_buffers(page); @@ -595,7 +596,7 @@ page_is_mapped: * This page will go to BIO. Do we need to send this BIO off first? */ if (bio && mpd->last_block_in_bio != blocks[0] - 1) - bio = mpage_bio_submit(wr, bio); + bio = mpage_bio_submit(REQ_OP_WRITE, op_flags, bio); alloc_new: if (bio == NULL) { @@ -622,7 +623,7 @@ alloc_new: wbc_account_io(wbc, page, PAGE_SIZE); length = first_unmapped << blkbits; if (bio_add_page(bio, page, length, 0) < length) { - bio = mpage_bio_submit(wr, bio); + bio = mpage_bio_submit(REQ_OP_WRITE, op_flags, bio); goto alloc_new; } @@ -632,7 +633,7 @@ alloc_new: set_page_writeback(page); unlock_page(page); if (boundary || (first_unmapped != blocks_per_page)) { - bio = mpage_bio_submit(wr, bio); + bio = mpage_bio_submit(REQ_OP_WRITE, op_flags, bio); if (boundary_block) { write_boundary_block(boundary_bdev, boundary_block, 1 << blkbits); @@ -644,7 +645,7 @@ alloc_new: confused: if (bio) - bio = mpage_bio_submit(wr, bio); + bio = mpage_bio_submit(REQ_OP_WRITE, op_flags, bio); if (mpd->use_writepage) { ret = mapping->a_ops->writepage(page, wbc); @@ -701,9 +702,9 @@ mpage_writepages(struct address_space *mapping, ret = write_cache_pages(mapping, wbc, __mpage_writepage, &mpd); if (mpd.bio) { - int wr = (wbc->sync_mode == WB_SYNC_ALL ? - WRITE_SYNC : WRITE); - mpage_bio_submit(wr, mpd.bio); + int op_flags = (wbc->sync_mode == WB_SYNC_ALL ? + WRITE_SYNC : 0); + mpage_bio_submit(REQ_OP_WRITE, op_flags, mpd.bio); } } blk_finish_plug(&plug); @@ -722,9 +723,9 @@ int mpage_writepage(struct page *page, get_block_t get_block, }; int ret = __mpage_writepage(page, wbc, &mpd); if (mpd.bio) { - int wr = (wbc->sync_mode == WB_SYNC_ALL ? - WRITE_SYNC : WRITE); - mpage_bio_submit(wr, mpd.bio); + int op_flags = (wbc->sync_mode == WB_SYNC_ALL ? + WRITE_SYNC : 0); + mpage_bio_submit(REQ_OP_WRITE, op_flags, mpd.bio); } return ret; } -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:31 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29]) by oss.sgi.com (Postfix) with ESMTP id 838A929E04 for ; Tue, 5 Jan 2016 14:54:31 -0600 (CST) Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11]) by relay2.corp.sgi.com (Postfix) with ESMTP id 4B810304039 for ; Tue, 5 Jan 2016 12:54:31 -0800 (PST) X-ASG-Debug-ID: 1452027270-04bdf03afa04e00001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id IkSBOuCtu0i7QHVs (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:30 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id D35F019F246; Tue, 5 Jan 2016 20:54:29 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuF024032; Tue, 5 Jan 2016 15:54:27 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 16/35] nilfs: set bi_op to REQ_OP Date: Tue, 5 Jan 2016 14:53:19 -0600 X-ASG-Orig-Subj: [PATCH 16/35] nilfs: set bi_op to REQ_OP Message-Id: <1452027218-32303-17-git-send-email-mchristi@redhat.com> In-Reply-To: <1452027218-32303-1-git-send-email-mchristi@redhat.com> References: <1452027218-32303-1-git-send-email-mchristi@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Barracuda-Connect: mx1.redhat.com[209.132.183.28] X-Barracuda-Start-Time: 1452027270 X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384 X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at sgi.com X-Barracuda-BRTS-Status: 1 From: Mike Christie This patch has nilfs set the bio bi_op to a REQ_OP, and rq_flag_bits to bi_rw. This patch is compile tested only. Signed-off-by: Mike Christie --- fs/nilfs2/segbuf.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c index 428ece8..8784272 100644 --- a/fs/nilfs2/segbuf.c +++ b/fs/nilfs2/segbuf.c @@ -350,7 +350,8 @@ static void nilfs_end_bio_write(struct bio *bio) } static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf, - struct nilfs_write_info *wi, int mode) + struct nilfs_write_info *wi, int mode, + int mode_flags) { struct bio *bio = wi->bio; int err; @@ -368,7 +369,8 @@ static int nilfs_segbuf_submit_bio(struct nilfs_segment_buffer *segbuf, bio->bi_end_io = nilfs_end_bio_write; bio->bi_private = segbuf; - bio->bi_rw |= mode; + bio->bi_op = mode; + bio->bi_rw |= mode_flags; submit_bio(bio); segbuf->sb_nbio++; @@ -442,7 +444,7 @@ static int nilfs_segbuf_submit_bh(struct nilfs_segment_buffer *segbuf, return 0; } /* bio is FULL */ - err = nilfs_segbuf_submit_bio(segbuf, wi, mode); + err = nilfs_segbuf_submit_bio(segbuf, wi, mode, 0); /* never submit current bh */ if (likely(!err)) goto repeat; @@ -466,19 +468,19 @@ static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf, { struct nilfs_write_info wi; struct buffer_head *bh; - int res = 0, rw = WRITE; + int res = 0; wi.nilfs = nilfs; nilfs_segbuf_prepare_write(segbuf, &wi); list_for_each_entry(bh, &segbuf->sb_segsum_buffers, b_assoc_buffers) { - res = nilfs_segbuf_submit_bh(segbuf, &wi, bh, rw); + res = nilfs_segbuf_submit_bh(segbuf, &wi, bh, REQ_OP_WRITE); if (unlikely(res)) goto failed_bio; } list_for_each_entry(bh, &segbuf->sb_payload_buffers, b_assoc_buffers) { - res = nilfs_segbuf_submit_bh(segbuf, &wi, bh, rw); + res = nilfs_segbuf_submit_bh(segbuf, &wi, bh, REQ_OP_WRITE); if (unlikely(res)) goto failed_bio; } @@ -488,8 +490,8 @@ static int nilfs_segbuf_write(struct nilfs_segment_buffer *segbuf, * Last BIO is always sent through the following * submission. */ - rw |= REQ_SYNC; - res = nilfs_segbuf_submit_bio(segbuf, &wi, rw); + res = nilfs_segbuf_submit_bio(segbuf, &wi, REQ_OP_WRITE, + REQ_SYNC); } failed_bio: -- 1.8.3.1 From mchristi@redhat.com Tue Jan 5 14:54:35 2016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham version=3.3.1 X-Original-To: xfs@oss.sgi.com Delivered-To: xfs@oss.sgi.com Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15]) by oss.sgi.com (Postfix) with ESMTP id D31CD29E14 for ; Tue, 5 Jan 2016 14:54:34 -0600 (CST) Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by relay3.corp.sgi.com (Postfix) with ESMTP id 63BB3AC003 for ; Tue, 5 Jan 2016 12:54:34 -0800 (PST) X-ASG-Debug-ID: 1452027272-04cbb07e1a05460001-NocioJ Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id p44sfcANbVXimBbU (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 05 Jan 2016 12:54:33 -0800 (PST) X-Barracuda-Envelope-From: mchristi@redhat.com X-Barracuda-Apparent-Source-IP: 209.132.183.28 X-ASG-Whitelist: Client Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id 885A0C0B7E27; Tue, 5 Jan 2016 20:54:32 +0000 (UTC) Received: from rh2.redhat.com (vpn-63-72.rdu2.redhat.com [10.10.63.72]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u05KrfuG024032; Tue, 5 Jan 2016 15:54:29 -0500 From: mchristi@redhat.com To: linux-f2fs-devel@lists.sourceforge.net, linux-ext4@vger.kernel.org, konrad.wilk@oracle.com, drbd-dev@lists.linbit.com, philipp.reisner@linbit.com, lars.ellenberg@linbit.com, linux-raid@vger.kernel.org, dm-devel@redhat.com, linux-fsdevel@vger.kernel.org, linux-bcache@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org, linux-mtd@lists.infradead.org, target-devel@vger.kernel.org, linux-btrfs@vger.kernel.org, osd-dev@open-osd.org, xfs@oss.sgi.com, ocfs2-devel@oss.oracle.com Cc: Mike Christie Subject: [PATCH 17/35] ocfs2: set bi_op to REQ_OP Dat