From guillaume.thouvenin@bull.net Fri Sep 3 06:34:07 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 03 Sep 2004 06:34:14 -0700 (PDT) Received: from ecbull20.frec.bull.fr (ecbull20.frec.bull.fr [129.183.4.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i83DY3ZJ015889 for ; Fri, 3 Sep 2004 06:34:04 -0700 Received: from ecn002.frec.bull.fr (ecn002.frec.bull.fr [129.183.4.6]) by ecbull20.frec.bull.fr (8.9.2/8.9.1) with ESMTP id PAA17514; Fri, 3 Sep 2004 15:31:14 +0200 Received: from thouvenig ([129.183.101.50]) by ecn002.frec.bull.fr (Lotus Domino Release 5.0.12) with ESMTP id 2004090315361658:292 ; Fri, 3 Sep 2004 15:36:16 +0200 Received: from guill by thouvenig with local (Exim 4.34) id 1C3E9Q-0000mK-Ex; Fri, 03 Sep 2004 15:30:48 +0200 Date: Fri, 3 Sep 2004 15:30:48 +0200 From: Guillaume Thouvenin To: lse-tech@lists.sourceforge.net Cc: Tim Schmielau , Arthur Corliss , Jay Lan , Erik , guillaume.thouvenin@bull.net, Limin Gu , John Hesterberg , csa@oss.sgi.com Subject: Proposal for Enhanced Accounting HOWTO Message-ID: <20040903133048.GA2979@frec.bull.fr> Mime-Version: 1.0 User-Agent: Mutt/1.5.6+20040722i X-MIMETrack: Itemize by SMTP Server on ECN002/FR/BULL(Release 5.0.12 |February 13, 2003) at 03/09/2004 15:36:16, Serialize by Router on ECN002/FR/BULL(Release 5.0.12 |February 13, 2003) at 03/09/2004 15:36:41, Serialize complete at 03/09/2004 15:36:41 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-archive-position: 18 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: guillaume.thouvenin@bull.net Precedence: bulk X-list: csa Enhanced Accounting HOWTO ========================= According to discussion on the lse-tech mailing-list, it appears that three steps (at least) are required to improve accounting. 1) Improve accounting structure ---------------------------- The current BSD-accounting structure doesn't have enough informations. Metrics computed by CSA module can be added to BSD accounting. According to other discussion (like Andi Kleen's comment on the patch I wrote when I wanted to add CSA IO values in the BSD accounting ( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics about blocks/char read/write is not accurate since most writes can be accounted by some pdflush threads. Maybe add a counter in the routine mpage_writepages() but I don't know if we can recover a process ID from the struct page and I don't know if it will be enough... I'm looking if this is the right way. 2) Group of processes management ------------------------------ We need to be able to manage groups of processes as it's clear that a major accounting improvement is the per-job accounting. I don't know if "job" is the right noun. There are several implementation that already exist and some of them are already in the kernel. The property needed here is that if a process is in a container, its children will be in the same container. Different implementations can be: - PAGG + JOB (job) - ELSA (bank) - CKRM (class) - CPUSET (a cpuset of all CPUs can act as a container) The interface between kernel and user space application can be a new virtual file system (like CKRM does with /rcfs) but it can also be a device driver with ioctl operations (like ELSA does). Both solutions are interesting and need to be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and it seems that PAGG has been removed (don't know why) from -mm tree. 3) Data presentation ----------------- We can have several different implementations. 4) General overview ---------------- KERNEL SPACE | USER SPACE (or MODULE) | ---------------- | | BSD accounting | | | + | | ------------ | CSA |<=======>| | ---------------- | | Enhanced | | | Accounting | ------------- | | Core | | group of |<=======>| | | processes | | ------------ | manager | | ------------- | | Communication between EAC (Enhanced Accounting Core) and kernel space can be done via virtual fs or a device. The goal of EAC is to keep a trace of the different groups of processes during the accounting period using the group of processes management module. The idea is that the group of processes management can send a message to EAC when a process ended to indicate the job ID if any. The other thing that has to be done by EAC is the data presentation. Here is an example of what can be done: 1) First we can add processes into containers using the GPM (group of processes manager) module. For example we can add an ftp server with pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have: container 1 -> 123 container 2 -> 234 2) Now, a user can login via ssh, so, sshd will create new children. Thus, inside the GPM you will have something like: container 1 -> 123 container 2 -> 234 333 334 335 3) As soon as a process is terminated, the EAC must be aware of that fact (using a signal from GPM for example). There is just this modification that is needed. With the signal, the GPM will transfer information like pid, jib (job ID), command, ... to the EAC. This will allow to keep trace about what happens in the system. Thus, in the EAC we keep information about all processes that are finished. 4) If the sysadmin asks information about process 234, the EAC will know that it belongs to container 2 and there are other processes in this container. As all processes are exited, accounting information are written in the accounting file (like it is currently with BSD accounting). With those informations (accounting + job information) we can do per-job accounting. I am working on the EAC implementation to check how we can make it work. Any comments? Best Guillaume From jfm@ii.uib.no Fri Sep 3 07:13:08 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 03 Sep 2004 07:13:15 -0700 (PDT) Received: from eik.ii.uib.no (eik.ii.uib.no [129.177.16.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i83ED75G017589 for ; Fri, 3 Sep 2004 07:13:08 -0700 Received: from lapprose.ii.uib.no ([129.177.20.37]:43697) by eik.ii.uib.no with esmtp (TLSv1:AES256-SHA:256) (Exim 4.30) id 1C3Eo7-000642-OP; Fri, 03 Sep 2004 16:12:51 +0200 Received: (from jfm@localhost) by lapprose.ii.uib.no (8.12.11/8.12.11/Submit) id i83ECesM016519; Fri, 3 Sep 2004 16:12:40 +0200 Date: Fri, 3 Sep 2004 16:12:40 +0200 From: Jan-Frode Myklebust To: Guillaume Thouvenin Cc: lse-tech@lists.sourceforge.net, Tim Schmielau , Arthur Corliss , Jay Lan , Erik , Limin Gu , John Hesterberg , csa@oss.sgi.com Subject: Re: Proposal for Enhanced Accounting HOWTO Message-ID: <20040903141240.GB16082@ii.uib.no> References: <20040903133048.GA2979@frec.bull.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040903133048.GA2979@frec.bull.fr> X-archive-position: 19 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: janfrode@parallab.uib.no Precedence: bulk X-list: csa On Fri, Sep 03, 2004 at 03:30:48PM +0200, Guillaume Thouvenin wrote: > Enhanced Accounting HOWTO > ========================= > > Here is an example of what can be done: > > 1) First we can add processes into containers using the GPM (group of > processes manager) module. For example we can add an ftp server with > pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have: > container 1 -> 123 > container 2 -> 234 > > 2) Now, a user can login via ssh, so, sshd will create new children. Thus, > inside the GPM you will have something like: > container 1 -> 123 > container 2 -> 234 333 334 335 Can/will this support project based accounting ? i.e. Users being member of several projects. Each project having multiple users as members. And the user is allowed to decide which project-id each container is going to run under / be accounted to ? -jf From jlan@sgi.com Fri Sep 3 10:52:02 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 03 Sep 2004 10:52:08 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i83Hq0xg029662 for ; Fri, 3 Sep 2004 10:52:01 -0700 Received: from spindle.corp.sgi.com (spindle.corp.sgi.com [198.29.75.13]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i83Hpo0f025816 for ; Fri, 3 Sep 2004 12:51:50 -0500 Received: from mtv-vpn-hw-jlan-2.corp.sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [134.15.18.195]) by spindle.corp.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id i83HpgKh42096656; Fri, 3 Sep 2004 10:51:47 -0700 (PDT) Received: from sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [127.0.0.1]) by mtv-vpn-hw-jlan-2.corp.sgi.com (8.12.8/8.12.8) with ESMTP id i83HmXVN002887; Fri, 3 Sep 2004 10:48:55 -0700 Message-ID: <4138AE71.6090803@sgi.com> Date: Fri, 03 Sep 2004 10:48:33 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: zh-tw, en-us, en, zh-cn, zh-hk MIME-Version: 1.0 To: Jan-Frode Myklebust CC: Guillaume Thouvenin , lse-tech@lists.sourceforge.net, Tim Schmielau , Arthur Corliss , Erik , Limin Gu , John Hesterberg , csa@oss.sgi.com Subject: Re: [Lse-tech] Re: Proposal for Enhanced Accounting HOWTO References: <20040903133048.GA2979@frec.bull.fr> <20040903141240.GB16082@ii.uib.no> In-Reply-To: <20040903141240.GB16082@ii.uib.no> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 20 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@sgi.com Precedence: bulk X-list: csa Support of project concept is certainly one thing we need to consider. We have received requests to support project in CSA. Cheers, - jay Jan-Frode Myklebust wrote: > On Fri, Sep 03, 2004 at 03:30:48PM +0200, Guillaume Thouvenin wrote: > >> Enhanced Accounting HOWTO >> ========================= >> >> Here is an example of what can be done: >> >> 1) First we can add processes into containers using the GPM (group of >> processes manager) module. For example we can add an ftp server with >> pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have: >> container 1 -> 123 >> container 2 -> 234 >> >> 2) Now, a user can login via ssh, so, sshd will create new children. Thus, >> inside the GPM you will have something like: >> container 1 -> 123 >> container 2 -> 234 333 334 335 > > > Can/will this support project based accounting ? > > i.e. Users being member of several projects. Each project having > multiple users as members. And the user is allowed to decide which > project-id each container is going to run under / be accounted to ? > > > -jf > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click > _______________________________________________ > Lse-tech mailing list > Lse-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/lse-tech From jlan@sgi.com Fri Sep 3 12:22:24 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 03 Sep 2004 12:22:30 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i83JMOkN005194 for ; Fri, 3 Sep 2004 12:22:24 -0700 Received: from spindle.corp.sgi.com (spindle.corp.sgi.com [198.29.75.13]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i83KSjHJ031455 for ; Fri, 3 Sep 2004 13:28:45 -0700 Received: from mtv-vpn-hw-jlan-2.corp.sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [134.15.18.195]) by spindle.corp.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id i83JL8Kh41477087; Fri, 3 Sep 2004 12:21:13 -0700 (PDT) Received: from sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [127.0.0.1]) by mtv-vpn-hw-jlan-2.corp.sgi.com (8.12.8/8.12.8) with ESMTP id i83JHqVN013837; Fri, 3 Sep 2004 12:18:18 -0700 Message-ID: <4138C360.5080806@sgi.com> Date: Fri, 03 Sep 2004 12:17:52 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: zh-tw, en-us, en, zh-cn, zh-hk MIME-Version: 1.0 To: Guillaume Thouvenin CC: lse-tech@lists.sourceforge.net, Tim Schmielau , Arthur Corliss , Jay Lan , Erik , Limin Gu , John Hesterberg , csa@oss.sgi.com, Dave Wright , Andrew Morton Subject: Re: [Lse-tech] Proposal for Enhanced Accounting HOWTO References: <20040903133048.GA2979@frec.bull.fr> In-Reply-To: <20040903133048.GA2979@frec.bull.fr> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 21 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@sgi.com Precedence: bulk X-list: csa I think Andrew removed pagg for lack of code that uses pagg? I added Andrew to cc on this. Pagg has completed code review cycle. I am not concerned of this. The users of pagg are trying to get accepted :) CSA is one user of pagg and we have other software that needs pagg is working on release under GPL license. Thanks, - jay Guillaume Thouvenin wrote: > Enhanced Accounting HOWTO > ========================= > > According to discussion on the lse-tech mailing-list, it appears that > three steps (at least) are required to improve accounting. > > 1) Improve accounting structure > ---------------------------- > > The current BSD-accounting structure doesn't have enough informations. > Metrics computed by CSA module can be added to BSD accounting. According > to other discussion (like Andi Kleen's comment on the patch I wrote when > I wanted to add CSA IO values in the BSD accounting > ( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics > about blocks/char read/write is not accurate since most writes can be > accounted by some pdflush threads. Maybe add a counter in the routine > mpage_writepages() but I don't know if we can recover a process ID from > the struct page and I don't know if it will be enough... I'm looking if > this is the right way. > > > 2) Group of processes management > ------------------------------ > > We need to be able to manage groups of processes as it's clear that > a major accounting improvement is the per-job accounting. I don't know if > "job" is the right noun. There are several implementation that already exist > and some of them are already in the kernel. The property needed here is that > if a process is in a container, its children will be in the same container. > Different implementations can be: > > - PAGG + JOB (job) > - ELSA (bank) > - CKRM (class) > - CPUSET (a cpuset of all CPUs can act as a container) > > The interface between kernel and user space application can be a new virtual > file system (like CKRM does with /rcfs) but it can also be a device driver with > ioctl operations (like ELSA does). Both solutions are interesting and need to > be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and it seems > that PAGG has been removed (don't know why) from -mm tree. > > 3) Data presentation > ----------------- > > We can have several different implementations. > > 4) General overview > ---------------- > > KERNEL SPACE | USER SPACE (or MODULE) > | > ---------------- | > | BSD accounting | | > | + | | ------------ > | CSA |<=======>| | > ---------------- | | Enhanced | > | | Accounting | > ------------- | | Core | > | group of |<=======>| | > | processes | | ------------ > | manager | | > ------------- | > | > > Communication between EAC (Enhanced Accounting Core) and kernel space can be > done via virtual fs or a device. The goal of EAC is to keep a trace of the > different groups of processes during the accounting period using the group of > processes management module. The idea is that the group of processes management > can send a message to EAC when a process ended to indicate the job > ID if any. The other thing that has to be done by EAC is the data presentation. > > Here is an example of what can be done: > > 1) First we can add processes into containers using the GPM (group of > processes manager) module. For example we can add an ftp server with > pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have: > container 1 -> 123 > container 2 -> 234 > > 2) Now, a user can login via ssh, so, sshd will create new children. Thus, > inside the GPM you will have something like: > container 1 -> 123 > container 2 -> 234 333 334 335 > > 3) As soon as a process is terminated, the EAC must be aware of that fact > (using a signal from GPM for example). There is just this modification > that is needed. With the signal, the GPM will transfer information like > pid, jib (job ID), command, ... to the EAC. This will allow to keep trace > about what happens in the system. Thus, in the EAC we keep information > about all processes that are finished. > > 4) If the sysadmin asks information about process 234, the EAC will know that > it belongs to container 2 and there are other processes in this container. > As all processes are exited, accounting information are written in the > accounting file (like it is currently with BSD accounting). With those > informations (accounting + job information) we can do per-job accounting. > > I am working on the EAC implementation to check how we can make it work. > > Any comments? > > Best > Guillaume > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click > _______________________________________________ > Lse-tech mailing list > Lse-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/lse-tech From akpm@osdl.org Fri Sep 3 13:42:17 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 03 Sep 2004 13:42:23 -0700 (PDT) Received: from mail.osdl.org (fw.osdl.org [65.172.181.6]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i83KgFYP007372 for ; Fri, 3 Sep 2004 13:42:17 -0700 Received: from bix (build.pdx.osdl.net [172.20.1.2]) by mail.osdl.org (8.11.6/8.11.6) with SMTP id i83Kf7111806; Fri, 3 Sep 2004 13:41:07 -0700 Date: Fri, 3 Sep 2004 13:39:16 -0700 From: Andrew Morton To: Jay Lan Cc: guillaume.thouvenin@bull.net, lse-tech@lists.sourceforge.net, tim@physik3.uni-rostock.de, corliss@digitalmages.com, jlan@engr.sgi.com, erikj@dbear.engr.sgi.com, limin@dbear.engr.sgi.com, jh@sgi.com, csa@oss.sgi.com, daw@sgi.com Subject: Re: [Lse-tech] Proposal for Enhanced Accounting HOWTO Message-Id: <20040903133916.3e6c2d17.akpm@osdl.org> In-Reply-To: <4138C360.5080806@sgi.com> References: <20040903133048.GA2979@frec.bull.fr> <4138C360.5080806@sgi.com> X-Mailer: Sylpheed version 0.9.7 (GTK+ 1.2.10; i386-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 22 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: akpm@osdl.org Precedence: bulk X-list: csa Jay Lan wrote: > > I think Andrew removed pagg for lack of code that uses pagg? Yup. It was just sitting there not doing anything and we were getting no new information from retaining it. From guillaume.thouvenin@bull.net Mon Sep 6 01:10:10 2004 Received: with ECARTIS (v1.0.0; list csa); Mon, 06 Sep 2004 01:10:17 -0700 (PDT) Received: from ecbull20.frec.bull.fr (ecbull20.frec.bull.fr [129.183.4.3]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i868A7Cx007550 for ; Mon, 6 Sep 2004 01:10:09 -0700 Received: from ecn002.frec.bull.fr (ecn002.frec.bull.fr [129.183.4.6]) by ecbull20.frec.bull.fr (8.9.2/8.9.1) with ESMTP id KAA15984; Mon, 6 Sep 2004 10:06:34 +0200 Received: from thouvenig ([129.183.101.50]) by ecn002.frec.bull.fr (Lotus Domino Release 5.0.12) with ESMTP id 2004090610120069:113 ; Mon, 6 Sep 2004 10:12:00 +0200 Received: from guill by thouvenig with local (Exim 4.34) id 1C4EWB-0001V1-Cp; Mon, 06 Sep 2004 10:06:27 +0200 Date: Mon, 6 Sep 2004 10:06:27 +0200 From: Guillaume Thouvenin To: Jay Lan Cc: Jan-Frode Myklebust , Guillaume Thouvenin , lse-tech@lists.sourceforge.net, Tim Schmielau , Arthur Corliss , Erik , Limin Gu , John Hesterberg , csa@oss.sgi.com Subject: Re: [Lse-tech] Re: Proposal for Enhanced Accounting HOWTO Message-ID: <20040906080627.GA5714@frec.bull.fr> References: <20040903133048.GA2979@frec.bull.fr> <20040903141240.GB16082@ii.uib.no> <4138AE71.6090803@sgi.com> Mime-Version: 1.0 In-Reply-To: <4138AE71.6090803@sgi.com> User-Agent: Mutt/1.5.6+20040722i X-MIMETrack: Itemize by SMTP Server on ECN002/FR/BULL(Release 5.0.12 |February 13, 2003) at 06/09/2004 10:12:01, Serialize by Router on ECN002/FR/BULL(Release 5.0.12 |February 13, 2003) at 06/09/2004 10:12:19, Serialize complete at 06/09/2004 10:12:19 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-archive-position: 23 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: guillaume.thouvenin@bull.net Precedence: bulk X-list: csa On Fri, Sep 03, 2004 at 10:48:33AM -0700, Jay Lan wrote: > Support of project concept is certainly one thing we need to consider. > We have received requests to support project in CSA. > > Cheers, > - jay > > > >Can/will this support project based accounting ? > > > >i.e. Users being member of several projects. Each project having > >multiple users as members. And the user is allowed to decide which > >project-id each container is going to run under / be accounted to ? Here is an update of the Enhanced Accounting HOWTO. Here is the ChangeLog: 6/09/2004: Guillaume Thouvenin - Add a section concerning projects based accounting proposed by Jan-Frode Myklebust. - Add a section concerning the EAC (Enhanced Accounting Core) part. And here is the proposal: Enhanced Linux accounting Howto? ================================ According to discussion on the lse-tech mailing-list, here are some stages that are required to improve accounting. 1) Improve accounting structure ---------------------------- The current BSD-accounting structure doesn't have enough informations. Metrics computed by CSA module can be added to BSD accounting. According to other discussion (like Andi Kleen's comment on the patch I wrote when I wanted to add CSA IO values in the BSD accounting ( http://lkml.org/lkml/2004/8/2/70 ) the current method to get metrics about blocks/char read/write is not accurate since most writes can be accounted to pdflsuhd. 2) Group of processes management ------------------------------ We need to be able to manage groups of processes as it's clear that a major accounting improvement is the per-job accounting. I don't know if "job" is the right noun. There are several implementation that already exist and some of them are already in the kernel. The property needed here is that if a process is in a container, its children will be in the same container. Different implementations can be: - PAGG + JOB (job) - ELSA (bank) - CKRM (class) - CPUSET (a cpuset of all CPUs can act as a container) The interface between kernel and user space application can be a new virtual file system (like CKRM does with /rcfs) but it can also be a device driver with ioctl operations (like ELSA does). Both solutions are interesting and need to be discussed. We can notice that CPUSET is in 2.6.9-rc1-mm2 tree and PAGG has been removed from -mm tree because there is a lack of code that uses PAGG. 3) Project based accounting ------------------------ It could be interesting to add a support for project based accounting. It means that a user can be a member of one or more projects and he is allowed to decide which project-id each container is going to run. This could be done inside the Enhanced Accounting Core module. 4) Data presentation ----------------- We can have several different implementations. 5) General overview ---------------- KERNEL SPACE | USER SPACE (or MODULE) | ---------------- | | BSD accounting | | | + | | ------------ | CSA |<=======>| | ---------------- | | Enhanced | | | Accounting | ------------- | | Core | | group of |<=======>| | | processes | | ------------ | manager | | ------------- | | Communication between EAC (Enhanced Accounting Core) and kernel space can be done via virtual fs or a device. The goal of EAC is to keep a trace of the different groups of processes during the accounting period using the group of processes management module. The idea is that the group of processes management can send a message to EAC when a process ended to indicate the job ID if any. The other thing that has to be done by EAC is the data presentation. Here is an example of what can be done: a) First we can add processes into containers using the GPM (group of processes manager) module. For example we can add an ftp server with pid #123 and a daemon ssh with pid #234. Thus, inside the GPM you have: container 1 -> 123 container 2 -> 234 b) Now, a user can login via ssh, so, sshd will create new children. Thus, inside the GPM you will have something like: container 1 -> 123 container 2 -> 234 333 334 335 c) As soon as a process is terminated, the EAC must be aware of that fact (using a signal from GPM for example). There is just this modification that is needed. With the signal, the GPM will transfer information like pid, jib (job ID), command, ... to the EAC. This will allow to keep trace about what happens in the system. Thus, in the EAC we keep information about all processes that are finished. d) If the sysadmin asks information about process 234, the EAC will know that it belongs to container 2 and there are other processes in this container. As all processes are exited, accounting information are written in the accounting file (like it is currently with BSD accounting). With those informations (accounting + job information) we can do per-job accounting. 6) Enhanced Accounting Core (EAC) implementation --------------------------------------------- The EAC can be implemented in the user space or as a kernel module. We will see later differences between both implementation. The EAC has several goals. o It will manage a group of processes historic. It means that it will keep a trace concerning processes that belong to the same container. We don't care about how container are managed, we just need to know when a new process is created and if he is in a container. It means that the job manager (or group of processes manager) must provide a mechanism to share this information. If we decide to implement EAC in user space, information can be pass through a signal or a device. If it's a module, we can add an hook in the code of the job manager. o It will provide routine to add/remove processes from container. It will act as a wrapper for routine that already exist in the job manager. o It will get information from the accounting file. Probably from BSD accounting extended with CSA information. Those informations will be used in conjunction with the group of process historic to provide the enhanced accounting which is a per-job accounting information. o Finally, EAC we will support the project based accounting as describe in section 3) Any comments? I am working on the EAC implementation to check how we can make it work. Guillaume From kaigai@ak.jp.nec.com Tue Sep 7 02:52:20 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 07 Sep 2004 02:52:24 -0700 (PDT) Received: from tyo202.gate.nec.co.jp (TYO202.gate.nec.co.jp [202.32.8.202]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i879qJxr014699 for ; Tue, 7 Sep 2004 02:52:20 -0700 Received: from mailgate4.nec.co.jp (mailgate53.nec.co.jp [10.7.69.184]) by tyo202.gate.nec.co.jp (8.11.7/3.7W01080315) with ESMTP id i879pv227521; Tue, 7 Sep 2004 18:51:57 +0900 (JST) Received: (from root@localhost) by mailgate4.nec.co.jp (8.11.7/3.7W-MAILGATE-NEC) id i879pu107233; Tue, 7 Sep 2004 18:51:56 +0900 (JST) Received: from mailsv.bs1.fc.nec.co.jp (venus.hpc.bs1.fc.nec.co.jp [10.34.77.164]) by mailsv.nec.co.jp (8.11.7/3.7W-MAILSV-NEC) with ESMTP id i879puq22098; Tue, 7 Sep 2004 18:51:56 +0900 (JST) Received: from mailsv.linux.bs1.fc.nec.co.jp (IDENT:postfix@namesv2.linux.bs1.fc.nec.co.jp [10.34.125.2]) by mailsv.bs1.fc.nec.co.jp (8.12.10/3.7W-HPC5.2F(mailsv)04081615) with ESMTP id i879pt6i015246; Tue, 7 Sep 2004 18:51:55 +0900 (JST) Received: from sanma (sanma.linux.bs1.fc.nec.co.jp [10.34.125.249]) by mailsv.linux.bs1.fc.nec.co.jp (Postfix) with SMTP id 7C10F30806; Tue, 7 Sep 2004 18:51:55 +0900 (JST) Message-ID: <06e201c494c0$5c567c60$f97d220a@linux.bs1.fc.nec.co.jp> From: "Kaigai Kohei" To: "LSE-Tech" , "CSA-ML" Subject: [HELP] Building CSA tools failed on ia-64 environment. Date: Tue, 7 Sep 2004 18:52:17 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 X-archive-position: 24 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: kaigai@ak.jp.nec.com Precedence: bulk X-list: csa Hello, everyone. I'm interrested in accounting solution for Linux as PAGG/CSA or ELSA, and so on. First, I tried to build up the PAGG/CSA environment, and downloaded the related patches and comprehensive tools for CSA/PAGG. Then, I succeeded to build up the 2.6.8.1 + PAGG/CSA kernel for IA-64 archtecture and PAGG related tools like jstat,jkill and so on. But I failed to build up the CSA related tools. (1) I tried to build up this one from SRPM. # rpmbuild --rebuild csa-2.1.0-1.src.rpm Installing csa-2.1.0-1.src.rpm Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.7433 : + automake -a -c -i error: Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) # (2) I tried to build up this one from tarball. # mkdir -p csa # cd csa # rpm2cpio ../csa-2.1.0-1.src.rpm | cpio -i --make-directories 762 blocks # ls csa-2.1.0.tar.gz csa.spec # tar zxf csa-2.1.0.tar.gz # cd csa-2.1.0 # libtoolize --force --copy # aclocal # automake -a -c -i # autoconf # autoheader # ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/share/man # make make all-recursive make[1]: Entering directory `/root/rpm/csa/csa-2.1.0' Making all in include make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/include' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/include' Making all in lib make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/lib' gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I/lib/modules/2.6.8.1.pagg.csa/bu ild/include -D__USE_GNU -Dlinux -g -O2 -c config.c In file included from config.c:51: /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:4:2: #error "GCC no l onger implements ." /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:5:2: #error "Revise y our code to use ." make[2]: *** [config.o] Error 1 make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/lib' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rpm/csa/csa-2.1.0' make: *** [all] Error 2 # What environment do these tools depend on ? Any comment please. Thanks. I got those packages from as follows. The kernel patch: ftp://oss.sgi.com/projects/pagg/download/ ftp://oss.sgi.com/projects/csa/download/ The tools for PAGG: ftp://oss.sgi.com/projects/pagg/download/job-1.4.0-1.tar.gz The tools for CSA: ftp://oss.sgi.com/projects/csa/download/csa-2.1.0-1.src.rpm -------- Kai Gai From jlan@engr.sgi.com Tue Sep 7 12:20:43 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 07 Sep 2004 12:20:48 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i87JKgJa013033 for ; Tue, 7 Sep 2004 12:20:42 -0700 Received: from nodin.corp.sgi.com (nodin.corp.sgi.com [192.26.51.193]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i87JKX0f030766 for ; Tue, 7 Sep 2004 14:20:33 -0500 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by nodin.corp.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i87JKWl226835706 for ; Tue, 7 Sep 2004 12:20:32 -0700 (PDT) Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i87JJRY98775045; Tue, 7 Sep 2004 12:19:28 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i87JHAxV031637; Tue, 7 Sep 2004 12:17:11 -0700 Message-ID: <413E0936.8080804@engr.sgi.com> Date: Tue, 07 Sep 2004 12:17:10 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kaigai Kohei CC: LSE-Tech , CSA-ML Subject: Re: [HELP] Building CSA tools failed on ia-64 environment. References: <06e201c494c0$5c567c60$f97d220a@linux.bs1.fc.nec.co.jp> In-Reply-To: <06e201c494c0$5c567c60$f97d220a@linux.bs1.fc.nec.co.jp> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 25 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa The gcc version i used in building and creating CSA rpm was 3.2.3. Has anyone else experience the similar problem with 3.3.3? I will find a 3.3.3 machine and try it out later. Thanks, - jay Kaigai Kohei wrote: > Hello, everyone. > I'm interrested in accounting solution for Linux as PAGG/CSA or ELSA, > and so on. First, I tried to build up the PAGG/CSA environment, > and downloaded the related patches and comprehensive tools for CSA/PAGG. > > Then, I succeeded to build up the 2.6.8.1 + PAGG/CSA kernel for IA-64 > archtecture and PAGG related tools like jstat,jkill and so on. > But I failed to build up the CSA related tools. > > > (1) I tried to build up this one from SRPM. > # rpmbuild --rebuild csa-2.1.0-1.src.rpm > Installing csa-2.1.0-1.src.rpm > Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.7433 > : > + automake -a -c -i > error: Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) > > RPM build errors: > Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) > # > > > (2) I tried to build up this one from tarball. > # mkdir -p csa > # cd csa > # rpm2cpio ../csa-2.1.0-1.src.rpm | cpio -i --make-directories > 762 blocks > # ls > csa-2.1.0.tar.gz csa.spec > # tar zxf csa-2.1.0.tar.gz > # cd csa-2.1.0 > # libtoolize --force --copy > # aclocal > # automake -a -c -i > # autoconf > # autoheader > # ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/share/man > # make > make all-recursive > make[1]: Entering directory `/root/rpm/csa/csa-2.1.0' > Making all in include > make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/include' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/include' > Making all in lib > make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/lib' > gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I/lib/modules/2.6.8.1.pagg.csa/bu > ild/include -D__USE_GNU -Dlinux -g -O2 -c config.c > In file included from config.c:51: > /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:4:2: #error "GCC no l > onger implements ." > /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:5:2: #error "Revise y > our code to use ." > make[2]: *** [config.o] Error 1 > make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/lib' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/rpm/csa/csa-2.1.0' > make: *** [all] Error 2 > # > > What environment do these tools depend on ? > Any comment please. Thanks. > > I got those packages from as follows. > The kernel patch: ftp://oss.sgi.com/projects/pagg/download/ > ftp://oss.sgi.com/projects/csa/download/ > The tools for PAGG: ftp://oss.sgi.com/projects/pagg/download/job-1.4.0-1.tar.gz > The tools for CSA: ftp://oss.sgi.com/projects/csa/download/csa-2.1.0-1.src.rpm > > -------- > Kai Gai > From jlan@engr.sgi.com Wed Sep 8 11:35:08 2004 Received: with ECARTIS (v1.0.0; list csa); Wed, 08 Sep 2004 11:35:18 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i88IZ7xt003387 for ; Wed, 8 Sep 2004 11:35:08 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i88JgCln026820 for ; Wed, 8 Sep 2004 12:42:12 -0700 Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i88IYkY99037115; Wed, 8 Sep 2004 11:34:47 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i88IWIxV002282; Wed, 8 Sep 2004 11:32:19 -0700 Message-ID: <413F5032.1070008@engr.sgi.com> Date: Wed, 08 Sep 2004 11:32:18 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jay Lan CC: Tim Schmielau , Guillaume Thouvenin , Arthur Corliss , Andrew Morton , lkml , erikj@dbear.engr.sgi.com, limin@engr.sgi.com, lse-tech@lists.sourceforge.net, ? , Yoshitaka ISHIKAWA , csa@oss.sgi.com Subject: Re: [Lse-tech] Re: [PATCH] new CSA patchset for 2.6.8 References: <412D2E10.8010406@engr.sgi.com> <20040825221842.72dd83a4.akpm@osdl.org> <412E4C27.1010805@engr.sgi.com> <20040830122614.GA2518@frec.bull.fr> <4135032E.7060605@engr.sgi.com> In-Reply-To: <4135032E.7060605@engr.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 26 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa The block IO data was not used in billing for CSA customers. Nobody has ever charged for this data. It is more for accountability and resource consumption tracking. Accuracy changes as the program runs for hours or days in our market. Repeatable billing is critical and block IO is one that is not repeatable. The write blocks was useful and with the type of programs running for hours and days (which are the ones of interest in block IO data). The delayed write after the process terminates is ignored because that is insignificant for a process doing io for hours or days. We use the bytes transferred vs blocks transferred to see programs dominating and polluting the cache. Processes big in one and not the other are of interest to profile more closely. However, if nobody else wants to have this feature, we can pull it out until we can find a way of doing this that makes people happy. I will submit a new set of patch based on 2.6.8.1 later. Thanks! - jay Jay Lan wrote: > Adding csa@oss.sgi.com, the CSA user group mailing list, to Cc. > > Tim Schmielau wrote: > >> On Mon, 30 Aug 2004, Guillaume Thouvenin wrote: >> >> >>> Thus, to be clear, the enhanced accounting can be divided into >>> three parts: >>> >>> 1) A common data collection method in the kernel. >>> We could start from BSD-accounting and add CSA information. Could >>> it be something like BSD version4? >> >> >> >> I've had a quick look at the CSA data collection patches. To get the >> discussion started, here are my comments: >> >> >>> --- linux.orig/drivers/block/ll_rw_blk.c 2004-08-13 >>> 22:36:16.000000000 -0700 >>> +++ linux/drivers/block/ll_rw_blk.c 2004-08-18 12:07:10.000000000 >>> -0700 >>> @@ -1948,10 +1950,12 @@ >>> >>> if (rw == READ) { >>> disk_stat_add(rq->rq_disk, read_sectors, nr_sectors); >>> + current->rblk += nr_sectors; >>> if (!new_io) >>> disk_stat_inc(rq->rq_disk, read_merges); >>> } else if (rw == WRITE) { >>> disk_stat_add(rq->rq_disk, write_sectors, nr_sectors); >>> + current->wblk += nr_sectors; >>> if (!new_io) >>> disk_stat_inc(rq->rq_disk, write_merges); >>> } >> >> >> >> Andi Kleen's comment on the ELSA patch also applies here - most writes >> will get accounted to pdflushd. See >> >> http://www.lib.uaa.alaska.edu/linux-kernel/archive/2004-Week-31/0047.html >> >> for his comment. > > > I need more time on this. :) > >> >> >>> --- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>> +++ linux/include/linux/csa_internal.h 2004-08-19 15:19:05.000000000 >>> -0700 >> >> >> [...] >> >>> +#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ >>> + >>> +#define csa_update_integrals() do { } while (0); >>> +#define csa_clear_integrals(task) do { } while (0); >>> +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ >> >> >> >> I suppose the semicolons are unintentional. > > > Good catch! I fixed this in our internal tree. > >> >> >>> --- linux.orig/include/linux/sched.h 2004-08-19 15:17:52.000000000 >>> -0700 >>> +++ linux/include/linux/sched.h 2004-08-19 15:19:05.000000000 -0700 >> >> >> [...] >> >>> @@ -525,6 +527,10 @@ >>> >>> /* i/o counters(bytes read/written, blocks read/written, #syscalls, >>> waittime */ >>> unsigned long rchar, wchar, rblk, wblk, syscr, syscw, bwtime; >>> +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) >>> + unsigned long csa_rss_mem1, csa_vm_mem1; >>> + clock_t csa_stimexpd; >>> +#endif >> >> >> >> These probably need to be u64, otherwise they might easily overflow >> within >> a view seconds on 32 bit platforms. > > > Will fix it. > >> >> >>> --- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>> +++ linux/include/linux/acct_eop.h 2004-08-19 18:48:44.000000000 >>> -0700 >> >> >> >> This should probably be unified with BSD accounting to a general >> accounting >> hook. > > > Do you suggest to merge acct_eop.h into acct.h? It sounds good to me! > > Thanks! > - jay > >> >> >> Tim >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by BEA Weblogic Workshop >> FREE Java Enterprise J2EE developer tools! >> Get your free copy of BEA WebLogic Workshop 8.1 today. >> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click >> _______________________________________________ >> Lse-tech mailing list >> Lse-tech@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/lse-tech > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click > _______________________________________________ > Lse-tech mailing list > Lse-tech@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/lse-tech From jlan@engr.sgi.com Thu Sep 9 12:09:51 2004 Received: with ECARTIS (v1.0.0; list csa); Thu, 09 Sep 2004 12:10:07 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i89J9nJu012070 for ; Thu, 9 Sep 2004 12:09:49 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i89J9d0f030397 for ; Thu, 9 Sep 2004 14:09:40 -0500 Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i89J9YY99284369; Thu, 9 Sep 2004 12:09:34 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i89J6wxV004773; Thu, 9 Sep 2004 12:06:59 -0700 Message-ID: <4140A9D2.3010602@engr.sgi.com> Date: Thu, 09 Sep 2004 12:06:58 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: Andrew Morton , lse-tech , Guillaume Thouvenin , CSA-ML , Arthur Corliss , Tim Schmielau , Erik Jacobson , Limin Gu , John Hesterberg Subject: [PATCH 2.6.8.1 1/4] CSA csa_io: accounting IO data gathering Content-Type: multipart/mixed; boundary="------------030802040406070309060909" X-archive-position: 27 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is a multi-part message in MIME format. --------------030802040406070309060909 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Linux Comprehensive System Accounting (CSA) is a set of C programs and shell scripts that, like other accounting packages, provide methods for collecting per-process resource usage data, monitoring disk usage, and charging fees to specific login accounts. The CSA patchset includes csa_io, csa_mm, csa_eop and csa_module. Patches csa_io, csa_mm, and csa_eop are responsible for system accounting data collection and are independent of each other. The csa_io is a patch that gathers I/O activity data. Please find CSA project at http://oss.sgi.com/projects/csa. This set of csa patches has been tested with the pagg and job kernel patches. The information of pagg and job project can be found at http://oss.sgi.com/projects/pagg/ Signed-off-by: Jay Lan --------------030802040406070309060909 Content-Type: text/plain; name="linux-2.6.8.1.csa_io.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="linux-2.6.8.1.csa_io.patch" Index: linux/drivers/block/ll_rw_blk.c =================================================================== --- linux.orig/drivers/block/ll_rw_blk.c 2004-08-13 22:36:16.000000000 -0700 +++ linux/drivers/block/ll_rw_blk.c 2004-09-03 17:15:06.000000000 -0700 @@ -1674,6 +1674,7 @@ { DEFINE_WAIT(wait); struct request *rq; + unsigned long start_wait = jiffies; generic_unplug_device(q); do { @@ -1702,6 +1703,7 @@ finish_wait(&rl->wait[rw], &wait); } while (!rq); + current->bwtime += (unsigned long) jiffies - start_wait; return rq; } Index: linux/fs/read_write.c =================================================================== --- linux.orig/fs/read_write.c 2004-08-13 22:37:15.000000000 -0700 +++ linux/fs/read_write.c 2004-09-03 12:59:18.000000000 -0700 @@ -216,8 +216,11 @@ ret = file->f_op->read(file, buf, count, pos); else ret = do_sync_read(file, buf, count, pos); - if (ret > 0) + if (ret > 0) { dnotify_parent(file->f_dentry, DN_ACCESS); + current->rchar += ret; + } + current->syscr++; } } @@ -260,8 +263,11 @@ ret = file->f_op->write(file, buf, count, pos); else ret = do_sync_write(file, buf, count, pos); - if (ret > 0) + if (ret > 0) { dnotify_parent(file->f_dentry, DN_MODIFY); + current->wchar += ret; + } + current->syscw++; } } @@ -540,6 +546,10 @@ fput_light(file, fput_needed); } + if (ret > 0) { + current->rchar += ret; + } + current->syscr++; return ret; } @@ -558,6 +568,10 @@ fput_light(file, fput_needed); } + if (ret > 0) { + current->wchar += ret; + } + current->syscw++; return ret; } @@ -636,6 +650,13 @@ retval = in_file->f_op->sendfile(in_file, ppos, count, file_send_actor, out_file); + if (retval > 0) { + current->rchar += retval; + current->wchar += retval; + } + current->syscr++; + current->syscw++; + if (*ppos > max) retval = -EOVERFLOW; Index: linux/kernel/fork.c =================================================================== --- linux.orig/kernel/fork.c 2004-09-03 12:59:18.000000000 -0700 +++ linux/kernel/fork.c 2004-09-03 16:21:20.000000000 -0700 @@ -964,6 +964,8 @@ p->utime = p->stime = 0; p->cutime = p->cstime = 0; + p->rchar = p->wchar = p->syscr = p->syscw = 0; + p->bwtime = 0; p->lock_depth = -1; /* -1 = no lock */ p->start_time = get_jiffies_64(); p->security = NULL; Index: linux/include/linux/sched.h =================================================================== --- linux.orig/include/linux/sched.h 2004-09-03 12:59:18.000000000 -0700 +++ linux/include/linux/sched.h 2004-09-03 16:18:31.000000000 -0700 @@ -523,6 +523,9 @@ unsigned long ptrace_message; siginfo_t *last_siginfo; /* For ptrace use. */ +/* i/o counters(bytes read/written, #syscalls, waittime */ + unsigned long rchar, wchar, syscr, syscw, bwtime; + #ifdef CONFIG_NUMA struct mempolicy *mempolicy; short il_next; /* could be shared with used_math */ --------------030802040406070309060909-- From jlan@engr.sgi.com Thu Sep 9 12:14:27 2004 Received: with ECARTIS (v1.0.0; list csa); Thu, 09 Sep 2004 12:14:44 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i89JEQ3l012153 for ; Thu, 9 Sep 2004 12:14:27 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i89JEG0f001225 for ; Thu, 9 Sep 2004 14:14:17 -0500 Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i89JECY99293725; Thu, 9 Sep 2004 12:14:13 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i89JBdxV004788; Thu, 9 Sep 2004 12:11:40 -0700 Message-ID: <4140AAEB.1010209@engr.sgi.com> Date: Thu, 09 Sep 2004 12:11:39 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: Andrew Morton , lse-tech , Guillaume Thouvenin , CSA-ML , Arthur Corliss , Tim Schmielau , Erik Jacobson , Limin Gu Subject: [PATCH 2.6.8.1 2/4] CSA csa_mm: accounting mm data gathering References: <4140A9D2.3010602@engr.sgi.com> In-Reply-To: <4140A9D2.3010602@engr.sgi.com> Content-Type: multipart/mixed; boundary="------------080108070403060202040704" X-archive-position: 28 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is a multi-part message in MIME format. --------------080108070403060202040704 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Linux Comprehensive System Accounting (CSA) is a set of C programs and shell scripts that, like other accounting packages, provide methods for collecting per-process resource usage data, monitoring disk usage, and charging fees to specific login accounts. The CSA patchset includes csa_io, csa_mm, csa_eop and csa_module. Patches csa_io, csa_mm, and csa_eop are responsible for system accounting data collection and are independent of each other. The csa_mm is a patch that gathers mm activity data. Please find CSA project at http://oss.sgi.com/projects/csa. This set of csa patches has been tested with the pagg and job kernel patches. The information of pagg and job project can be found at http://oss.sgi.com/projects/pagg/ Signed-off-by: Jay Lan --------------080108070403060202040704 Content-Type: text/plain; name="linux-2.6.8.1.csa_mm.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="linux-2.6.8.1.csa_mm.patch" Index: linux/fs/exec.c =================================================================== --- linux.orig/fs/exec.c 2004-09-03 12:59:18.000000000 -0700 +++ linux/fs/exec.c 2004-09-03 16:22:11.000000000 -0700 @@ -47,6 +47,7 @@ #include #include #include +#include #include #include @@ -1145,6 +1146,9 @@ /* execve success */ security_bprm_free(&bprm); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return retval; } Index: linux/include/linux/csa_internal.h =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linux/include/linux/csa_internal.h 2004-09-03 16:22:11.000000000 -0700 @@ -0,0 +1,70 @@ +/* + * Copyright (c) 2000-2002 Silicon Graphics, Inc and LANL All Rights Reserved. + * Copyright (c) 2004 Silicon Graphics, Inc All Rights Reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it would be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA. + * + * Contact information: Silicon Graphics, Inc., 1500 Crittenden Lane, + * Mountain View, CA 94043, or: + * + * http://www.sgi.com + */ + +/* + * CSA (Comprehensive System Accounting) + * Job Accounting for Linux + * + * This header file contains the definitions needed for communication + * between the kernel and the CSA module. + */ + +#ifndef _LINUX_CSA_INTERNAL_H +#define _LINUX_CSA_INTERNAL_H + +#include + +#if defined (CONFIG_CSA) || defined (CONFIG_CSA_MODULE) + +#include +#include + +static inline void csa_update_integrals(void) +{ + long delta; + + if (current->mm) { + delta = current->stime - current->csa_stimexpd; + current->csa_stimexpd = current->stime; + current->csa_rss_mem1 += delta * current->mm->rss; + current->csa_vm_mem1 += delta * current->mm->total_vm; + } +} + +static inline void csa_clear_integrals(struct task_struct *tsk) +{ + if (tsk) { + tsk->csa_stimexpd = 0; + tsk->csa_rss_mem1 = 0; + tsk->csa_vm_mem1 = 0; + } +} + +#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ + +#define csa_update_integrals() do { } while (0) +#define csa_clear_integrals(task) do { } while (0) +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ + +#endif /* _LINUX_CSA_INTERNAL_H */ Index: linux/include/linux/sched.h =================================================================== --- linux.orig/include/linux/sched.h 2004-09-03 16:18:31.000000000 -0700 +++ linux/include/linux/sched.h 2004-09-03 16:27:19.000000000 -0700 @@ -229,6 +229,8 @@ struct kioctx *ioctx_list; struct kioctx default_kioctx; + + unsigned long hiwater_rss, hiwater_vm; }; extern int mmlist_nr; @@ -525,6 +527,10 @@ /* i/o counters(bytes read/written, #syscalls, waittime */ unsigned long rchar, wchar, syscr, syscw, bwtime; +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) + u64 csa_rss_mem1, csa_vm_mem1; + clock_t csa_stimexpd; +#endif #ifdef CONFIG_NUMA struct mempolicy *mempolicy; @@ -873,6 +879,19 @@ /* Remove the current tasks stale references to the old mm_struct */ extern void mm_release(struct task_struct *, struct mm_struct *); +/* Update highwater values */ +static inline void update_mem_hiwater(void) +{ + if (current->mm) { + if (current->mm->hiwater_rss < current->mm->rss) { + current->mm->hiwater_rss = current->mm->rss; + } + if (current->mm->hiwater_vm < current->mm->total_vm) { + current->mm->hiwater_vm = current->mm->total_vm; + } + } +} + extern int copy_thread(int, unsigned long, unsigned long, unsigned long, struct task_struct *, struct pt_regs *); extern void flush_thread(void); extern void exit_thread(void); Index: linux/kernel/exit.c =================================================================== --- linux.orig/kernel/exit.c 2004-09-03 12:59:18.000000000 -0700 +++ linux/kernel/exit.c 2004-09-03 16:22:11.000000000 -0700 @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -821,6 +822,9 @@ ptrace_notify((PTRACE_EVENT_EXIT << 8) | SIGTRAP); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); acct_process(code); __exit_mm(tsk); Index: linux/kernel/fork.c =================================================================== --- linux.orig/kernel/fork.c 2004-09-03 16:21:20.000000000 -0700 +++ linux/kernel/fork.c 2004-09-03 16:22:11.000000000 -0700 @@ -37,7 +37,7 @@ #include #include #include - +#include #include #include #include @@ -576,6 +576,9 @@ if (retval) goto free_pt; + mm->hiwater_rss = mm->rss; + mm->hiwater_vm = mm->total_vm; + good_mm: tsk->mm = mm; tsk->active_mm = mm; @@ -966,6 +969,8 @@ p->cutime = p->cstime = 0; p->rchar = p->wchar = p->syscr = p->syscw = 0; p->bwtime = 0; + /* no-op if CONFIG_CSA not set */ + csa_clear_integrals(p); p->lock_depth = -1; /* -1 = no lock */ p->start_time = get_jiffies_64(); p->security = NULL; Index: linux/mm/memory.c =================================================================== --- linux.orig/mm/memory.c 2004-08-13 22:36:57.000000000 -0700 +++ linux/mm/memory.c 2004-09-03 16:22:11.000000000 -0700 @@ -44,6 +44,7 @@ #include #include #include +#include #include #include @@ -605,6 +606,8 @@ tlb = tlb_gather_mmu(mm, 0); unmap_vmas(&tlb, mm, vma, address, end, &nr_accounted, details); tlb_finish_mmu(tlb, address, end); + /* no-op unless CONFIG_CSA is set */ + csa_update_integrals(); spin_unlock(&mm->page_table_lock); } @@ -1095,9 +1098,12 @@ spin_lock(&mm->page_table_lock); page_table = pte_offset_map(pmd, address); if (likely(pte_same(*page_table, pte))) { - if (PageReserved(old_page)) + if (PageReserved(old_page)) { ++mm->rss; - else + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + } else page_remove_rmap(old_page); break_cow(vma, new_page, address, page_table); lru_cache_add_active(new_page); @@ -1378,6 +1384,10 @@ remove_exclusive_swap_page(page); mm->rss++; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + pte = mk_pte(page, vma->vm_page_prot); if (write_access && can_share_swap_page(page)) { pte = maybe_mkwrite(pte_mkdirty(pte), vma); @@ -1443,6 +1453,9 @@ goto out; } mm->rss++; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); entry = maybe_mkwrite(pte_mkdirty(mk_pte(page, vma->vm_page_prot)), vma); @@ -1552,6 +1565,10 @@ if (pte_none(*page_table)) { if (!PageReserved(new_page)) ++mm->rss; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + flush_icache_page(vma, new_page); entry = mk_pte(new_page, vma->vm_page_prot); if (write_access) Index: linux/mm/mmap.c =================================================================== --- linux.orig/mm/mmap.c 2004-08-13 22:37:15.000000000 -0700 +++ linux/mm/mmap.c 2004-09-03 16:22:11.000000000 -0700 @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -988,6 +989,9 @@ pgoff, flags & MAP_NONBLOCK); down_write(&mm->mmap_sem); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return addr; unmap_and_free_vma: @@ -1227,6 +1231,9 @@ vma->vm_mm->total_vm += grow; if (vma->vm_flags & VM_LOCKED) vma->vm_mm->locked_vm += grow; + /* no-op if CONFIG_CSA_JOB_ACCT not set */ + csa_update_integrals(); + update_mem_hiwater(); anon_vma_unlock(vma); return 0; } @@ -1688,6 +1695,9 @@ mm->locked_vm += len >> PAGE_SHIFT; make_pages_present(addr, addr + len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return addr; } Index: linux/mm/mremap.c =================================================================== --- linux.orig/mm/mremap.c 2004-08-13 22:36:59.000000000 -0700 +++ linux/mm/mremap.c 2004-09-03 16:22:11.000000000 -0700 @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -231,6 +232,10 @@ new_addr + new_len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + return new_addr; } @@ -363,6 +368,9 @@ make_pages_present(addr + old_len, addr + new_len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); ret = addr; goto out; } Index: linux/mm/rmap.c =================================================================== --- linux.orig/mm/rmap.c 2004-08-13 22:37:42.000000000 -0700 +++ linux/mm/rmap.c 2004-09-03 16:22:11.000000000 -0700 @@ -29,6 +29,7 @@ #include #include #include +#include #include #include @@ -515,6 +516,8 @@ mm->rss--; BUG_ON(!page->mapcount); page->mapcount--; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); page_cache_release(page); out_unmap: @@ -614,6 +617,8 @@ page_remove_rmap(page); page_cache_release(page); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); mm->rss--; (*mapcount)--; } Index: linux/mm/swapfile.c =================================================================== --- linux.orig/mm/swapfile.c 2004-08-13 22:36:32.000000000 -0700 +++ linux/mm/swapfile.c 2004-09-03 16:22:11.000000000 -0700 @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -435,6 +436,9 @@ set_pte(dir, pte_mkold(mk_pte(page, vma->vm_page_prot))); page_add_anon_rmap(page, vma, address); swap_free(entry); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); } /* vma->vm_mm->page_table_lock is held */ --------------080108070403060202040704-- From jlan@engr.sgi.com Thu Sep 9 12:19:23 2004 Received: with ECARTIS (v1.0.0; list csa); Thu, 09 Sep 2004 12:19:39 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i89JJMs2015392 for ; Thu, 9 Sep 2004 12:19:23 -0700 Received: from nodin.corp.sgi.com (nodin.corp.sgi.com [192.26.51.193]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i89JJD0f004273 for ; Thu, 9 Sep 2004 14:19:13 -0500 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by nodin.corp.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i89JJCl228208356 for ; Thu, 9 Sep 2004 12:19:12 -0700 (PDT) Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i89JI4Y99300495; Thu, 9 Sep 2004 12:18:04 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i89JFVxV004799; Thu, 9 Sep 2004 12:15:32 -0700 Message-ID: <4140ABD2.90908@engr.sgi.com> Date: Thu, 09 Sep 2004 12:15:30 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: Andrew Morton , lse-tech , Guillaume Thouvenin , CSA-ML , Arthur Corliss , Tim Schmielau , Erik Jacobson , Limin Gu Subject: [PATCH 2.6.8.1 3/4] CSA csa_eop: accounting end-of-process hook References: <4140A9D2.3010602@engr.sgi.com> In-Reply-To: <4140A9D2.3010602@engr.sgi.com> Content-Type: multipart/mixed; boundary="------------060502060308080802040301" X-archive-position: 29 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is a multi-part message in MIME format. --------------060502060308080802040301 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Linux Comprehensive System Accounting (CSA) is a set of C programs and shell scripts that, like other accounting packages, provide methods for collecting per-process resource usage data, monitoring disk usage, and charging fees to specific login accounts. The CSA patchset includes csa_io, csa_mm, csa_eop and csa_module. Patches csa_io, csa_mm, and csa_eop are responsible for system accounting data collection and are independent of each other. csa_eop is a patch that provides a hook for end-of-process handling. Please find CSA project at http://oss.sgi.com/projects/csa. This set of csa patches has been tested with the pagg and job kernel patches. The information of pagg and job project can be found at http://oss.sgi.com/projects/pagg/ Signed-off-by: Jay Lan --------------060502060308080802040301 Content-Type: text/plain; name="linux-2.6.8.1.csa_eop.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="linux-2.6.8.1.csa_eop.patch" Index: linux/kernel/exit.c =================================================================== --- linux.orig/kernel/exit.c 2004-09-03 17:17:03.000000000 -0700 +++ linux/kernel/exit.c 2004-09-03 17:17:03.000000000 -0700 @@ -33,6 +33,8 @@ extern void sem_exit (void); extern struct task_struct *child_reaper; +void (*do_eop_acct) (int, struct task_struct *) = NULL; +EXPORT_SYMBOL(do_eop_acct); int getrusage(struct task_struct *, int, struct rusage __user *); @@ -826,6 +828,9 @@ csa_update_integrals(); update_mem_hiwater(); acct_process(code); + /* Handle end-of-process accounting */ + if (do_eop_acct != NULL) + do_eop_acct(code, tsk); __exit_mm(tsk); exit_sem(tsk); Index: linux/include/linux/acct.h =================================================================== --- linux.orig/include/linux/acct.h 2004-08-13 22:36:32.000000000 -0700 +++ linux/include/linux/acct.h 2004-09-03 17:17:03.000000000 -0700 @@ -185,6 +185,13 @@ return x; } +/* + * extern declaration that provides the hook needed for processing of + * end-of-process accounting record + * + */ +extern void (*do_eop_acct) (int, struct task_struct *); + #endif /* __KERNEL */ #endif /* _LINUX_ACCT_H */ --------------060502060308080802040301-- From jlan@sgi.com Fri Sep 17 15:37:46 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 17 Sep 2004 15:37:54 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8HMbkxS020527 for ; Fri, 17 Sep 2004 15:37:46 -0700 Received: from spindle.corp.sgi.com (spindle.corp.sgi.com [198.29.75.13]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8HMbZ0f010574 for ; Fri, 17 Sep 2004 17:37:35 -0500 Received: from mtv-vpn-hw-jlan-2.corp.sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [134.15.18.195]) by spindle.corp.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id i8HMY4Kh45294463; Fri, 17 Sep 2004 15:34:09 -0700 (PDT) Received: from sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [127.0.0.1]) by mtv-vpn-hw-jlan-2.corp.sgi.com (8.12.8/8.12.8) with ESMTP id i8HMUSa5014401; Fri, 17 Sep 2004 15:30:48 -0700 Message-ID: <414B6583.5080809@sgi.com> Date: Fri, 17 Sep 2004 15:30:27 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: zh-tw, en-us, en, zh-cn, zh-hk MIME-Version: 1.0 To: Andrew Morton , Tim Schmielau CC: LKML , lse-tech , Guillaume Thouvenin , CSA-ML , Arthur Corliss , Erik Jacobson , Limin Gu , John Hesterberg Subject: Re: [PATCH 2.6.8.1 3/4] CSA csa_eop: accounting end-of-process hook References: <4140A9D2.3010602@engr.sgi.com> <4140ABD2.90908@engr.sgi.com> In-Reply-To: <4140ABD2.90908@engr.sgi.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 30 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@sgi.com Precedence: bulk X-list: csa Hi Andrew and Tim, I found i can eliminate this csa_eop patch. Two ways to do this, depending on whether we want to view linux/kernel/acct.c as a non-essential part of kernel and another accounting package that CSA will replace. If we decide we do not need to have acct.c in the kernel, it can be controlled by a new config flag, say CONFIG_BSD_ACCT, and do this: do_eop_acct = acct_process; at BSD acct initialization. If we decide to always have BSD acct on, i can propose a new patch to move do_eop_acct declaration into acct.c and at the beginning of acct_process() routine, which is invoked at do_exit(), i will check do_eop_acctL: if do_eop_acct is not NULL, do_eop_acct() would be called and exits acct_process() upon return (ie, a competing accounting package is running); otherwise, it continues do BSD acct processing. What do you think? Which path i should walk down? Thanks, - jay Jay Lan wrote: > Linux Comprehensive System Accounting (CSA) is a set of C programs and > shell scripts that, like other accounting packages, provide methods for > collecting per-process resource usage data, monitoring disk usage, and > charging fees to specific login accounts. > > The CSA patchset includes csa_io, csa_mm, csa_eop and csa_module. > Patches csa_io, csa_mm, and csa_eop are responsible for system > accounting data collection and are independent of each other. > > csa_eop is a patch that provides a hook for end-of-process handling. > > Please find CSA project at http://oss.sgi.com/projects/csa. This set of > csa patches has been tested with the pagg and job kernel patches. > The information of pagg and job project can be found at > http://oss.sgi.com/projects/pagg/ > > > Signed-off-by: Jay Lan > > > ------------------------------------------------------------------------ > > Index: linux/kernel/exit.c > =================================================================== > --- linux.orig/kernel/exit.c 2004-09-03 17:17:03.000000000 -0700 > +++ linux/kernel/exit.c 2004-09-03 17:17:03.000000000 -0700 > @@ -33,6 +33,8 @@ > > extern void sem_exit (void); > extern struct task_struct *child_reaper; > +void (*do_eop_acct) (int, struct task_struct *) = NULL; > +EXPORT_SYMBOL(do_eop_acct); > > int getrusage(struct task_struct *, int, struct rusage __user *); > > @@ -826,6 +828,9 @@ > csa_update_integrals(); > update_mem_hiwater(); > acct_process(code); > + /* Handle end-of-process accounting */ > + if (do_eop_acct != NULL) > + do_eop_acct(code, tsk); > __exit_mm(tsk); > > exit_sem(tsk); > Index: linux/include/linux/acct.h > =================================================================== > --- linux.orig/include/linux/acct.h 2004-08-13 22:36:32.000000000 -0700 > +++ linux/include/linux/acct.h 2004-09-03 17:17:03.000000000 -0700 > @@ -185,6 +185,13 @@ > return x; > } > > +/* > + * extern declaration that provides the hook needed for processing of > + * end-of-process accounting record > + * > + */ > +extern void (*do_eop_acct) (int, struct task_struct *); > + > #endif /* __KERNEL */ > > #endif /* _LINUX_ACCT_H */ From jlan@sgi.com Fri Sep 24 18:25:08 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 24 Sep 2004 18:25:13 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8P1P5Ee014072 for ; Fri, 24 Sep 2004 18:25:08 -0700 Received: from spindle.corp.sgi.com (spindle.corp.sgi.com [198.29.75.13]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8P2YYVL017090 for ; Fri, 24 Sep 2004 19:34:34 -0700 Received: from mtv-vpn-hw-jlan-2.corp.sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [134.15.18.195]) by spindle.corp.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id i8P1OmKh45721234; Fri, 24 Sep 2004 18:24:53 -0700 (PDT) Received: from sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [127.0.0.1]) by mtv-vpn-hw-jlan-2.corp.sgi.com (8.12.8/8.12.8) with ESMTP id i8P1MX1f014970; Fri, 24 Sep 2004 18:22:54 -0700 Message-ID: <4154C859.4060406@sgi.com> Date: Fri, 24 Sep 2004 18:22:33 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: zh-tw, en-us, en, zh-cn, zh-hk MIME-Version: 1.0 To: csa@oss.sgi.com, lse-tech Subject: csa-2.2.0-1 rpm released to oss.sgi.com Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 31 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@sgi.com Precedence: bulk X-list: csa Hi, csa-2.2.0-1.src.rpm was released to oss.sgi.com. It can be downloaded at ftp://oss.sgi.com/projects/csa/download The csa-2.2.0-1 contains bug fixes since csa-2.1.0: * Tue Sep 21 2004 Jay Lan - Fixed compilation issue with gcc 3.3.3 * Tue Jul 27 2004 Jay Lan - csa-2.1.1 - Remove static API library libcsa.a from rpm package - Fixed a bug at csacom with invalid -u option - cascom headers and data not aligned correctly at output As always, bug reports or feedback are welcome! Signed-off-by: Jay Lan --- Jay Lan - Linux System Software Silicon Graphics Inc., Mountain View, CA From jlan@sgi.com Fri Sep 24 18:30:54 2004 Received: with ECARTIS (v1.0.0; list csa); Fri, 24 Sep 2004 18:31:00 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8P1UsZl014177 for ; Fri, 24 Sep 2004 18:30:54 -0700 Received: from spindle.corp.sgi.com (spindle.corp.sgi.com [198.29.75.13]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8P2eM91019333 for ; Fri, 24 Sep 2004 19:40:22 -0700 Received: from mtv-vpn-hw-jlan-2.corp.sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [134.15.18.195]) by spindle.corp.sgi.com (8.12.9/8.12.9/generic_config-1.2) with ESMTP id i8P1UbKh47124041; Fri, 24 Sep 2004 18:30:42 -0700 (PDT) Received: from sgi.com (mtv-vpn-hw-jlan-2.corp.sgi.com [127.0.0.1]) by mtv-vpn-hw-jlan-2.corp.sgi.com (8.12.8/8.12.8) with ESMTP id i8P1S51f014987; Fri, 24 Sep 2004 18:28:32 -0700 Message-ID: <4154C9A5.4060402@sgi.com> Date: Fri, 24 Sep 2004 18:28:05 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: zh-tw, en-us, en, zh-cn, zh-hk MIME-Version: 1.0 To: Kaigai Kohei CC: LSE-Tech , CSA-ML Subject: Re: [HELP] Building CSA tools failed on ia-64 environment. References: <06e201c494c0$5c567c60$f97d220a@linux.bs1.fc.nec.co.jp> In-Reply-To: <06e201c494c0$5c567c60$f97d220a@linux.bs1.fc.nec.co.jp> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-archive-position: 32 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@sgi.com Precedence: bulk X-list: csa Hi Kaigai, A csa-2.2.0-1.src.rpm was released to oss.sgi.com. This release should fix your compilation issues with gcc 3.3.3 along with a few other bug fixes. Thanks! - jay --- Jay Lan - Linux System Software Silicon Graphics Inc., Mountain View, CA Kaigai Kohei wrote: > Hello, everyone. > I'm interrested in accounting solution for Linux as PAGG/CSA or ELSA, > and so on. First, I tried to build up the PAGG/CSA environment, > and downloaded the related patches and comprehensive tools for CSA/PAGG. > > Then, I succeeded to build up the 2.6.8.1 + PAGG/CSA kernel for IA-64 > archtecture and PAGG related tools like jstat,jkill and so on. > But I failed to build up the CSA related tools. > > > (1) I tried to build up this one from SRPM. > # rpmbuild --rebuild csa-2.1.0-1.src.rpm > Installing csa-2.1.0-1.src.rpm > Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.7433 > : > + automake -a -c -i > error: Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) > > RPM build errors: > Bad exit status from /var/tmp/rpm-tmp.7433 (%prep) > # > > > (2) I tried to build up this one from tarball. > # mkdir -p csa > # cd csa > # rpm2cpio ../csa-2.1.0-1.src.rpm | cpio -i --make-directories > 762 blocks > # ls > csa-2.1.0.tar.gz csa.spec > # tar zxf csa-2.1.0.tar.gz > # cd csa-2.1.0 > # libtoolize --force --copy > # aclocal > # automake -a -c -i > # autoconf > # autoheader > # ./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/share/man > # make > make all-recursive > make[1]: Entering directory `/root/rpm/csa/csa-2.1.0' > Making all in include > make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/include' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/include' > Making all in lib > make[2]: Entering directory `/root/rpm/csa/csa-2.1.0/lib' > gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../include -I/lib/modules/2.6.8.1.pagg.csa/bu > ild/include -D__USE_GNU -Dlinux -g -O2 -c config.c > In file included from config.c:51: > /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:4:2: #error "GCC no l > onger implements ." > /usr/lib/gcc-lib/ia64-redhat-linux/3.3.3/include/varargs.h:5:2: #error "Revise y > our code to use ." > make[2]: *** [config.o] Error 1 > make[2]: Leaving directory `/root/rpm/csa/csa-2.1.0/lib' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/rpm/csa/csa-2.1.0' > make: *** [all] Error 2 > # > > What environment do these tools depend on ? > Any comment please. Thanks. > > I got those packages from as follows. > The kernel patch: ftp://oss.sgi.com/projects/pagg/download/ > ftp://oss.sgi.com/projects/csa/download/ > The tools for PAGG: ftp://oss.sgi.com/projects/pagg/download/job-1.4.0-1.tar.gz > The tools for CSA: ftp://oss.sgi.com/projects/csa/download/csa-2.1.0-1.src.rpm > > -------- > Kai Gai > From jlan@engr.sgi.com Mon Sep 27 15:36:20 2004 Received: with ECARTIS (v1.0.0; list csa); Mon, 27 Sep 2004 15:36:25 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8RMaI3i015774 for ; Mon, 27 Sep 2004 15:36:19 -0700 Received: from nodin.corp.sgi.com (nodin.corp.sgi.com [192.26.51.193]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8RMa6Qh028527 for ; Mon, 27 Sep 2004 17:36:06 -0500 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by nodin.corp.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i8RMa6l234717648 for ; Mon, 27 Sep 2004 15:36:06 -0700 (PDT) Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i8RMYxY914255444; Mon, 27 Sep 2004 15:35:00 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i8RMYNEq024418; Mon, 27 Sep 2004 15:34:25 -0700 Message-ID: <4158956F.3030706@engr.sgi.com> Date: Mon, 27 Sep 2004 15:34:23 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: lse-tech , CSA-ML , Andrew Morton , Guillaume Thouvenin , Tim Schmielau , Arthur Corliss Subject: [PATCH 2.6.9-rc2 0/2] enhanced accounting data collection Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-archive-position: 33 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is an effort of providing an enhanced accounting data collection. It is intended to offer common data collection method for various accounting packages including BSD accouting, ELSA, CSA, and any other acct packages that favor a common layer of data collection, separated from data presentation layer and management of process groups layer. This patchset consists of two parts: acct_io and acct_mm as we identified useful spots for improved data collection in the area of IO and MM. This patchset is to replace the previously submitted CSA patchset of four. The CSA kernel module is a standalone module. The csa_eop patch was to provide a hook for end-of-process handling and that can be considered separately unless there is enough common interest. Now that the patchset is down to IO and MM, i hope it is more appealing :) Comments? Best Regards, - jay --- Jay Lan - Linux System Software Silicon Graphics Inc., Mountain View, CA From jlan@engr.sgi.com Mon Sep 27 15:44:50 2004 Received: with ECARTIS (v1.0.0; list csa); Mon, 27 Sep 2004 15:44:57 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8RMioSU016588 for ; Mon, 27 Sep 2004 15:44:50 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8RMicQh000564 for ; Mon, 27 Sep 2004 17:44:39 -0500 Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i8RMiXY914254192; Mon, 27 Sep 2004 15:44:35 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i8RMi0Eq024449; Mon, 27 Sep 2004 15:44:00 -0700 Message-ID: <415897B0.3060008@engr.sgi.com> Date: Mon, 27 Sep 2004 15:44:00 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: lse-tech , CSA-ML , Andrew Morton , Guillaume Thouvenin , Tim Schmielau , Arthur Corliss Subject: Re: [PATCH 2.6.9-rc2 1/2] enhanced I/O accounting data collection References: <4158956F.3030706@engr.sgi.com> In-Reply-To: <4158956F.3030706@engr.sgi.com> Content-Type: multipart/mixed; boundary="------------020609000803060702000401" X-archive-position: 34 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is a multi-part message in MIME format. --------------020609000803060702000401 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit 1/2: acct_io Enhanced I/O accounting data collection. Signed-off-by: Jay Lan --------------020609000803060702000401 Content-Type: text/plain; name="acct_io" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="acct_io" Index: linux/drivers/block/ll_rw_blk.c =================================================================== --- linux.orig/drivers/block/ll_rw_blk.c 2004-09-12 22:31:31.000000000 -0700 +++ linux/drivers/block/ll_rw_blk.c 2004-09-27 12:37:04.374234677 -0700 @@ -1741,6 +1741,7 @@ { DEFINE_WAIT(wait); struct request *rq; + unsigned long start_wait = jiffies; generic_unplug_device(q); do { @@ -1769,6 +1770,7 @@ finish_wait(&rl->wait[rw], &wait); } while (!rq); + current->bwtime += (unsigned long) jiffies - start_wait; return rq; } Index: linux/fs/read_write.c =================================================================== --- linux.orig/fs/read_write.c 2004-09-12 22:32:55.000000000 -0700 +++ linux/fs/read_write.c 2004-09-27 12:37:04.381070659 -0700 @@ -216,8 +216,11 @@ ret = file->f_op->read(file, buf, count, pos); else ret = do_sync_read(file, buf, count, pos); - if (ret > 0) + if (ret > 0) { dnotify_parent(file->f_dentry, DN_ACCESS); + current->rchar += ret; + } + current->syscr++; } } @@ -260,8 +263,11 @@ ret = file->f_op->write(file, buf, count, pos); else ret = do_sync_write(file, buf, count, pos); - if (ret > 0) + if (ret > 0) { dnotify_parent(file->f_dentry, DN_MODIFY); + current->wchar += ret; + } + current->syscw++; } } @@ -540,6 +546,10 @@ fput_light(file, fput_needed); } + if (ret > 0) { + current->rchar += ret; + } + current->syscr++; return ret; } @@ -558,6 +568,10 @@ fput_light(file, fput_needed); } + if (ret > 0) { + current->wchar += ret; + } + current->syscw++; return ret; } @@ -636,6 +650,13 @@ retval = in_file->f_op->sendfile(in_file, ppos, count, file_send_actor, out_file); + if (retval > 0) { + current->rchar += retval; + current->wchar += retval; + } + current->syscr++; + current->syscw++; + if (*ppos > max) retval = -EOVERFLOW; Index: linux/include/linux/sched.h =================================================================== --- linux.orig/include/linux/sched.h 2004-09-27 11:57:40.220967100 -0700 +++ linux/include/linux/sched.h 2004-09-27 12:52:51.305237393 -0700 @@ -591,6 +591,9 @@ struct rw_semaphore pagg_sem; #endif +/* i/o counters(bytes read/written, #syscalls, waittime */ + unsigned long rchar, wchar, syscr, syscw, bwtime; + }; static inline pid_t process_group(struct task_struct *tsk) --------------020609000803060702000401-- From jlan@engr.sgi.com Mon Sep 27 15:52:03 2004 Received: with ECARTIS (v1.0.0; list csa); Mon, 27 Sep 2004 15:52:11 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8RMq3Af016943 for ; Mon, 27 Sep 2004 15:52:03 -0700 Received: from nodin.corp.sgi.com (nodin.corp.sgi.com [192.26.51.193]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8RMppQh003569 for ; Mon, 27 Sep 2004 17:51:51 -0500 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by nodin.corp.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i8RMpol234639118 for ; Mon, 27 Sep 2004 15:51:50 -0700 (PDT) Received: from aware.engr.sgi.com (aware.engr.sgi.com [163.154.6.184]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with ESMTP id i8RMomY914230168; Mon, 27 Sep 2004 15:50:49 -0700 (PDT) Received: from engr.sgi.com (aware.engr.sgi.com [127.0.0.1]) by aware.engr.sgi.com (8.12.8/8.12.8) with ESMTP id i8RMoFEq024477; Mon, 27 Sep 2004 15:50:15 -0700 Message-ID: <41589927.5080803@engr.sgi.com> Date: Mon, 27 Sep 2004 15:50:15 -0700 From: Jay Lan User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20030225 X-Accept-Language: en-us, en MIME-Version: 1.0 To: LKML CC: lse-tech , CSA-ML , Andrew Morton , Guillaume Thouvenin , Tim Schmielau , Arthur Corliss Subject: Re: [PATCH 2.6.9-rc2 2/2] enhanced MM accounting data collection References: <4158956F.3030706@engr.sgi.com> In-Reply-To: <4158956F.3030706@engr.sgi.com> Content-Type: multipart/mixed; boundary="------------010002010506080406040700" X-archive-position: 35 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: jlan@engr.sgi.com Precedence: bulk X-list: csa This is a multi-part message in MIME format. --------------010002010506080406040700 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit 2/2: acct_mm Enhanced MM accounting data collection. Signed-off-by: Jay Lan --------------010002010506080406040700 Content-Type: text/plain; name="acct_mm" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="acct_mm" Index: linux/fs/exec.c =================================================================== --- linux.orig/fs/exec.c 2004-09-27 11:57:40.201435722 -0700 +++ linux/fs/exec.c 2004-09-27 14:05:41.266160725 -0700 @@ -47,6 +47,7 @@ #include #include #include +#include #include #include @@ -1163,6 +1164,9 @@ /* execve success */ security_bprm_free(&bprm); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return retval; } Index: linux/include/linux/csa_internal.h =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ linux/include/linux/csa_internal.h 2004-09-27 14:05:41.279832688 -0700 @@ -0,0 +1,70 @@ +/* + * Copyright (c) 2000-2002 Silicon Graphics, Inc and LANL All Rights Reserved. + * Copyright (c) 2004 Silicon Graphics, Inc All Rights Reserved. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it would be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 59 Temple Place - Suite 330, Boston MA 02111-1307, USA. + * + * Contact information: Silicon Graphics, Inc., 1500 Crittenden Lane, + * Mountain View, CA 94043, or: + * + * http://www.sgi.com + */ + +/* + * CSA (Comprehensive System Accounting) + * Job Accounting for Linux + * + * This header file contains the definitions needed for communication + * between the kernel and the CSA module. + */ + +#ifndef _LINUX_CSA_INTERNAL_H +#define _LINUX_CSA_INTERNAL_H + +#include + +#if defined (CONFIG_CSA) || defined (CONFIG_CSA_MODULE) + +#include +#include + +static inline void csa_update_integrals(void) +{ + long delta; + + if (current->mm) { + delta = current->stime - current->csa_stimexpd; + current->csa_stimexpd = current->stime; + current->csa_rss_mem1 += delta * current->mm->rss; + current->csa_vm_mem1 += delta * current->mm->total_vm; + } +} + +static inline void csa_clear_integrals(struct task_struct *tsk) +{ + if (tsk) { + tsk->csa_stimexpd = 0; + tsk->csa_rss_mem1 = 0; + tsk->csa_vm_mem1 = 0; + } +} + +#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ + +#define csa_update_integrals() do { } while (0) +#define csa_clear_integrals(task) do { } while (0) +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ + +#endif /* _LINUX_CSA_INTERNAL_H */ Index: linux/include/linux/sched.h =================================================================== --- linux.orig/include/linux/sched.h 2004-09-27 14:04:52.905497872 -0700 +++ linux/include/linux/sched.h 2004-09-27 14:06:35.938387661 -0700 @@ -249,6 +249,8 @@ struct kioctx *ioctx_list; struct kioctx default_kioctx; + + unsigned long hiwater_rss, hiwater_vm; }; extern int mmlist_nr; @@ -593,6 +595,10 @@ /* i/o counters(bytes read/written, #syscalls, waittime */ unsigned long rchar, wchar, syscr, syscw, bwtime; +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) + u64 csa_rss_mem1, csa_vm_mem1; + clock_t csa_stimexpd; +#endif }; @@ -817,6 +823,19 @@ /* Remove the current tasks stale references to the old mm_struct */ extern void mm_release(struct task_struct *, struct mm_struct *); +/* Update highwater values */ +static inline void update_mem_hiwater(void) +{ + if (current->mm) { + if (current->mm->hiwater_rss < current->mm->rss) { + current->mm->hiwater_rss = current->mm->rss; + } + if (current->mm->hiwater_vm < current->mm->total_vm) { + current->mm->hiwater_vm = current->mm->total_vm; + } + } +} + extern int copy_thread(int, unsigned long, unsigned long, unsigned long, struct task_struct *, struct pt_regs *); extern void flush_thread(void); extern void exit_thread(void); Index: linux/kernel/exit.c =================================================================== --- linux.orig/kernel/exit.c 2004-09-27 11:57:40.247334460 -0700 +++ linux/kernel/exit.c 2004-09-27 14:05:41.292528082 -0700 @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -808,6 +809,9 @@ ptrace_notify((PTRACE_EVENT_EXIT << 8) | SIGTRAP); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); acct_process(code); __exit_mm(tsk); Index: linux/kernel/fork.c =================================================================== --- linux.orig/kernel/fork.c 2004-09-27 12:35:06.585377528 -0700 +++ linux/kernel/fork.c 2004-09-27 14:05:41.296434358 -0700 @@ -39,7 +39,7 @@ #include #include #include - +#include #include #include #include @@ -607,6 +607,9 @@ if (retval) goto free_pt; + mm->hiwater_rss = mm->rss; + mm->hiwater_vm = mm->total_vm; + good_mm: tsk->mm = mm; tsk->active_mm = mm; @@ -995,6 +998,8 @@ p->utime = p->stime = 0; p->rchar = p->wchar = p->syscr = p->syscw = 0; p->bwtime = 0; + /* no-op if CONFIG_CSA not set */ + csa_clear_integrals(p); p->lock_depth = -1; /* -1 = no lock */ p->start_time = get_jiffies_64(); p->security = NULL; Index: linux/mm/memory.c =================================================================== --- linux.orig/mm/memory.c 2004-09-12 22:32:26.000000000 -0700 +++ linux/mm/memory.c 2004-09-27 14:05:41.304246908 -0700 @@ -44,6 +44,7 @@ #include #include #include +#include #include #include @@ -605,6 +606,8 @@ tlb = tlb_gather_mmu(mm, 0); unmap_vmas(&tlb, mm, vma, address, end, &nr_accounted, details); tlb_finish_mmu(tlb, address, end); + /* no-op unless CONFIG_CSA is set */ + csa_update_integrals(); spin_unlock(&mm->page_table_lock); } @@ -1095,9 +1098,12 @@ spin_lock(&mm->page_table_lock); page_table = pte_offset_map(pmd, address); if (likely(pte_same(*page_table, pte))) { - if (PageReserved(old_page)) + if (PageReserved(old_page)) { ++mm->rss; - else + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + } else page_remove_rmap(old_page); break_cow(vma, new_page, address, page_table); lru_cache_add_active(new_page); @@ -1379,6 +1385,10 @@ remove_exclusive_swap_page(page); mm->rss++; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + pte = mk_pte(page, vma->vm_page_prot); if (write_access && can_share_swap_page(page)) { pte = maybe_mkwrite(pte_mkdirty(pte), vma); @@ -1444,6 +1454,9 @@ goto out; } mm->rss++; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); entry = maybe_mkwrite(pte_mkdirty(mk_pte(page, vma->vm_page_prot)), vma); @@ -1553,6 +1566,10 @@ if (pte_none(*page_table)) { if (!PageReserved(new_page)) ++mm->rss; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + flush_icache_page(vma, new_page); entry = mk_pte(new_page, vma->vm_page_prot); if (write_access) Index: linux/mm/mmap.c =================================================================== --- linux.orig/mm/mmap.c 2004-09-12 22:32:54.000000000 -0700 +++ linux/mm/mmap.c 2004-09-27 14:05:41.308153183 -0700 @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -1014,6 +1015,9 @@ down_write(&mm->mmap_sem); } __vm_stat_account(mm, vm_flags, file, len >> PAGE_SHIFT); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return addr; unmap_and_free_vma: @@ -1360,6 +1364,9 @@ if (vma->vm_flags & VM_LOCKED) vma->vm_mm->locked_vm += grow; __vm_stat_account(vma->vm_mm, vma->vm_flags, vma->vm_file, grow); + /* no-op if CONFIG_CSA_JOB_ACCT not set */ + csa_update_integrals(); + update_mem_hiwater(); anon_vma_unlock(vma); return 0; } @@ -1816,6 +1823,9 @@ mm->locked_vm += len >> PAGE_SHIFT; make_pages_present(addr, addr + len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); return addr; } Index: linux/mm/mremap.c =================================================================== --- linux.orig/mm/mremap.c 2004-09-12 22:32:48.000000000 -0700 +++ linux/mm/mremap.c 2004-09-27 14:05:41.312059458 -0700 @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -232,6 +233,10 @@ new_addr + new_len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); + return new_addr; } @@ -368,6 +373,9 @@ make_pages_present(addr + old_len, addr + new_len); } + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); ret = addr; goto out; } Index: linux/mm/rmap.c =================================================================== --- linux.orig/mm/rmap.c 2004-09-12 22:33:36.000000000 -0700 +++ linux/mm/rmap.c 2004-09-27 14:05:41.315965733 -0700 @@ -50,6 +50,7 @@ #include #include #include +#include #include #include @@ -580,6 +581,8 @@ } mm->rss--; + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); page_remove_rmap(page); page_cache_release(page); @@ -679,6 +682,8 @@ page_remove_rmap(page); page_cache_release(page); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); mm->rss--; (*mapcount)--; } Index: linux/mm/swapfile.c =================================================================== --- linux.orig/mm/swapfile.c 2004-09-12 22:31:57.000000000 -0700 +++ linux/mm/swapfile.c 2004-09-27 14:05:41.320848577 -0700 @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -435,6 +436,9 @@ set_pte(dir, pte_mkold(mk_pte(page, vma->vm_page_prot))); page_add_anon_rmap(page, vma, address); swap_free(entry); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); + update_mem_hiwater(); } /* vma->vm_mm->page_table_lock is held */ --------------010002010506080406040700-- From pj@sgi.com Tue Sep 28 02:35:02 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 28 Sep 2004 02:35:09 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8S9Z1Nv014696 for ; Tue, 28 Sep 2004 02:35:01 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8SAiwJg020501 for ; Tue, 28 Sep 2004 03:44:58 -0700 Received: from mtv-vpn-hw-pj-2 (mtv-vpn-hw-pj-2.corp.sgi.com [134.15.25.219]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with SMTP id i8S9YgY914370741; Tue, 28 Sep 2004 02:34:43 -0700 (PDT) Date: Tue, 28 Sep 2004 02:33:50 -0700 From: Paul Jackson To: Jay Lan Cc: linux-kernel@vger.kernel.org, lse-tech@lists.sourceforge.net, csa@oss.sgi.com, akpm@osdl.org, guillaume.thouvenin@bull.net, tim@physik3.uni-rostock.de, corliss@digitalmages.com Subject: Re: [PATCH 2.6.9-rc2 2/2] enhanced MM accounting data collection Message-Id: <20040928023350.611c84d8.pj@sgi.com> In-Reply-To: <41589927.5080803@engr.sgi.com> References: <4158956F.3030706@engr.sgi.com> <41589927.5080803@engr.sgi.com> Organization: SGI X-Mailer: Sylpheed version 0.9.12 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 36 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: pj@sgi.com Precedence: bulk X-list: csa nits: 1) I'm not sure the "no-op if CONFIG_CSA not set" comments are worthwhile - it does not seem to be a common practice to mark macros that collapse under certain CONFIG's with such comments, and some code, such as in fork.c, would become quite a bit less readable if such comments were widely used. 2) Three of the added csa_update_integrals() lines have leading spaces, instead of a tab char, such as in: =================================================================== --- linux.orig/fs/exec.c 2004-09-27 11:57:40.201435722 -0700 +++ linux/fs/exec.c 2004-09-27 14:05:41.266160725 -0700 @@ -1163,6 +1164,9 @@ /* execve success */ security_bprm_free(&bprm); + /* no-op if CONFIG_CSA not set */ + csa_update_integrals(); <========= + update_mem_hiwater(); <========= return retval; } 3) Is it always the case that csa_update_integrals() and update_mem_hiwater() are used together? If so, perhaps they could be collapsed into one? Even the current->mm test inside them could be made one test, perhaps? 4) What kind of kernel text size expansion does this cause? There seem to be about a dozen of these calls. What are the pros and cons of inlining csa_update_integrals() and update_mem_hiwater()? Are these on hot enough kernel code paths that we should benchmark with and without these hooks enabled, both inline and out-of-line? -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson 1.650.933.1373 From holt@lnx-holt.americas.sgi.com Tue Sep 28 04:39:43 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 28 Sep 2004 04:39:49 -0700 (PDT) Received: from omx1.americas.sgi.com (omx1-ext.sgi.com [192.48.179.11]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8SBdgAA029696 for ; Tue, 28 Sep 2004 04:39:43 -0700 Received: from flecktone.americas.sgi.com (flecktone.americas.sgi.com [192.48.203.135]) by omx1.americas.sgi.com (8.12.10/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8SBdVQh011341 for ; Tue, 28 Sep 2004 06:39:31 -0500 Received: from thistle-e236.americas.sgi.com (thistle-e236.americas.sgi.com [128.162.236.204]) by flecktone.americas.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i8SBd7OV48233332; Tue, 28 Sep 2004 06:39:07 -0500 (CDT) Received: from lnx-holt.americas.sgi.com (lnx-holt.americas.sgi.com [128.162.233.109]) by thistle-e236.americas.sgi.com (8.12.9/SGI-server-1.8) with ESMTP id i8SBd4tC8994065; Tue, 28 Sep 2004 06:39:05 -0500 (CDT) Received: from lnx-holt.americas.sgi.com (localhost.localdomain [127.0.0.1]) by lnx-holt.americas.sgi.com (8.12.11/8.12.11) with ESMTP id i8SBd2PF001343; Tue, 28 Sep 2004 06:39:03 -0500 Received: (from holt@localhost) by lnx-holt.americas.sgi.com (8.12.11/8.12.11/Submit) id i8SBcwJs001342; Tue, 28 Sep 2004 06:38:58 -0500 Date: Tue, 28 Sep 2004 06:38:58 -0500 From: Robin Holt To: Paul Jackson Cc: Jay Lan , linux-kernel@vger.kernel.org, lse-tech@lists.sourceforge.net, csa@oss.sgi.com, akpm@osdl.org, guillaume.thouvenin@bull.net, tim@physik3.uni-rostock.de, corliss@digitalmages.com Subject: Re: [PATCH 2.6.9-rc2 2/2] enhanced MM accounting data collection Message-ID: <20040928113858.GA1090@lnx-holt.americas.sgi.com> References: <4158956F.3030706@engr.sgi.com> <41589927.5080803@engr.sgi.com> <20040928023350.611c84d8.pj@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040928023350.611c84d8.pj@sgi.com> User-Agent: Mutt/1.4.1i X-archive-position: 37 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: holt@sgi.com Precedence: bulk X-list: csa On Tue, Sep 28, 2004 at 02:33:50AM -0700, Paul Jackson wrote: > nits: > > 3) Is it always the case that csa_update_integrals() and > update_mem_hiwater() are used together? If so, perhaps > they could be collapsed into one? Even the current->mm > test inside them could be made one test, perhaps? This sounds like a really good idea. Maybe update_mem_hiwater should have the #ifdef CONFIG_CSA inside it. This really sounds like a good idea. Is update_mem_hiwater everywhere that csa_update_integrals needs to be? I seem to remember one or two places where that was not the case. Of course that was a few years ago and my memory is really fuzzy > > 4) What kind of kernel text size expansion does this cause? > There seem to be about a dozen of these calls. What are > the pros and cons of inlining csa_update_integrals() and > update_mem_hiwater()? Are these on hot enough kernel code > paths that we should benchmark with and without these hooks > enabled, both inline and out-of-line? The size was never very noticable. It usually did not even cause overflow to the next 4k page. The csa_job module added the biggest bloat, but I have nearly always compiled that as a module. I have benchmarked these hooks a very long time ago. The number and location has not changed appreciably. I ran three seperate tests. The first was without any csa config'd on. The second was with csa config'd on, but no job containers and the writing of the accounting file turned off. Last was with everything. We ran 7 runs of each config on a 32 cpu system. There was no delta between the two kernels that were not writing accounting files. Actually, the average for the with integrals was 0.3% above the without integrals, but this is well within the noise range. Originally, there was a 5% decrease in performance with the writing of the accounting data. There was another unfortunate side effect that some of the CSA metrics became much worse. This problem was later identified and fixed. At that point, CSA logging caused a 2.7% drop in AIM7 peak with shifting the transition point (I think that was the name I remember) towards a higher number of processes. Jack said that was due to a slight serializing of process exits. Robin From pj@sgi.com Tue Sep 28 06:31:08 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 28 Sep 2004 06:31:13 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8SDV7TY000942 for ; Tue, 28 Sep 2004 06:31:08 -0700 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8SEf1Zr000933 for ; Tue, 28 Sep 2004 07:41:01 -0700 Received: from mtv-vpn-hw-pj-2 (mtv-vpn-hw-pj-2.corp.sgi.com [134.15.25.219]) by cthulhu.engr.sgi.com (SGI-8.12.5/8.12.5) with SMTP id i8SDUgY914407210; Tue, 28 Sep 2004 06:30:42 -0700 (PDT) Date: Tue, 28 Sep 2004 06:29:49 -0700 From: Paul Jackson To: Robin Holt Cc: jlan@engr.sgi.com, linux-kernel@vger.kernel.org, lse-tech@lists.sourceforge.net, csa@oss.sgi.com, akpm@osdl.org, guillaume.thouvenin@bull.net, tim@physik3.uni-rostock.de, corliss@digitalmages.com Subject: Re: [PATCH 2.6.9-rc2 2/2] enhanced MM accounting data collection Message-Id: <20040928062949.2ab2249e.pj@sgi.com> In-Reply-To: <20040928113858.GA1090@lnx-holt.americas.sgi.com> References: <4158956F.3030706@engr.sgi.com> <41589927.5080803@engr.sgi.com> <20040928023350.611c84d8.pj@sgi.com> <20040928113858.GA1090@lnx-holt.americas.sgi.com> Organization: SGI X-Mailer: Sylpheed version 0.9.12 (GTK+ 1.2.10; i686-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-archive-position: 38 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: pj@sgi.com Precedence: bulk X-list: csa Robin wrote: > I have benchmarked these hooks a very long time ago. The number and > location has not changed appreciably. These results seem reasonable ... thanks. > The size was never very noticable. But would the time cost of being out of line be noticable either? Actually, being out of line might be a tick faster, if it reduced by a cache line what was needed for a common execution path. > Originally, there was a 5% decrease in performance with the writing of > the accounting data. There was another unfortunate side effect that some > of the CSA metrics became much worse. This problem was later identified > and fixed. Is there any non-trivial risk that some other "unfortunate side affect" exists today, that we'd find on benchmarking? I'm not sure its worth benchmarking again, but I slightly suspect it is, and if benchmarking was done, I'd do it with these calls both inline and out of line, to see what affect that had on runtime. If no affect on runtime, I'd tend toward the out of line calls - at least saving a little kernel text space. -- I won't rest till it's the best ... Programmer, Linux Scalability Paul Jackson 1.650.933.1373 From holt@lnx-holt.americas.sgi.com Tue Sep 28 07:35:44 2004 Received: with ECARTIS (v1.0.0; list csa); Tue, 28 Sep 2004 07:35:49 -0700 (PDT) Received: from omx2.sgi.com (omx2-ext.sgi.com [192.48.171.19]) by oss.sgi.com (8.13.0/8.13.0) with ESMTP id i8SEZhiY002516 for ; Tue, 28 Sep 2004 07:35:43 -0700 Received: from flecktone.americas.sgi.com (flecktone.americas.sgi.com [192.48.203.135]) by omx2.sgi.com (8.12.11/8.12.9/linux-outbound_gateway-1.1) with ESMTP id i8SFjgkr021649 for ; Tue, 28 Sep 2004 08:45:42 -0700 Received: from thistle-e236.americas.sgi.com (thistle-e236.americas.sgi.com [128.162.236.204]) by flecktone.americas.sgi.com (8.12.9/8.12.10/SGI_generic_relay-1.2) with ESMTP id i8SEYJOX48176773; Tue, 28 Sep 2004 09:34:20 -0500 (CDT) Received: from lnx-holt.americas.sgi.com (lnx-holt.americas.sgi.com [128.162.233.109]) by thistle-e236.americas.sgi.com (8.12.9/SGI-server-1.8) with ESMTP id i8SEYHtC11304775; Tue, 28 Sep 2004 09:34:17 -0500 (CDT) Received: from lnx-holt.americas.sgi.com (localhost.localdomain [127.0.0.1]) by lnx-holt.americas.sgi.com (8.12.11/8.12.11) with ESMTP id i8SEYGj0005730; Tue, 28 Sep 2004 09:34:16 -0500 Received: (from holt@localhost) by lnx-holt.americas.sgi.com (8.12.11/8.12.11/Submit) id i8SEYCnl005729; Tue, 28 Sep 2004 09:34:12 -0500 Date: Tue, 28 Sep 2004 09:34:12 -0500 From: Robin Holt To: Paul Jackson Cc: Robin Holt , jlan@engr.sgi.com, linux-kernel@vger.kernel.org, lse-tech@lists.sourceforge.net, csa@oss.sgi.com, akpm@osdl.org, guillaume.thouvenin@bull.net, tim@physik3.uni-rostock.de, corliss@digitalmages.com Subject: Re: [PATCH 2.6.9-rc2 2/2] enhanced MM accounting data collection Message-ID: <20040928143412.GA5608@lnx-holt.americas.sgi.com> References: <4158956F.3030706@engr.sgi.com> <41589927.5080803@engr.sgi.com> <20040928023350.611c84d8.pj@sgi.com> <20040928113858.GA1090@lnx-holt.americas.sgi.com> <20040928062949.2ab2249e.pj@sgi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040928062949.2ab2249e.pj@sgi.com> User-Agent: Mutt/1.4.1i X-archive-position: 39 X-ecartis-version: Ecartis v1.0.0 Sender: csa-bounce@oss.sgi.com Errors-to: csa-bounce@oss.sgi.com X-original-sender: holt@sgi.com Precedence: bulk X-list: csa > > Is there any non-trivial risk that some other "unfortunate side affect" > exists today, that we'd find on benchmarking? When I last owned csa, I was running benchmarks before each SGI release. The tests were a simple matter of grabbing belay or belay2 and running setting up an FC disk vault (one was usually attached that had 16 disks and use Jack's runit script to launch it. I would then take the output and use Jack's web page to graph and compare it to the previous. Additionally, every time I got access to a new larger system, I would run the tests on there and check for any odd affects of CSA. Nothing interesting ever popped up from LBS2.1.1 all the way through to LBS3.0. > > I'm not sure its worth benchmarking again, but I slightly suspect it is, > and if benchmarking was done, I'd do it with these calls both inline and > out of line, to see what affect that had on runtime. If no affect on > runtime, I'd tend toward the out of line calls - at least saving a > little kernel text space. AIM7 is far to big of a hammer to find this level of micro-optimization. You could probably find or write a simple microbenchmark which shows the difference that introducing the code causes, but I would doubt it would show the inline versus the callout. Either way, we have probably spent more time discussing benchmarking this than it is worth at this point of time. I would expect the do_no_page() path will be the easiest to identify the change. I have a simple test which maps a large region and then touches a large number of pages. The whole loop is surronded by reading of the Shub RTC register. This was done to determine the effect of quicklists on page faulting. That type of microbenchmark might be your best bet at finding the problem. Robin