From owner-lockmeter@oss.sgi.com Tue Dec 4 21:39:35 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fB55dZl11680 for lockmeter-outgoing; Tue, 4 Dec 2001 21:39:35 -0800 Received: from mailsmart1.eth.net ([202.9.178.26]) by oss.sgi.com (8.11.2/8.11.3) with SMTP id fB55dVo11677 for ; Tue, 4 Dec 2001 21:39:32 -0800 Received: from l9a1c0 ([202.9.171.53] unverified) by mailsmart1.eth.net with Microsoft SMTPSVC(5.0.2172.1); Wed, 5 Dec 2001 10:09:31 +0530 Message-ID: <152202001122416396100@l9a1c0> X-EM-Version: 5, 0, 0, 19 X-EM-Registration: #01B0530810E603002D00 X-Priority: 3 X-MSMail-Priority: Normal From: "Murali Kumar" To: lockmeter@oss.sgi.com Subject: Site Name Change Softlandindia.com Date: Tue, 4 Dec 2001 22:09:06 +0500 MIME-Version: 1.0 Content-type: text/plain; charset="US-ASCII" X-OriginalArrivalTime: 05 Dec 2001 04:39:32.0687 (UTC) FILETIME=[D58A2DF0:01C17D46] Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by oss.sgi.com id fB55dXo11678 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk Dear Sir / Madam We have changed our domain name from softlandindia.com to softlandmark.com due to technical problems beyond our control. We greatly regret any inconvenience caused to all our visitors and friends. We humbly request you to bear with us and assure to all that everything else would be the same as before. Softlandmark would always keep in sight the high professional standards for which softlandindia was famous for. We ask your continuous support. You the visitors are the reason for our existence, sole sustenance and the hope for a New Creation. Linux Applications Area Site URL www.softlandmark.com/Linux/ Windows Applications Area Site URL www.softlandmark.com Description India's Website for freeware and shareware. All the Best softwares are listed and organized in Categories with the latest versions. Thanks and Regards, K.Murali Kumar Webmaster www.softlandmark.com From owner-lockmeter@oss.sgi.com Tue Dec 11 13:58:01 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBBLw1D21487 for lockmeter-outgoing; Tue, 11 Dec 2001 13:58:01 -0800 Received: (from hawkes@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBBLw0H21483 for lockmeter@oss.sgi.com; Tue, 11 Dec 2001 13:58:00 -0800 Date: Tue, 11 Dec 2001 13:58:00 -0800 From: John Hawkes Message-Id: <200112112158.fBBLw0H21483@oss.sgi.com> To: lockmeter@oss.sgi.com Subject: Lockmeter available for 2.4.16 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk FYI, the latest Lockmeter, version 1.4.10, is now available as a patch against the 2.4.16 kernel (as well as against the 2.4.14 kernel): http:/oss.sgi.com/projects/lockmeter/download/lockmeter1.4.10-2.4.16.diff.gz John Hawkes hawkes@sgi.com From owner-lockmeter@oss.sgi.com Fri Dec 14 07:49:03 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBEFn3A02565 for lockmeter-outgoing; Fri, 14 Dec 2001 07:49:03 -0800 Received: from e31.co.us.ibm.com (e31.co.us.ibm.com [32.97.110.129]) by oss.sgi.com (8.11.2/8.11.3) with SMTP id fBEFmvo02562; Fri, 14 Dec 2001 07:48:57 -0800 Received: from westrelay03.boulder.ibm.com (westrelay03.boulder.ibm.com [9.99.140.24]) by e31.co.us.ibm.com (8.9.3/8.9.3) with ESMTP id JAA09224; Fri, 14 Dec 2001 09:46:05 -0500 Received: from maze.in.ibm.com (maze.in.ibm.com [9.186.135.29]) by westrelay03.boulder.ibm.com (8.11.1m3/NCO v5.01) with ESMTP id fBEEmix29758; Fri, 14 Dec 2001 07:48:50 -0700 Received: (from maneesh@localhost) by maze.in.ibm.com (8.11.2/8.11.2) id fBEEuDd20751; Fri, 14 Dec 2001 20:26:13 +0530 Date: Fri, 14 Dec 2001 20:26:13 +0530 From: Maneesh Soni To: hawkes@oss.sgi.com Cc: lse-tech , lockmeter@oss.sgi.com Subject: Lockmeter 1.4.10 problem Message-ID: <20011214202613.C17111@in.ibm.com> Reply-To: maneesh@in.ibm.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-lockmeter@oss.sgi.com Precedence: bulk Hi John, The changes you have done for atomic_dec_and_lock() in Lockmeter v1.4.10 does not seem to give correct statistics. As with your changes now the lock is taken all the time in atomic_dec_and_lock, which is not the case without lockmeter. This leads to wrong analysis of locks. If you want to see what's the difference just look at the folowing numbers With Lockmeter v1.4.10 ---------------------- 6.3% 9.2% 0.4us(1659us) 3.4us(1648us)( 1.3%) 23182304 90.8% 9.2% 0% dcache_lock 2.3% 6.4% 0.3us( 120us) 3.3us(1379us)(0.48%) 12336769 93.6% 6.4% 0% dput+0x18 Making actual atomic_dec_and_lock inline ---------------------------------------- 4.0% 8.6% 0.5us(1333us) 1.9us(1309us)(0.35%) 11940384 91.4% 8.6% 0% dcache_lock 0.3% 7.0% 0.4us( 191us) 1.9us(1197us)(0.03%) 1096658 93.0% 7.0% 0% dput+0x30 There is a difference of more than 10 times in number of times dcache_lock is taken from dput. I didnot face any problem in making the actual atomic_dec_and_lock() inline and could not understand why you choose the less efficient version. Regards, Maneesh -- Maneesh Soni IBM Linux Technology Center, IBM India Software Lab, Bangalore. Phone: +91-80-5044999 email: maneesh@in.ibm.com http://lse.sourceforge.net/locking/rcupdate.html From owner-lockmeter@oss.sgi.com Fri Dec 14 11:15:46 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBEJFkp06486 for lockmeter-outgoing; Fri, 14 Dec 2001 11:15:46 -0800 Received: from rj.sgi.com (rj.sgi.com [204.94.215.100]) by oss.sgi.com (8.11.2/8.11.3) with SMTP id fBEJFeo06483; Fri, 14 Dec 2001 11:15:40 -0800 Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by rj.sgi.com (8.11.4/8.11.4/linux-outbound_gateway-1.1) with ESMTP id fBEIFXY32680; Fri, 14 Dec 2001 10:15:33 -0800 Received: from wrlarun (sshgate.corp.sgi.com [169.238.216.146]) by cthulhu.engr.sgi.com (SGI-8.9.3/8.9.3) with SMTP id KAA16607; Fri, 14 Dec 2001 10:14:17 -0800 (PST) Message-ID: <004101c184ca$bee31d60$6601a8c0@attbi.com> From: "John Hawkes" To: , Cc: "lse-tech" , References: <20011214202613.C17111@in.ibm.com> Subject: Re: Lockmeter 1.4.10 problem Date: Fri, 14 Dec 2001 10:11:25 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6600 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk > The changes you have done for atomic_dec_and_lock() in Lockmeter v1.4.10 > does not seem to give correct statistics. I'm quite willing to believe my 1.4.10 is wrong, but I think the better terminology is "does not seem to give identical statistics", not "does not seem to give correct statistics." > If you want to see what's the difference just look at the folowing numbers > > With Lockmeter v1.4.10 > ---------------------- > 6.3% 9.2% 0.4us(1659us) 3.4us(1648us)( 1.3%) 23182304 90.8% 9.2% 0% dcache_lock > 2.3% 6.4% 0.3us( 120us) 3.3us(1379us)(0.48%) 12336769 93.6% 6.4% 0% dput+0x18 > > Making actual atomic_dec_and_lock inline > ---------------------------------------- > 4.0% 8.6% 0.5us(1333us) 1.9us(1309us)(0.35%) 11940384 91.4% 8.6% 0% dcache_lock > 0.3% 7.0% 0.4us( 191us) 1.9us(1197us)(0.03%) 1096658 93.0% 7.0% 0% dput+0x30 > > There is a difference of more than 10 times in number of times dcache_lock is > taken from dput. I didnot face any problem in making the actual > atomic_dec_and_lock() inline and could not understand why you choose the less > efficient version. I'm guessing that you're looking at i386, which I believe means that the code that executes is found in arch/i386/lib/dec_and_lock.c, right? That implementation has a "fast path" that uses a direct asm sequence using cmpxchgl, and a "slow path" that invokes spin_lock() and spin_unlock(). Lockmeter, of course, will only instrument this "slow path". You simply won't see any 1.4.9 statistics for the "fast path". What I did in 1.4.10 was to force everything through the "slow path", which means that all those formerly invisible "fast path" spinlock acquisitions are now showing up as metered spin_lock/spin_unlock pairs. So I would argue that 1.4.9 not only aggregates lots of spinlock/unlock activity into appearing as atomic_dec_and_lock() calls -- which 1.4.10 fixes to now make that activity visible in the actual spinlock acquisition routine, such as dput() -- but 1.4.9 also completely misses quite a few spinlock acquisitions that were going through that "fast path". So I would argue that 1.4.10 actually produces correct statistics, not simply *different* statistics, vs. 1.4.9. John Hawkes From owner-lockmeter@oss.sgi.com Sun Dec 16 20:52:58 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBH4qw727811 for lockmeter-outgoing; Sun, 16 Dec 2001 20:52:58 -0800 Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227]) by oss.sgi.com (8.11.2/8.11.3) with SMTP id fBH4qoo27808; Sun, 16 Dec 2001 20:52:51 -0800 Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.ibm.com [9.37.3.210]) by e21.nc.us.ibm.com (8.9.3/8.9.3) with ESMTP id VAA169798; Sun, 16 Dec 2001 21:49:09 -0600 Received: from maze.in.ibm.com ([9.182.12.243]) by southrelay03.raleigh.ibm.com (8.11.1m3/NCO v5.01) with ESMTP id fBH3qZs188088; Sun, 16 Dec 2001 22:52:36 -0500 Received: (from maneesh@localhost) by maze.in.ibm.com (8.11.2/8.11.2) id fBH406b32398; Mon, 17 Dec 2001 09:30:06 +0530 Date: Mon, 17 Dec 2001 09:30:06 +0530 From: Maneesh Soni To: John Hawkes Cc: hawkes@oss.sgi.com, lse-tech , lockmeter@oss.sgi.com Subject: Re: Lockmeter 1.4.10 problem Message-ID: <20011217093006.E17111@in.ibm.com> Reply-To: maneesh@in.ibm.com References: <20011214202613.C17111@in.ibm.com> <004101c184ca$bee31d60$6601a8c0@attbi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <004101c184ca$bee31d60$6601a8c0@attbi.com>; from hawkes@sgi.com on Fri, Dec 14, 2001 at 10:11:25AM -0800 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk On Fri, Dec 14, 2001 at 10:11:25AM -0800, John Hawkes wrote: > I'm guessing that you're looking at i386, which I believe means that the > code that executes is found in arch/i386/lib/dec_and_lock.c, right? Yes I am looking at i386 code. > That implementation has a "fast path" that uses a direct asm sequence > using cmpxchgl, and a "slow path" that invokes spin_lock() and > spin_unlock(). Lockmeter, of course, will only instrument this "slow > path". You simply won't see any 1.4.9 statistics for the "fast path". > > What I did in 1.4.10 was to force everything through the "slow path", > which means that all those formerly invisible "fast path" spinlock > acquisitions are now showing up as metered spin_lock/spin_unlock pairs. > > So I would argue that 1.4.9 not only aggregates lots of spinlock/unlock > activity into appearing as atomic_dec_and_lock() calls -- which 1.4.10 > fixes to now make that activity visible in the actual spinlock > acquisition routine, such as dput() -- but 1.4.9 also completely misses > quite a few spinlock acquisitions that were going through that "fast > path". So I would argue that 1.4.10 actually produces correct > statistics, not simply *different* statistics, vs. 1.4.9. The fast path does _not_ take global lock..and I don't think it is a problem if lockmeter does not measure stats for the fast path. I think it is more important to get correct stats for global lock acquisitions at-least for scalability goals. By saying "correct" I mean "as it happens without lockmeter" Without lockmeter all calls to atomic_dec_and_lock() do not acquire global lock and lockmeter (v1.4.10) changes this behavior by forcing all calls to atomic_dec_and_lock() to slow path. I got mislead due to this as I had to investigate why there was 10 fold increase in dcache_lock() acquisitions from dput(). The problem of aggregation can be solved if the whole of atomic_dec_and_lock() is made in-line, not only the slow path code. Regards, Maneesh -- Maneesh Soni IBM Linux Technology Center, IBM India Software Lab, Bangalore. Phone: +91-80-5044999 email: maneesh@in.ibm.com http://lse.sourceforge.net/locking/rcupdate.html From owner-lockmeter@oss.sgi.com Mon Dec 17 04:24:41 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBHCOfr11176 for lockmeter-outgoing; Mon, 17 Dec 2001 04:24:41 -0800 Received: from e21.nc.us.ibm.com (e21.nc.us.ibm.com [32.97.136.227]) by oss.sgi.com (8.11.2/8.11.3) with SMTP id fBHCObo11173; Mon, 17 Dec 2001 04:24:37 -0800 Received: from southrelay03.raleigh.ibm.com (southrelay03.raleigh.ibm.com [9.37.3.210]) by e21.nc.us.ibm.com (8.9.3/8.9.3) with ESMTP id FAA163968; Mon, 17 Dec 2001 05:21:03 -0600 Received: from quark.in.ibm.com ([9.182.12.244]) by southrelay03.raleigh.ibm.com (8.11.1m3/NCO v5.01) with ESMTP id fBHBOTs60738; Mon, 17 Dec 2001 06:24:29 -0500 Received: (from dipankar@localhost) by quark.in.ibm.com (8.11.6/8.11.6) id fBHBU3505423; Mon, 17 Dec 2001 17:00:03 +0530 Date: Mon, 17 Dec 2001 17:00:02 +0530 From: Dipankar Sarma To: John Hawkes Cc: maneesh@in.ibm.com, hawkes@oss.sgi.com, lse-tech , lockmeter@oss.sgi.com Subject: Re: [Lse-tech] Re: Lockmeter 1.4.10 problem Message-ID: <20011217170002.C1488@in.ibm.com> Reply-To: dipankar@in.ibm.com References: <20011214202613.C17111@in.ibm.com> <004101c184ca$bee31d60$6601a8c0@attbi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <004101c184ca$bee31d60$6601a8c0@attbi.com>; from hawkes@sgi.com on Fri, Dec 14, 2001 at 10:11:25AM -0800 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk Hi John, On Fri, Dec 14, 2001 at 10:11:25AM -0800, John Hawkes wrote: > > The changes you have done for atomic_dec_and_lock() in Lockmeter > v1.4.10 > > does not seem to give correct statistics. > > I'm guessing that you're looking at i386, which I believe means that the > code that executes is found in arch/i386/lib/dec_and_lock.c, right? > That implementation has a "fast path" that uses a direct asm sequence > using cmpxchgl, and a "slow path" that invokes spin_lock() and > spin_unlock(). Lockmeter, of course, will only instrument this "slow > path". You simply won't see any 1.4.9 statistics for the "fast path". AFAICS, the cmpxchgl code has nothing to do with the lock, it is used with the reference counter. If refcount is not zero, the lock is *not* taken. By taking the lock every time on invocation, the current lockmeter skews the real picture that is the acquisition of the global lock. Thanks Dipankar -- Dipankar Sarma http://lse.sourceforge.net Linux Technology Center, IBM Software Lab, Bangalore, India. From owner-lockmeter@oss.sgi.com Tue Dec 18 16:08:05 2001 Received: (from majordomo@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBJ085K23016 for lockmeter-outgoing; Tue, 18 Dec 2001 16:08:05 -0800 Received: (from hawkes@localhost) by oss.sgi.com (8.11.2/8.11.3) id fBJ082h23010 for lockmeter@oss.sgi.com; Tue, 18 Dec 2001 16:08:02 -0800 Date: Tue, 18 Dec 2001 16:08:02 -0800 From: John Hawkes Message-Id: <200112190008.fBJ082h23010@oss.sgi.com> To: lockmeter@oss.sgi.com Subject: Lockmeter 1.4.11 available against 2.4.16 Sender: owner-lockmeter@oss.sgi.com Precedence: bulk FYI, a new Lockmeter version 1.4.11 is now available against the 2.4.16 kernel: http://oss.sgi.com/projects/lockmeter/download/lockmeter1.4.11-2.4.16.diff.gz Version 1.4.10 inlined a "slow" version of atomic_dec_and_lock(), which meant that the callers of atomic_dec_and_lock() were credited with the underlying spin_lock() calls, instead of having the various different spinlocks be grossly aggregated and credited to the generic atomic_dec_and_lock(). Unfortunately, that version 1.4.10 overemphasized the number of spin_lock() calls because it was no longer using the common "fast path" of the i386 atomic_dec_and_lock(). This new version 1.4.11 remedies this shortcoming. The ia64 implementation continues to use a "slow path" inline implementation of atomic_dec_and_lock(). The sparc64 and alpha implementations continue to use the standard non-inline version of atomic_dec_and_lock(), which means (I believe) that the underlying spin_lock() calls aggregate to atomic_dec_and_lock(), rather than be broken up and credited to the callers of atomic_dec_and_lock(). John Hawkes hawkes@sgi.com