xfs
[Top] [All Lists]

Re: TAKE 950027 - xfs_icsb_lock_all_counters fails with CONFIG_PREEMPT a

To: Linus Torvalds <torvalds@xxxxxxxx>
Subject: Re: TAKE 950027 - xfs_icsb_lock_all_counters fails with CONFIG_PREEMPT and >=256p
From: Andi Kleen <ak@xxxxxxx>
Date: Thu, 2 Mar 2006 19:15:08 +0100
Cc: David Chinner <dgc@xxxxxxx>, linux-xfs@xxxxxxxxxxx, mingo@xxxxxxx, tony.luck@xxxxxxxxx
In-reply-to: <Pine.LNX.4.64.0603021003420.22647@xxxxxxxxxxx>
References: <20060301125320.20FDA49F1681@xxxxxxxxxxxxxxxxxxxxxxx> <200603021852.35443.ak@xxxxxxx> <Pine.LNX.4.64.0603021003420.22647@xxxxxxxxxxx>
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: KMail/1.9.1
On Thursday 02 March 2006 19:06, Linus Torvalds wrote:

> That's PLENTY.
> 
> First off, name somebody who sells bigger systems than that.

512 socket Altix with Montecitos - 4 threads per socket = 2048 virtual CPUs.
Not quite selling, but at some point it will.

> Second, taking two spinlocks per CPU is crazy in the first place. Who does 
> that? Talk about scaling badly.

It would be reasonable for slow path code. And I'm worrying more about 
corner cases that might be only taken in exception situations etc.

[e.g. i was just debugging a problem in stop_machine() on a large system - 
these are exactly these kinds of nasty corner cases]

Also I think we need some margin here - so that even code that is slow path etc.
is not likely to run into the ceiling. 


> Third, code that cares can avoid it entirely by just bumping the preempt 
> counter _once_, and then using the raw spinlock code. That's how you 
> should do it for big-rw-locks anyway, just because it's less insane, if 
> that is what XFS is doing with its "two spinlocks per CPU" thing.

The issue I'm worrying about is that this code will be likely written
to small machines and never bump into this so nobody will care. And then
suddenly it explodes when the code runs on a larger machine. It's a nasty
trap.
 
> Fourth, you ignore the very real possibility that decreasing the size of 
> the other fields can cause problems,

256 softirq nestings would be really crazy in my opinion. It cannot be 
real softirqs because the stack would just overflow and code just shouldn't
be that deeply nested.

Agreed 64bits would be better though.

-Andi


<Prev in Thread] Current Thread [Next in Thread>