netdev
[Top] [All Lists]

Re: ipvs_syncmaster brings cpu to 100%

To: Nishanth Aravamudan <nacc@xxxxxxxxxx>
Subject: Re: ipvs_syncmaster brings cpu to 100%
From: Horms <horms@xxxxxxxxxxxx>
Date: Wed, 28 Sep 2005 11:23:09 +0900
Cc: Luca Maranzano <liuk001@xxxxxxxxx>, Dave Miller <davem@xxxxxxxxxxxxx>, Wensong Zhang <wensong@xxxxxxxxxxxx>, Julian Anastasov <ja@xxxxxx>, netdev@xxxxxxxxxxx
In-reply-to: <20050926142109.GD7532@xxxxxxxxxx>
Mail-followup-to: Nishanth Aravamudan <nacc@xxxxxxxxxx>, Luca Maranzano <liuk001@xxxxxxxxx>, Dave Miller <davem@xxxxxxxxxxxxx>, Wensong Zhang <wensong@xxxxxxxxxxxx>, Julian Anastasov <ja@xxxxxx>, netdev@xxxxxxxxxxx
References: <498263350509081605956a771@xxxxxxxxxxxxxx> <68559cef05092207022f1f0df4@xxxxxxxxxxxxxx> <498263350509230815eb08a73@xxxxxxxxxxxxxx> <20050926032807.GI18357@xxxxxxxxxxxx> <20050926043400.GD5079@xxxxxxxxxx> <20050926080508.GF11027@xxxxxxxxxxxx> <20050926081229.GA23755@xxxxxxxxxxxx> <20050926131104.GA7532@xxxxxxxxxx> <68559cef05092606521cc13f9a@xxxxxxxxxxxxxx> <20050926142109.GD7532@xxxxxxxxxx>
Sender: netdev-bounce@xxxxxxxxxxx
User-agent: Mutt/1.5.10i
On Mon, Sep 26, 2005 at 07:21:09AM -0700, Nishanth Aravamudan wrote:
> On 26.09.2005 [15:52:02 +0200], Luca Maranzano wrote:
> > On 26/09/05, Nishanth Aravamudan <nacc@xxxxxxxxxx> wrote:
> > > On 26.09.2005 [17:12:32 +0900], Horms wrote:
> > > > On Mon, Sep 26, 2005 at 05:05:10PM +0900, Horms wrote:
> > > >
> > > > [snip]
> > > >
> > > > > > > > > Furthermore, if I make an "rgrep" in the source tree of 
> > > > > > > > > kernel 2.6.12
> > > > > > > > > the function schedule_timeout() is more used than the 
> > > > > > > > > ssleep() (517
> > > > > > > > > occurrencies vs. 43), so why in ip_vs_sync.c there was this 
> > > > > > > > > change?
> > > > > > > > >
> > > > > > > > > The other oddity is that Horms reported on this list that on 
> > > > > > > > > non Xeon
> > > > > > > > > CPU the same version of kernel of mine does not present the 
> > > > > > > > > problem.
> > > > > > > > >
> > > > > > > > > I'm getting crazy :-)
> > > > > > >
> > > > > > > I've prepared a patch, which reverts the change which was 
> > > > > > > introduced
> > > > > > > by Nishanth Aravamudan in February.
> > > > > >
> > > > > > Was the 100% cpu utilization only occurring on Xeon processors?
> > > > >
> > > > > That seems to be the only case where were this problem has been
> > > > > observed. I don't have such a processor myself, so I haven't actually
> > > > > been able to produce the problem locally.
> > > > >
> > > > > One reason I posted this issue to netdev was to get some more
> > > > > eyes on the problem as it is puzzling to say the least.
> > > > >
> > > > > > Care to try to use msleep_interruptible() instead of ssleep(), as
> > > > > > opposed to schedule_timeout()?
> > > > >
> > > > > I will send a version that does that shortly, Luca, can
> > > > > you plase check that too?
> > > >
> > > > Here is that version of the patch. Nishanth, I take it that I do not
> > > > need to set TASK_INTERRUPTABLE before calling msleep_interruptible(),
> > > > please let me know if I am wrong.
> > >
> > > Yes, exactly. I'm just trying to narrow it down to see if it's the task
> > > state that's causing the issue (which, to be honest, doesn't make a lot
> > > of sense to me -- with ssleep() your load average will go up as the task
> > > will be UNINTERRUPTIBLE state, but I am not sure why utilisation would
> > > rise, as you are still sleeping...)
> 
> [trimmed lvs-users from my reply, as it is a closed list]
> 
> > Just to add more info, please note the output of "ps":
> > 
> > debld1:~# ps aux|grep ipvs
> > root      3748  0.0  0.0      0     0 ?        D    12:09   0:00
> > [ipvs_syncmaster]
> > root      3757  0.0  0.0      0     0 ?        D    12:09   0:00
> > [ipvs_syncbackup]
> > 
> > Note the D status, i.e. (from ps(1) man page): Uninterruptible sleep
> > (usually IO)
> 
> The msleep_interruptible() change should fix that.
> 
> But that does not show 100% CPU utilisation at all, it shows 0. Did you
> mean to say your load increases?

he full discussion is available online at the follwoing URL:
I can get than information and post it all here if that is
desirable.

http://archive.linuxvirtualserver.org/html/lvs-users/2005-09/msg00031.html

-- 
Horms

<Prev in Thread] Current Thread [Next in Thread>