From: Marcin Romaszewicz (marcin++at++asmodean.engr.sgi.com)
Date: 01/31/2000 11:58:19
On Mon, 31 Jan 2000, Benedikt Kessler wrote:
> Hi!
>
> I also had the problems you mentionned! So I looked at the sourse that
> is provided in /usr/share/Performer/src/lib/libpfutil
>
> I discovered a few nasty things (kind of multiprocessing problem: class
> _pfuProc is now a subclass of pfObject and allocated on the shared arena
> in pfuDefaultProcessManager::addProc).
> The whole thing works much better now although I still have a problem on
> a two processor octane: sometimes my application hung when I did some
> mouse or keyboard input. Than I gave the X-server a higher priority that
> any of the performer processes. It's much better again but it still
> hangs sometimes and the x-server restarts then.
>
> Marcin, are these hangs due to the problem with the c-call you
> mentionned?
>
Some of the hangs I have seen (especially on 2 cpu machines) have been
caused by running performer processes too high, and starving some system
service that performer depends on. Running performer processes less than
priority 89 fixes that.
Another source of apparent lockups is isolating all cpus but cpu 0 and
running a realtime process or two (especuially app and compute) on cpu 0.
This prevents the X server from ever running and looks like a lockup. You
can't remotely log in to kill the offending process since the performer
processes preempt your shell.
The third, and most frequent cause of lockups is the libc problem I
mentioned. Performer uses amalloc/afree for memory management and uslocks
for synchronization. It turns out that these functions are unsafe to use
if you run two realtime processes on a single cpu. Say you have a low
priority and a high priority process forced to run on a single cpu. If the
low priority process acquires a lock and gets pre-empted by the high
priority one which tries to take the same lock, the high priority process
will spin trying to get the lock. The high priority process spinning will
prevent the low priority process from freeing the lock, and you have a
deadlock. If any other realtime process on any other cpu tries to take the
lock (this often happens), it will spin too. Now, if you have a realtime
process per cpu, all spinning in a loop on a lock, your system is
rendered useless.
-- Marcin
>
> I attached my modified version of pfuProcessManager.C and
> pfuProcessManager.h. Using gdiff you will see what I did (maybe some of
> the changes were not really necessay).
>
>
> Hope this helps!
>
> Bye! Benedikt
>
>
>
>
>
> Marcin Romaszewicz wrote:
> >
> > Hi Eylon,
> >
> > What you're seeing is a problem with some of the libc functions that
> > performer uses. We're aware of this problem and we're working on fixing it
> > right now. Until we can find a workaround in performer, or get the libc
> > functions fixed, you shouldn't put two realtime processes on a single cpu.
> >
> > I would suggest running compute at a normal (timeshare) priority, since
> > it's performance isn't critical to overall performance. That leaves 7
> > realtime processes and 6 cpu's, we need to find a way to reduce that. If
> > your two culls are cheap, isolate a CPU and put them both on it with a
> > timeshare priority. Since the cpu is isolated, no other processes will run
> > there, so your two culls will behave like they're realtime anyway. If you
> > can't put the culls on one cpu, try seeing if running app in a timeshare
> > priority is acceptable.
> >
> > Also, in irix 6.5+, do not set the priority of any performer process
> > higher than 89. The priorities the os runs certain services at have
> > changed since irix 6.4 days. The priority bands look like this:
> >
> > priorty description
> > -----------------------------------------------------------
> > 255 Reserved c-a thread
> > 240-254 Hard real time
> > 200-239 Interrupt threads
> > 110-199 Interactive real time
> > 90-109 System daemons
> > 0-89 Soft real time
> > (-1) Batch
> > (-3)-(-2) Timeshare
> > (-4) Batch
> > (-5) Weightless
> >
> > So, your priority of 110 for the culls would run them above system
> > daemons, which can cause problems.
> >
> > -- Marcin
> >
> > On Mon, 31 Jan 2000, eylon wrote:
> >
> > > hi performers
> > >
> > > I have 8 processes (3 * DRAW + 3 * CULL + APP + COMPUTE) running on 6
> > > cpu's.
> > > DRAW0, DRAW1, DRAW2 and CULL0 are running each one on its own cpu, CULL1
> > > and the COMPUTE are running on one cpu and so are CULL2 and APP.
> > > I tried to set CULL1 and CULL2 priority to 110 and all other priorities
> > > to 90.
> > > In order to lock the cpu's and set the priorities i used the functions
> > > from libpfutil/lockcpu.c.
> > >
> > > When using the old system call schedctl() it works fine except that it
> > > does not work on 6.5.
> > > On 6.5 when using the new system call sched_setscheduler(), a few
> > > seconds after the priorities where set the performer processes
> > > seem to enter some deadlock and they just stuck.
> > >
> > > I tried to change the priorities to be equal, to set the priorities
> > > without locking the cpu's, to change the scheduler
> > > policy to fifo and then to round-robin but nothing help
> > >
> > > does anyone familiar with the problem or has any idea what's going on
> > > here ???
> > >
> > > thanks in advance
> > >
> > > Eylon.
> > > -----------------------------------------------------------------------
> > > List Archives, FAQ, FTP: http://www.sgi.com/software/performer/
> > > Submissions: info-performer++at++sgi.com
> > > Admin. requests: info-performer-request++at++sgi.com
> > >
> >
> > -----------------------------------------------------------------------
> > List Archives, FAQ, FTP: http://www.sgi.com/software/performer/
> > Submissions: info-performer++at++sgi.com
> > Admin. requests: info-performer-request++at++sgi.com
>
> --
> +---------------------------------+----------------------------------+
> |Benedikt J. Kessler | Silicon Graphics GmbH |
> |Professional Services | Am Hochacker 3 - Technopark |
> |SGI | 85630 Grasbrunn-Neukeferloh, FRG |
> | --- __o ,__o | |
> | ------_ \<,_ _-\_<, | Phone: (+int) 89 46108-366 or -0 |
> |----- (*)/ (*) (*)/'(*) | Fax: (+int) 89 46107-366 |
> +---------------------------------+----------------------------------+
> |E-Mail: bjk++at++sgi.com Web (private): http://reality.sgi.com/bjk |
> | Web: http://www.sgi.de/dienstleistungen/ |
> +--------------------------------------------------------------------+
This archive was generated by hypermail 2b29 : Mon Jan 31 2000 - 11:58:33 PST