From: Jenny Zhao (zhz++at++dandan.engr.sgi.com)
Date: 06/06/2000 19:27:01
On Jun 5, 3:07pm, christopher.g.dorosky++at++lmco.com wrote:
> Subject: Merge and clean time stats
> pfHelp();
>
> Is it possible to get stats on what is happening in pfSync, specifically
> merge and clean time?
>
Hi, Chris,
Since you need some pretty detailed stats information, I'd like to
introduce you to a very powerful real-time tool called
IRIXView. It is a SGI product and shipped with IRIX6.5 system.
I am attaching a webpage describing the tool.
It takes a little while to learn but it is a supper powerful tool.
Performer collects all kinds of stats that can be displayed by
IRIXView. Some of our advanced users use IRIXView to tune
their database paging applications.
The delivered CD is REACT/Pro4.0. It is not free.
You need to work with our SGI sales rep on this.
Here is a portion of our IRIXview information sheet:
1. What is IRIXview?
IRIXview is a performance visualization tool originally designed for real-time
software, but useful for other applications under IRIX as
well. IRIXview helps developers watch the interactions between IRIX and their
software. Think of IRIXview as a Graphical User Interface
(GUI) to par - the process activity reporter. IRIXview displays process and
kernel information taken from timestamps collected by the
real-time monitoring daemon - rtmond (see rtmond(1)).
2. Is IRIXview an SGI product?
Yes, SGI owns, distributes, and supports IRIXview.
3. How does IRIXview differ from Performance Co-Pilot?
IRIXview allows you to visualize the relationships and timing between operating
system events. For example, you can measure how long it
took for an interrupt to execute, how much later that interrupt was serviced,
etc. Performance Co-Pilot, on the other hand, provides
performance monitoring and performance management services for SGI systems. It
is designed to provide information about overall system
performance behavior, resource utilization, activity levels, and bottlenecks in
large complex systems. IRIXview is ideal for the real-time
developer trying to track down latencies and jitters in applications were a few
milliseconds can dramatically change their results.
Performance Co-Pilot would be more useful to a systems administrator who is
concerned with solving the bottleneck problems on a system
that is functioning as a database, file, or web server.
4. What systems and versions of IRIX support IRIXview?
IRIXview can be installed on any SGI system running IRIX 6.5 or greater. Data
can be collected from any other IRIX 6.5 system provided the
system is running rtmond with permissions that permit remote capture of data
(see rtmond(1) for more information). Data CANNOT be
collected from pre-6.5 systems, WindView must be used for that purpose.
IRIXview requires a graphics head or remote color X server for
display.
5. What version of IRIXview should I be running?
It is recommended that the most recent version of IRIXview be used. To verify
that your version is the most recent version issue the
following command from the shell:
versions -b -d irixview
The latest version is dated 12/21/1998:
irixview 12/21/1998 1800 IRIXview system visualization for IRIX, 2.0
If you'd like to update to the latest version contact SGI support.
6. What additional documentation exists for IRIXview?
There is a complete Insight online manual, The IRIXview User's Guide, which is
installed as part of the IRIXview product (it can also be
found on the techpubs website). There is also a short IRIXview for Dummies
document to get one started.
GUI Operation
7. I'm seeing time gaps in the data that is being displayed with IRIXview, what
could be the problem?
By default, rtmond attemps to NOT be too intrusive to an application and runs
at a relatively low RT priority (88) and with less than large
buffers. It is possible, under heavily loaded conditions, for rtmond to drop
data because if this. If one suspects this is true, try running rtmond
in the debug mode:
/etc/init.d/rtmond stop
/usr/etc/rtmond -d
This will print debug messages to the terminal, after collecting a trace via
IRIXview, if events were dropped, you'll see something similar to
the following printed:
Feb 18 14:18:30.41: rtmond[43986]: (CPU 0) Client localhost:18812: 163279
events 202 dropped (0%) 481 writes (0% push) for 7857312 bytes, 5 push
buffers
for 80 KB
If no events were dropped, nothing will print. If events are being lost, try
running rtmond at a higher priority and using more memory as
follows and see if the lost events stop (parameters can be varied - see
rtmond(1)):
/etc/init.d/rtmond stop
/usr/etc/rtmond -b 32768 -p 188 -i 10 -d
If a set of options are found to work, them to the /etc/config/rtmond.options
file.
8. I receive "Host connection could not be established." when attempting to
gather events from other 6.5 systems, what could be
the problem?
In order for IRIXview to collect events from other systems, rtmond must be
initiated on the remote system with non-default options. To
enable event collection on the remote system, remove the following entry from
the /etc/config/rtmond.options file:
-a localhost
Then restart rtmond:
/etc/init.d/rtmond stop
/etc/init.d/rtmond start
9. There are some undocumented contexts displayed in my IRIXview graph, what
are they?
A few of the more frequently encountered undocumented contexts for Origin/Onyx2
systems are:
ip0intrd[x,y]
process states identified as ip0intrd[x,y] are "generic" interrupt
threads on IP27 systems with x being the node# where the interrupt
thread is directed and y being the interrupt level bit# (which I'm
not sure how to correlate to a specific interrupt).
timeinX
process states identified as timeinX are interrupt threads that handle
CPU timeouts (where X is the CPU# where they are running).
10. How do I add my own bitmap icons to user events in IRIXview?
IRIXview permits users to substitute custom bitmap icons to replace the default
user icon (shown as a yellow or orange "V" in the IRIXview
graph.) Once custom bitmaps are created (using bitmap for instance), save them
with filenames as follows:
for User Event 1000: user1000
One must then point IRIXview to their location via an environment variable, for
example:
setenv WV_USER_ICONS <dir>
where dir is the directory that contains the bitmap files. IRIXview must then
be restarted to find the new icon location.
11. How do I clear the verticle line cursors in the context view?
shift + left mouse button
Offline Data Collection (rtmon-client, par, etc.)
12. When should I use rtmon-client to collect IRIXview events?
Use rtmon-client for event data collection to mimimize intrusion to your
real-time process. The rtmon-client utility is used from the
command line to collect data on a per CPU or per system basis. See the
rtmon-clent man page for more information.
13. When should I use par to collect IRIXview events?
Use par to collect event data for a single process. For example, to collect
event data only for a single process, execute the process as follows:
par -r -i -O datafile.irv ./my_process
See the par man page for more information.
14. When I use rtmon-client on under 6.5.x it hangs, what is the problem?
It could be that you have encountered bug #767741. Using the -t option works
around the problem, from the bug report:
rtmon-client fails when multiple CPUs are specified with the
-p option AND if the -t option is not used. Any of the following
fail to produce data (jnk.irv has zero length):
rtmon-client -p 0,7 -f jnk.irv
rtmon-client -p 0-2 -f jnk.irv
While any of the following options are fine:
rtmon-client -p 0,7 -f jnk.irv -t 10
rtmon-client -p 0-2 -f jnk.irv -t 5
rtmon-client -p 0 -f jnk.irv
rtmon-client -f jnk.irv
15. Can I view IRIXview data in text format?
Yes, either through the event inspector window or once the data is saved to a
file via rtmon-dump (see the rtmon-dump man page).
16. Is the IRIXview file format documented anywhere?
No. The reason for not documenting the format is because historically the
format has changed dramatically between OS releases. We thought
it best to not expose this to customers.
17. I cannot seem to get rtmon_log_user_tstamps to work, what could be the
problem?
rtmon_log_user_tstamp requires at least 6.5.2. Also, see the sample code under
/usr/share/src/react/examples/rtmon_user_tstamps.
18. Is there anyway to grab only a certain time interval of an event file
(i.e., shrink the file to only a specific area of interest)?
Here is some unsupported sample code that can be used to create an event file
from a given time interval of a larger file: shrink_irv_file.c
Using IRIXview with Performer
19. Can I use IRIXview with Performer?
Yes, recent versions of Performer libraries has been instrumented with IRIXview
timestamps. Here are some snippets of code that could be
used to enable the production of rtmon/IRIXview data in Performer (C++
libraries):
#include <Performer/pfstats.h>
#include <Performer/pf/pfFrameStats.h>
.
.
pfFrameStats *myFrameStats;
.
.
pfChannel *center_chan = new pfChannel(Shared->p);
.
.
// start collecting irixview statistics
myFrameStats = new pfFrameStats;
myFrameStats = center_chan->getFStats();
myFrameStats->setClass(PFFSTATS_ENRTMON, PFSTATS_ON);
Then just use IRIXview to collect information on the pf processes that are
running.
20. I'm getting an error message when I run Performer code with
PFFSTATS_ENRTMON enabled, what could be the problem?
If you receive the following error message when running Performer:
PF WARNING: rtmon_log_user_tstamp not linked in. Link with -lrtmon.
Then you are probably trying to enable rtmon timestamps in a version of
Performer that does not support it. Try upgradding to the latest
version of Performer.
21. How do I view Performer event numbers?
You'll need to enable viewing of the user event numbers by bringing up the
"Display Events/States" window (the "E" button in the Context
View window) and selecting "Annotate Events" and "Apply".
22. What do the Performer event numbers mean?
The event numbers will then correspond to events in:
/usr/include/Performer/pf/pfFrameStats.h
> Thanks,
>
>
>
> Christopher Dorosky
> Lead Electronic Systems Engineer - Real Time Simulation
> Lockheed Martin Missiles and Fire Control - Dallas
> christopher.g.dorosky++at++lmco.com
> 972-603-2349
> -----------------------------------------------------------------------
> List Archives, FAQ, FTP: http://www.sgi.com/software/performer/
> Submissions: info-performer++at++sgi.com
> Admin. requests: info-performer-request++at++sgi.com
>-- End of excerpt from christopher.g.dorosky++at++lmco.com
-- Jenny Zhao zhz++at++engr.sgi.com 650 933-5091 IRIS Performer Manager
This archive was generated by hypermail 2b29 : Tue Jun 06 2000 - 19:27:09 PDT