Example 3

Table of Contents

• Example 3
  • Table of Contents
  • Simple example scenario 3 to show command usage
  • More detailed example scenario 3 to show command usage

Simple example scenario 3 to show command usage

• Simple list of the commands for the scenario that will create an experiment that inserts instrumentation into a running application that consists of 2 pids running on 2 separate host machines.

  The pid on host1 consists of 300 threads while the one on host2 has 400. The builtin pcsampling and io collectors are used in concert with 2 user-defined ones: user1 and user2.

  The original application consisted of 2 identical copies of a.out runing on 2 host machines. The application is being modified to use a new algorithm for one of it's processing steps. At this time, the code changes are only available in the executable that is on host2 . It will cause 100 more threads to be created on that host. The changes that are important are connected to threads 12 to 96. This experiment is to identify certain performance characteristics of the modified section of code.

  The new section of code is entered when the entry point startxyz on host2 is called in thread number 1. We will bypass data collection until that point.

  The startup code sequence that begins with entry point startxyz initiates set of Input and Output operations. The initial set of IO operations is generic and we will skip them. After that, we want to determine the IO behavior for both host1 and host2. Once thread 128 of host1 reaches the entry point flyingRT, we can start measuring the IO operations we care about. We can stop once thread 78 of host2 reaches the entry point flyingST. We know that thread #299 on host1 and thread #399 on host2 are the only ones that perform IO operations. We will use the io collector on just those threads.

  Once we have completed using the io collector and the program has completed it's initial set up operations, the new algorithm will be executed. We will use the user defined data collector, user2 to gather the information we are seeking and will apply it to only the threads on host2 where it has been implemented.

  Once the entry point flyingST is reached a second time, we know that the new algorithm has been completed. We do not need to collect anymore performance data and can let the program run to completion.

  We want to collect user1 measurements and pcsamp information for as much of the program as we can, without interfering with the performance collection related to analyzing the changed code. At various points in execution, we will need to disable these collectors.

  Before running the io collector, a user supplied script will be run that will generate a special report that will be used later on, for a purpose that we do not need to be concerned about.

  For simplicity, the focused experiment is the only one of concern snd does not need to be made explicit on the various commands.

More detailed example scenario 3 to show command usage

• An annotated list of the commands for the scenario that will create an experiment that inserts instrumentation into a running application that consists of 2 pids running on 2 separate host machines.

# Find the pids associated with the running execution of a.out on the machines named host1 and host2.

# Define an experiment with these pids. This application is known to have 300 threads on host1 and 400 on host2.

# Set a breakpoint that is known to be associated with the start of a processing step that is of interest for performance analysis. Then run the application until the next time that the breakpoint is reached.
# Since the application was already executing, the expGo command will restart the application.

# After some period of time, the breakpoint is reached and the entire running application is stopped.

# Decide how to instrument the application, based on the types of collectors available for use. There may be a lot more output, describing paramters, metrics, reports and views. The details of these commands have not yet been determined.

# Add the pcasmp collector to all the threads on both hosts.

# Add the user1 collector to all the threads except #299 and #399. Note that threads 300:399 don't exist on host1, but do on host2.

# Oops! It's only the last thread on each host where we don't want pcsamp to be collecting performance data, The previous command skipped a thread on host2 that we need to include.

# There are also some special threads on host2 that we don't want to collect pcasmp data for. Exclude them.

# Set a breakpoint that is known to be associated with the start of a processing step where we will want to add additional instrumentation.

# We're off to the races!!

# Dang! We don't care about breakpoint #1 anymore. Get rid of it and get going!

# After a while, we (finally) get to breakpoint #2.

# Get rid of breakpoint #2 and set another one of interest.

# Stop collection of pcsamp data for all threads on both hosts. Note that previously collected data samples will be preserved and that the user1 collector remains active.

# Collect io data about the only 2 threads that are known to handle intermediate files for the application.

# Now run a script that does some special anaylsis and generates a repoort that is of interest. Access to this script may be implementation dependant, since it is on another host. We may only be able to read cross-mounted files systems.

# Get back to the business of running the application.

# At this point we need to examine what is going on.

# At this point we need to examine the collected data and decide what to do. Open a GUI window to examine the data collected by user1.

# It is clear from the data that all the intermediate IO is complete. Remove the io collector from all threads, since it is no longer of interest. Note that the data accumulated by the io collector is not discarded and can be examined later.

# We will restart the pcsamp collector on all threads. New data will be added to the previosuly collected samples although there will be a gap in time where there are no samples about the program.

# Let the program run for an hour or two and then stop it to examine it's behavior.

# First, we take a look at the old io samples that were generated. This is not current information, but the realtime display for user1View suggests that there may be an anomoly that should be checked out.

# Hummm.... Something strange is going on. We don't want to get diverted from what we set out to do today, but we better save all the data that's been collected and we will look at it later. We will save it in a file on host1.

# What is happening on host1 is no longer of interest. Get rid of the pcsamp and user1 collectors that are still active on the host.

# Further more, we don't want to collect any more pcsamp data for host2, but keep the user1 collector.

# Information on the user1View display, indicate that this is the time to turn on our special user2 collector. It will be restricted to a subset of threads on host2.

# The user2 collector can generate a realtime display, so let's open it up and watch how the application is performing while it is executing.

# The user2 collector can generate a realtime display, so let's open it up and watch how the application is performing while it is executing.

# At some point, we reach the breakpoint that marks the end of an iteration in the application. We have collected data for a complete cycle and no longer need to instrument the program. The following expDetach command stops all data collection and disconnects the application from the performance analysis tool. It does not throw away the data that has been collected, although there is no way to View it unless a collector is re-attached to the experiment. Since the application is no longer controlled by the performance tool, it can continue to run.

# The exit command closes the performance analysis tool. All performance data is discarded and all related displays are closed. Since the application has already been disconnected, it continues to run to completion.