Frank
thanks for the prompt response. Answers inline below:
Joe
On 6/26/14, 4:43 PM, "Frank Ch. Eigler" <fche@xxxxxxxxxx> wrote:
>
>Hi, Joe -
>
>> [...] This linux-specific pmda exposes the hardware performance
>> counters available on recent x86 systems. This pmda has been running
>> on our cluster the last 10 months without problems (I've been
>> meaning to submit this for inclusion for a while!). [...]
>
>Thank you for posting. As you probably know, we have recently started
>expending effort in a very similar direction, so will need to compare
>& contrast the two to figure out how to proceed.
>
>Can you offer a few bits of explanation?
>
>- We have considered using libpfm*/etc. also, but have been leaning toward
> papi, for cross-OS portability. Did you also consider this?
At the time the PMDA was written, the PAPI library didn't have support for
the uncore counters which ruled it out. I think support for uncore and
system-wide counters is present in PAPI now, so this is no longer an
issue.
>
>- The systemwide permanent occupation of the hardware counters by the
> PMDA could easily be problematic. Did you consider on-demand
> enablement instead?
The PMDA does have support for disabling the all of counters on-demand,
but not on an individual basis. The default is for the counters to be
enabled and they are all disabled by locking a (world readable) lock file.
The motivation for this design is that the vast majority of our users
don't use the counters, but we are interesting in logging them. The users
that want to use the counters can get full access by locking the lockfile
(or running the provided binary which locks the file and then sleeps
indefinitely). The lockfile approach was chosen as a way to guarantee that
the counters get automatically re-enabled when the user's job ends (since
their processes are automatically killed by the resource manager at job
end time and the kernel automatically cleans up the lock).
>
>- The configuration file appears to hard-code the perfcounters to be
> multiplexed/available. Did you consider automatically enumerating
> all the available ones on the platform instead?
I don't understand your question. The code checks which hardware PMUs are
available and then enumerates the available counters. The code then
enables the counters that are specified in the relevant hardware PMU
section in the configuration file. Only the counters that are present on
the hardware get loaded. This mechanism allows the same configuration file
to work on multiple different hardware variants, so simplifies the
deployment process.
>
>- Were you considering extending the PMDA to per-process or per-cgroup
> bindings?
I had no current plans to do this, however this sort of thing is
interesting assuming the processing overhead is not too high.
>
>
>- FChE
|