pcp
[Top] [All Lists]

New open-source Java library to collect PCP metrics

To: pcp@xxxxxxxxxxx
Subject: New open-source Java library to collect PCP metrics
From: Paul Cowan <cowan@xxxxxxxxxx>
Date: Fri, 26 Jun 2009 15:23:56 +1000
Organization: Aconex
User-agent: Thunderbird 2.0.0.21 (X11/20090409)
Hi all,

As you may have gathered, I've been working with Nathan Scott on coming
up with a Java library to simply and easily pass values to PCP via the
recently-open-sourced MMV agent. We had a custom implementation of an
MMV-like agent here at Aconex, which was not really suitable for
general use; now that the MMV agent proper is freely available,
Nathan's done some grade-A hacking on it already to introduce
some missing features for our needs, and the Java library to go
along with it is likewise nearly ready; I thought I'd post a quick
intro here to get some feedback on the API before we do a first
formal release and it gets set in stone (well, MORE set).

First off, how to get it! (so you can read along; whenever you hear
*this* sound, you'll know it's time to turn the page, etc). The new
library is a (standalone) part of Parfait, an
in-the-process-of-being-open-sourced (i.e. it's still very rough and
ready, with much more to do) set of Java performance tools developed
here at Aconex. Parfait contains a bunch of stuff (more about that to
follow later, as the library proper may be of more general interest),
but the important part for now is DXM, the Java library for the
MMV agent [1].

The key facts:

License:
        Apache 2.

Project home page:
        http://code.google.com/p/parfait/

To view the source:
        http://code.google.com/p/parfait/source/browse/#svn/trunk

To check out the source using Subversion:
        svn co http://parfait.googlecode.com/svn/trunk/ parfait

The build process uses Maven 2; for those not familiar with it, building is as simple as (requires Java 1.6 ATM, no effort's been put
into backcompat):
        mvn package
(this builds the whole library; the DXM jar, specifically, will end
up in dxm/target/dxm-<VERSION>.jar)

Maven repo (as yet unpopulated, because there's been no release, but file it away for future reference)
        http://repo.custardsource.com/releases/

So, how does it work? Well, the API is (I think) extremely simple and
quite clean. You create a 'writer' object, add the values you'll be
monitoring, start the writer (which creates the memory-mapped file), and
update as you please.

A given metric is associated with a converter responsible for mapping Java types to PCP bytes; mappings are provided for most common
types (all the primitives, and a couple of extras, like AtomicInteger).

Custom converters for your own domain types can be added through a simple API, either by registering the converter to handle all values of a given class or by choosing a converter one-by-one for individual metrics.

The best way to explain is, as always, a demonstration; here's some
dummy code (currently still sitting in the source as a test, until I
hook it out) which demonstrates most of the features. I've annotated it
to make it clear what's going on. Formatting's a bit naff, sorry.

--

  PcpMmvWriter bridge = new PcpMmvWriter(
    new File("/var/tmp/mmv/mmvtest"));

  // Automatically uses default int handler
  bridge.addMetric(
    MetricName.parse("sheep[baabaablack].bagsfull.count"), 3);

  // Automatically uses default boolean-to-int handler
  bridge.addMetric(
    MetricName.parse("sheep[baabaablack].bagsfull.haveany"),
          new AtomicBoolean(true));

  bridge.addMetric(
    MetricName.parse("sheep[limpy].bagsfull.haveany"),
          new AtomicBoolean(false));

  // Automatically uses default long handler
  bridge.addMetric(
    MetricName.parse("sheep[insomniac].jumps"), 12345678901234L);

  // Automatically uses default double handler
  bridge.addMetric(
    MetricName.parse("sheep[limpy].legs.available"), 0.75);

  // Uses this class' custom String handler
  bridge.addMetric(MetricName.parse("sheep[limpy].jumpitem"), "fence");

  // addMetric(GregorianCalendar) would fail, as there's no
  // handler registered by default for GregorianCalendars; use
  // a custom one which puts the year as an int
  bridge.addMetric(
    MetricName.parse("sheep[insomniac].lastjumped"),
    new GregorianCalendar(),
    new AbstractTypeHandler<GregorianCalendar>(MmvMetricType.I32, 4) {
      public void putBytes(ByteBuffer buffer, GregorianCalendar value) {
        buffer.putInt(value.get(GregorianCalendar.YEAR));
        }
    });


  // addMetric(Date) would fail, as there's no handler registered;
  // register one for all date types from now on
  bridge.registerType(Date.class,
    new AbstractTypeHandler<Date>(MmvMetricType.I64, 8) {
      public void putBytes(ByteBuffer buffer, Date value) {
          buffer.putLong(value.getTime());
      }
  });

  // These will both use the handler we just registered
  bridge.addMetric(MetricName.parse("cow.how.now"),
    new Date());
  bridge.addMetric(MetricName.parse("cow.how.then"),
    new GregorianCalendar(1990, 1, 1, 12, 34, 56).getTime());

  // Set up some help text
  bridge.setInstanceDomainHelpText(
    "sheep", "sheep in the paddock",
"List of all the sheep in the paddock. Includes 'baabaablack', 'insomniac' (who likes to jump fences), and 'limpy' the three-legged wonder sheep.");

  bridge.setMetricHelpText("sheep.jumps", "# of jumps done",
          "Number of times the sheep has jumped over its jumpitem");

  // All the metrics are added; write the file
  bridge.start();
  // Metrics are visible to the agent from this point on

  // Sold a bag! Better update the count
  bridge.updateMetric(
    MetricName.parse("sheep[baabaablack].bagsfull.count"), 2);

  // The fence broke! Need something new to jump over
  bridge.updateMetric(MetricName.parse("sheep[limpy].jumpitem"),
    "Honda Civic");

  // Values will be reflected in the agent immediately

--

I trust this is all clear; I think the API is very self-explanatory,
hopefully you should see where we're going with this. If you have any
questions, yell out.

I'd very much like to know if anyone's interested in using or
contributing to this library: I'm a huge fan of PCP and think that the
easier it is to get data into it, the better. Making that easy to do
from Java is, I think, a useful step.

Let me know what you think.

Kind regards,

Paul

[1] I know someone's going to ask 'why DXM?', so:

  "Dextromethorphan (DXM or DM) is ... one of the active
   ingredients used to prevent coughs in many over-the-counter
   cold and cough medicines ... dextromethorphan acts as a
   dissociative psychedelic drug ... [it is] an NMDA receptor
   antagonist, producing effects similar to ...   phencyclidine (PCP)."

  "... preparations containing dextromethorphan are legal to purchase
  from most pharmacies worldwide ... [it] was specifically excluded
  from the Single Convention on Narcotic Drugs.".

In other words, it's a lightweight, more accessible, easier-to-get-into
'bridge' to PCP. Ahem. :)


<Prev in Thread] Current Thread [Next in Thread>
  • New open-source Java library to collect PCP metrics, Paul Cowan <=