On Thu, Oct 15, 2009 at 09:54:56AM +1100, Nathan Scott wrote:
> > but are they shipping the version with Max's change included?
> > (was that v1.4?). Without that, the IB PMDA wont work, or did we
> > figure out a workaround for that? In any case, it's complicated
> > because some of the already released Red Hat and SuSE products will
> > not have the right version of the library, or wont have it at all.
> > > would like to see a Build-dep added to the specfile to get these
> > PMDAs
> > > to build, but I don't like adding another dependency to the core
> > pcp
> > > RPM.
> > >
> > > How should this be handled? Should we just stick the ib PMDA into
> > a
> > > separate sub-package, like pcp-infiniband?
> > yeah that'll work. I can pull together the RPM spec changes to make
> > this happen fairly easily (and I assume Nathan can ditto for deb).
> I'm not 100% sure about this approach - another alternative which I
> think would be cleaner would be to have a separate source tree for
> Infiniband (obviously, with its own packaging, etc).
> I think for the complex PMDAs with complex dependencies, this lets
> us keep the core PCP sources cleaner and lets us release this piece
> independently. This is something that it would be good to have an
> oss example of, cos its what companies that embrace PCP need to do
> for their own custom PMDAs (we do this here for our custom apps,
> for example), so a maintained example of doing a PMDA & its configs
> and any custom tools like this would be generally beneficial IMO.
> IIRC, theres other examples @sgi like this as well - XVM springs to
> mind, I think theres others too.
> It would be a bit along the lines of pcp-gui, in some respects,
> although less C++ish and more Cish, obviously. The builddefs and
> other build changes made pre-3.0.0 mean the infiniband build system
> can just use the installed builddefs, and not have to worry about
> most of the complexity in the PCP build.
> > >
> > > The dependencies for the cluster PMDA are only really build-time.
> Did you mean run-time? I just tweaked the makefile and built the
> cluster PMDA code here, it seems to have no build deps on Infiniband.
No. Build-time. I pushed out a change to my pcp tree that works around
this. the cluster PMDA needs the IB domain number, which it gets from
src/pmdas/ib/domain.h. I just made it so domain.h is still built, even
if we don't have the IB libraries installed.
I also verified what happens when the IB PMDA fails to DSO load, and
what happens when the ib pmda doesn't exist. In both cases the cluster
PMDA starts fairly cleanly. When there is no IB PMDA installed we get:
__pmConnectLocal: Warning: cannot find DSO "/var/lib/pcp/pmdas/ib/pmda_ib.so"
I'd be okay with splitting the infiniband package out separately. It
certainly would keep the specfile for the base PCP a lot cleaner.
> > > you run the cluster PMDA you'll currently end up with a log message
> > > about failing to open the local context to the ib pmda (or
> > something
> > > else like that...I'll have to check this out). If I'm wrong, then
> > this
> > > is what *should* happen. :)
> > can we some how manage the dependency between the cluster PMDA and
> > the
> > IB PMDA, or should the cluster PMDA go into it's own sub-package too?
> Seems to me we could build & install the cluster PMDA now, as long
> as it handles missing infiniband data correctly? Theres long been
> talk of making this more generic too, so it seems this is something
> that should stay in core PCP.
I agree, and it seems to be nearly there now. I'm able to run it on my
desktop with "pmclusterd -h localhost". It needs some enhancements to
be really great, but I think SGI might prioritize that work in the next