I am calling things sequentially. I do suspect that a multithreaded
approach would help. What language did you write your tools in, and could
I see an example?
Thanks a lot.
Alan
On Tue, 9 Jan 2001, Troy Dawson wrote:
> Howdy,
> How are you calling this? Is it a script that goes one at a time, I'd really
> say it's your script, but I could be wrong.
>
> Basically I've found that pcp is quite fast beyond anything else I've tried.
> Just for your info, I collect dozens of stuff from over 250 machines, and the
> only slowness I experience is from my scripts that gather things (ie ... I
> intentially put a pause in my collections so I don't overwhelm anything).
>
> One thing might be that the time to get alot of data, and the time to get just
> one metric, is about the same. So for me the time/data ratio is quite high,
> because I have alot of data. And for you, the time/data ratio is quite low
> because you are only getting one metric. If that makes sense.
>
> I've also found that doing multiple threads, and having them all get the info
> at the same time, speeds things up immensly. Instead of doing just one at a
> time.
>
> Troy
>
> Alan Bailey wrote:
> >
> > I'm wondering what timing tests were used when developing PCP. I don't
> > know if this could be answered by the open source developers, or would
> > need to be redirected to the main developers. Basically, one of my
> > coworkers believes that the response time for PCP is pretty slow, and
> > believes that a better solution for sending data between hosts could be
> > made and would be significantly better. [1]
> >
> > Right now it takes almost exactly 4 seconds to call pminfo on 87 hosts for
> > mem.freemem. I think this is very good. We will be expanding to a few
> > hundred hosts in the near future though. So basically, was any optimizing
> > done in regards to speed when designing PCP? If so, is it possible to see
> > any kind of tests or decisions that were made to show that PCP is truly
> > optimized for speed?
> >
> > Thanks,
> > Alan
> >
> > [1] - One of the ideas he had was to use pcp to just collect data on the
> > localhost and store it in some structure. The query mechanism between
> > hosts would then ask each individual host for data, and this cached data
> > would be returned. This is obviously very similar to a caching PMDA,
> > which might be an option, but I forgot to mention it to him. He was also
> > thinking of using UDP, a hierarchical grabbing scheme, and some other
> > techniques for getting data. I don't think it would help much, networks
> > can only go so fast ;)
>
> --
> __________________________________________________
> Troy Dawson dawson@xxxxxxxx (630)840-6468
> Fermilab ComputingDivision/OSS SCS Group
> __________________________________________________
>
--
Alan Bailey
|