Troy, these are all pretty tough questions. The only short term
solution I can think of is to rebuild the kernel on your logging
machine to handle more file descriptors and/or split the logging
load between several machines and cross mount the archives. You
should also be able to run the loggers on an SGI IRIX system, if
you happen to have one - IRIX generated PCP archives are binary
compatible with those from PCP on Linux.
Other than that, I'm sure you'll raise the issues with Ken when
he gets there - ask him why we don't support multiple hosts
per archive ... that'll get things moving .. ;-)
thanks
-- Mark
On Fri, 4 Aug 2000, Troy Dawson wrote:
> Howdy,
> I just ran into a problem, that might already be fixed (I'm on pcp 2.1.4) but
> I thought I'd bring it up. It concerns pmlogger when you are monitoring lots
> of systems.
> Basically there is a seperate process that runs for each machine that you are
> logging. I'm sure that this makes the gathering of data and such much
> quicker, but it does have a drawback when the number of machines you are
> monitoring gets up high, like several hundred or thousand.
> Basically the problem is this. According to the error message I have, the VFS
> (Virtual File Server) running on Linux can only access a maximum of 4096 files
> at a time. After that the machine basically goes belly up. So if you have
> 250 loggers going, each of them normally open 5 files, you have 1250 files
> open. Now when you do the log rotate, I can't tell for sure, but I believe
> you have a minumum of 10 files open, and possibly 15, the number jumps to 2500
> (for 10 files) plus the original 1250 equals 3750, which is getting awfully
> close to the limit. If it is 15, your already there.
> OK, so you can guess why I'm writting this, yesterday, I added 50 more
> machines to my logger, and at log rotation time, the machine choked. (Just to
> note, it didn't crash, you just couldn't actually do anything useful)
> Anyway, this is a problem that probrubly needs to be looked at.
> Troy
>
|