pcp
[Top] [All Lists]

Re: pcp+graphite, take 2

To: pcp@xxxxxxxxxxx
Subject: Re: pcp+graphite, take 2
From: fche@xxxxxxxxxx (Frank Ch. Eigler)
Date: Wed, 18 Jun 2014 18:39:35 -0400
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <y0m61jyw965.fsf@xxxxxxxx> (Frank Ch. Eigler's message of "Wed, 18 Jun 2014 11:04:50 -0400")
References: <20140616214301.GB6693@xxxxxxxxxx> <53A181C4.5050402@xxxxxxxxxx> <y0m61jyw965.fsf@xxxxxxxx>
User-agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
I wrote:

> By the way, pmwebd would be a natural candidate for internal
> multithreading, at the http request level and/or at the per-archive
> data-extraction level.  I wonder if I should throw caution [1] to the
> wind and turn it on.

Done & some early results are interesting.  pmwebd -M<N> for N>0
results in some happy operations, but too brief to get a good sense of
speedups.  That's because we get crashes pretty quickly.

% valgrind .../pmwebd ... -M1
==21880== Memcheck, a memory error detector
==21880== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==21880== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==21880== Command: ./pmwebd -G -R jsdemos -A .../path -v -v -M1
==21880== 
[Wed Jun 18 22:34:15] pmwebd(21880): pmwebd PID = 21880 PMAPI URL = pmapi
     Started daemon on IPv4 TCP port 44323
     Using libmicrohttpd 0.9.33
     Serving non-pmwebapi URLs under directory .../jsdemos
     Remote context creation requests enabled
     Archive base directory: .../path
     Graphite API enabled with Cairo graphics
     Periodic client statistics dumped roughly every 300s
     Using up to 1 auxiliary threads
[Wed Jun 18 22:34:16] pmwebd(21880): [127.0.0.1:57296] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:17] pmwebd(21880): [127.0.0.1:57296] digested 16 metrics, 
timespan [1403116440-1403126340 by 60], in 256.715ms 
[Wed Jun 18 22:34:17] pmwebd(21880): [127.0.0.1:57297] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:19] pmwebd(21880): [127.0.0.1:57297] digested 64 metrics, 
timespan [1403116440-1403126340 by 60], in 909.521ms 
[Wed Jun 18 22:34:19] pmwebd(21880): [127.0.0.1:57298] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:20] pmwebd(21880): [127.0.0.1:57298] digested 32 metrics, 
timespan [1403116440-1403126340 by 60], in 389.512ms 
[Wed Jun 18 22:34:23] pmwebd(21880): [127.0.0.1:57297] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:23] pmwebd(21880): [127.0.0.1:57297] digested 16 metrics, 
timespan [1403116440-1403126340 by 60], in 180.102ms 
[Wed Jun 18 22:34:23] pmwebd(21880): [127.0.0.1:57298] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:25] pmwebd(21880): [127.0.0.1:57298] digested 64 metrics, 
timespan [1403116440-1403126340 by 60], in 823.622ms 
[Wed Jun 18 22:34:25] pmwebd(21880): [127.0.0.1:57296] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:26] pmwebd(21880): [127.0.0.1:57296] digested 32 metrics, 
timespan [1403116440-1403126340 by 60], in 389.781ms 
[Wed Jun 18 22:34:29] pmwebd(21880): [127.0.0.1:57297] HTTP/1.1 POST 
/graphite/render
[Wed Jun 18 22:34:30] pmwebd(21880): [127.0.0.1:57297] digested 16 metrics, 
timespan [1403116440-1403126340 by 60], in 187.731ms 
[Wed Jun 18 22:34:30] pmwebd(21880): [127.0.0.1:57298] HTTP/1.1 POST 
/graphite/render
==21880== Invalid read of size 8
==21880==    at 0x4E609C0: update_bounds.isra.1 (interp.c:411)
==21880==    by 0x4E63103: __pmLogFetchInterp (interp.c:916)
==21880==    by 0x4E5E43E: __pmLogFetch (logutil.c:1745)
==21880==    by 0x4E45B65: pmFetch (fetch.c:151)
==21880==    by 0x120A41: pmgraphite_fetch_series(fetch_series_jobspec*) 
(pmgraphite.cxx:872)
==21880==    by 0x129507: 
fetch_series_jobqueue<fetch_series_jobspec>::thread_main(void*) 
(pmgraphite.cxx:638)
==21880==    by 0x12C86C: fetch_series_jobqueue<fetch_series_jobspec>::run() 
(pmgraphite.cxx:659)
==21880==    by 0x11A22C: pmgraphite_fetch_all_series(MHD_Connection*, 
std::vector<std::string, std::allocator<std::string> > const&, long, long, 
long) (pmgraphite.cxx:972)
==21880==    by 0x12618F: pmgraphite_respond_render_json(MHD_Connection*, 
http_params const&, std::vector<std::string, std::allocator<std::string> > 
const&, bool) (pmgraphite.cxx:2018)
==21880==    by 0x1291C2: pmgraphite_respond(MHD_Connection*, http_params 
const&, std::vector<std::string, std::allocator<std::string> > const&) 
(pmgraphite.cxx:2126)
==21880==    by 0x1105AF: mhd_respond(void*, MHD_Connection*, char const*, char 
const*, char const*, char const*, unsigned long*, void**) (main.cxx:238)
==21880==    by 0x50AB528: call_connection_handler (connection.c:1295)
==21880==  Address 0x12657278 is 744 bytes inside a block of size 1,752 free'd
==21880==    at 0x4C294C4: free (in 
/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==21880==    by 0x4E603D5: cache_read.isra.0 (interp.c:191)
==21880==    by 0x4E62E43: __pmLogFetchInterp (interp.c:1013)
==21880==    by 0x4E5E43E: __pmLogFetch (logutil.c:1745)
==21880==    by 0x4E45B65: pmFetch (fetch.c:151)
==21880==    by 0x120A41: pmgraphite_fetch_series(fetch_series_jobspec*) 
(pmgraphite.cxx:872)
==21880==    by 0x129507: 
fetch_series_jobqueue<fetch_series_jobspec>::thread_main(void*) 
(pmgraphite.cxx:638)
==21880==    by 0x668BC52: start_thread (pthread_create.c:308)
==21880==    by 0x63B8DBC: clone (clone.S:113)


It doesn't look too serious; will look into it tomorrow.


- FChE

<Prev in Thread] Current Thread [Next in Thread>