Hi Nathan,
On 01/28/2016 03:21 AM, Nathan Scott wrote:
Hi Hemant,
----- Original Message -----
Hi All,
Getting the maximum memory bandwidth per node on intel machines isn't a
trivial problem. This patch is to initiate a discussion as to how to get
this metric value.
[...]
Solutions/Alternatives :
Due to the above listed issues, can we switch to an alternative, where
the client/user can configure the max bandwidth per node similar to what
this patch does? Or, are there any other alternatives which will help in
solving this problem.
[...]
What this patch does ? :
This patch is not a solution to finding the memory bandwidth, rather an
alternative. This takes help of a bandwidth.conf file which can be filled
up by an user/client according to their system's configuration in the format
"node:bandwidth". The linux pmda then reads this file and displays the metric
"hinv.node.max_memory_bandwidth" in a per node manner.
Hmm, yes, as you say - this leaves the open question for the user to know
what the right memory bandwidth value to put in the config file should be
for each node.
Right.
Another approach might be to run some code to attempt to saturate memory
bandwidth for a short time (perhaps using set_mempolicy(2) and cycling
through each node), then measure/export the max observed bandwidth? (or
at least, use that to set initial per-node values in the config file).
Hmm, its a good idea, we can use "stream" to do that.
But again, the theoretical max bandwidth will be something which
will be difficult to saturate, even for the benchmark tools (like
"stream"). Also, depending on the no. of nodes, it will take time to
cycle, saturate and collect the observed max bandwidth.
Either way, this is a fairly specialized domain so I guess it may be more
suited to a specialized PMDA rather than pmdalinux itself. The ./Install
script might prove a good place to run the memory test and populate those
initial config file values?
Agreed. Will add a more specific pmda to find the max memory
bandwidth metric.
Can we put a check for the user if they want to proceed with the
memory test (for the initialization of the config file) inside the Install
script? That way, the user will have the choice to either populate
the config file by themselves or by running some saturation tool
(and hence, the user is aware of the risks).
--
Thanks,
Hemant Kumar
|