Exposing Control Points in a Charm++ Program
- 28 . 2 Linking With The Control Point Framework
- 28 . 3 Runtime Command Line Arguments
Warning: this is still an experimental feature not meant for production applications
The control point framework initializes itself, so no changes need to be made at startup in the program. The program will request the values for each control point on PE 0. Control point values are non-negative integers:
To specify information about the effects of each control point, make calls such as these once on PE 0 before accessing any control point values:
my_var = controlPoint("any_name", 5, 10); my_var2 = controlPoint("another_name", 100,101);
ControlPoint::EffectDecrease::Granularity("num_chare_rows"); ControlPoint::EffectDecrease::Granularity("num_chare_cols"); ControlPoint::EffectIncrease::NumMessages("num_chare_rows"); ControlPoint::EffectIncrease::NumMessages("num_chare_cols"); ControlPoint::EffectDecrease::MessageSizes("num_chare_rows"); ControlPoint::EffectDecrease::MessageSizes("num_chare_cols"); ControlPoint::EffectIncrease::Concurrency("num_chare_rows"); ControlPoint::EffectIncrease::Concurrency("num_chare_cols"); ControlPoint::EffectIncrease::NumComputeObjects("num_chare_rows"); ControlPoint::EffectIncrease::NumComputeObjects("num_chare_cols");
For a complete list of these functions, see
The program, of course, has to adapt its behavior to use these new control point values. There are two ways for a the control point values to change over time. The program can request that a new phase (with its own control point values) be used whenever it wants, or the control point framework can automatically advance to a new phase periodically. The structure of the program will be slightly different in these to cases. Sections 28.1.1 and 28.1.2 describe the additional changes to the program that should be made for each case.
The program provides a callback to the control point framework in a manner such as this:
// Once early on in program, create a callback, and register it CkCallback cb(CkIndex_Main::granularityChange(NULL),thisProxy); registerCPChangeCallback(cb, true);
In the callback or after the callback has executed, the program should request the new control point values on PE 0, and adapt its behavior appropriately.
Alternatively, the program can specify that it wants to call
itself when it is ready. Perhaps the program wishes to delay its adaptation for a while. To do this, it specifies
as the final parameter to
registerControlPointTiming(duration); // called after each program iteration on PE 0 gotoNextPhase(); // called after some number of iterations on PE 0 // Then request new control point values
The control point tuning framework is now an integral part of the Charm++ runtime system. It does not need to be linked in to an application in any special way. It contains the framework code responsible for recording information about the running program as well as adjust the control point values. The trace module will enable measurements to be gathered including information about utilization, idle time, and memory usage.
Various following command line arguments will affect the behavior of the program when running with the control point framework. As this is an experimental framework, these are subject to change.
The scheme used for tuning can be selected at runtime by the use of one of the following options:
+CPSchemeRandom Randomly Select Control Point Values +CPExhaustiveSearch Exhaustive Search of Control Point Values +CPSimulAnneal Simulated Annealing Search of Control Point Values +CPCriticalPathPrio Use Critical Path to adapt Control Point Values +CPBestKnown Use BestKnown Timing for Control Point Values +CPSteering Use Steering to adjust Control Point Values +CPMemoryAware Adjust control points to approach available memory
To intelligently tune or steer an application's performance, performance measurements ought to be used. Some of the schemes above require that memory footprint statistics and utilization statistics be gathered. All measurements are performed by a tracing module that has some overheads, so is not enabled by default. To use any type of measurement based steering scheme, it is necessary to add a runtime command line argument to the user program to enable the tracing module:
The following flags enable the gathering of the different types of available measurements.
+CPGatherAll Gather all types of measurements for each phase +CPGatherMemoryUsage Gather memory usage after each phase +CPGatherUtilization Gather utilization & Idle time after each phase
The control point framework will periodically adapt the control point values. The following command line flag determines the frequency at which the control point framework will attempt to adjust things.
+CPSamplePeriod number The time between Control Point Framework samples (in seconds)
The data from one run of the program can be saved and used in subsequent runs. The following command line arguments specify that a file named
should be created or loaded. This file contains measurements for each phase as well as the control point values used in each phase.
+CPSaveData Save Control Point timings & configurations at completion +CPLoadData Load Control Point timings & configurations at startup +CPDataFilename Specify the data filename
It might be useful for example, to run once with
to try to find a good configuration for the program, and then use
for all subsequent production runs.