ReductionStarting overhead in SMP mode
On a benchmark where every PE has array elements, I'm still seeing a lot of
CkReductionMgr::ReductionStarting messages, taking up a large fraction of runtime during that phase. Given the changes a while back to quiet them when they're not needed, I don't think they should appear at all.
#3 Updated by Phil Miller over 2 years ago
In projections traces, I noticed that each contribution was only triggering sends of ReductionStarting messages to other PEs in the same process. So, I tried a non-SMP build to see what would happen there. I saw no ReductionStarting messages at all, and performance was substantially improved.
There is some optimization missing in the node-level array reduction consolidation to match the non-SMP case.
#10 Updated by Sam White over 2 years ago
- Subject changed from SMP builds: Large fraction of time in CG solver taken by repetitive ReductionStarting to ReductionStarting overhead in SMP mode
In Core we discussed the possibility of passing CkArrayOptions through to the reduction managers just like what is already done in CkArrayBroadcaster (called 'stableLocations' there).
#14 Updated by Sam White over 2 years ago
The above commit leaves a TODO behind for the following:
// TODO: This is currently gotten from the command-line...but it could // probably just be determined at runtime. I don't know if the CL option is // even documented anywhere. if(disableNotifyChildrenStart) return;
Documenting here so it is not forgotten