CkArrayOptions callback for completion of chare array initialization
Add a callback to CkArrayOptions to be triggered when all of the initial objects are constructed in a chare array.
Right now we don't have a good way to do this short of QD, which is global.
AMPI's intercommunicator creation is an example where this is needed, and where QD is not possible (since intercommunicators are created dynamically and other messages may be in flight on other communicators during creation of an intercommunicator).
#3 Updated by Phil Miller over 2 years ago
I'm not sure of what you mean by 'all types of chare array creation' - I think we only need to be concerned with bulk construction, which is done when the code returns from the populateInitial call on each PE. The array manager can contribute to a reduction targeting the callback.
#4 Updated by Sam White over 2 years ago
AMPI's use case for this (intercomm creation) is one-at-a-time/element-by-element initial insertion. Perhaps we should refactor AMPI's array creation to use bulk insertion though...
One issue I ran into when initially trying to implement this was that the contribute() methods are not defined in CkArray.
#7 Updated by Sam White over 2 years ago
- Status changed from New to In Progress
Hacky implementation using pt2pt sends, rather than a reduction, here: https://charm.cs.illinois.edu/gerrit/#/c/2209/
I tried adding contribute() definitions/declarations to CkReductionMgr, which CkArray inherits from. But then when running with more chare array elements than PEs, we get a hang. The reduction mgr for some reason is expecting more contributions than it should...