Reduce CkReductionMsg envelope size
CkReductionMsg has accumulated fields that are often unused. Here are its fields that are unconditionally compiled:
int dataSize; //Length of array below, in bytes void *data; //Reduction data CMK_REFNUM_TYPE userFlag; //Some sort of identifying flag, for client use CkCallback callback; //What to do when done CkCallback secondaryCallback; // the group callback is piggybacked on the nodegrp reduction bool migratableContributor; // are the contributors migratable int sourceFlag; int fromPE; CkReduction::reducerType reducer; int redNo; //The serial number of this reduction int gcount; //Contribution to the global contributor count // for section multicast/reduction library CkSectionInfo sid; // section cookie for multicast char rebuilt; // indicate if the multicast tree needs rebuilt int nFrags; int fragNo; // fragment of a reduction msg (when pipelined) double dataStorage; //Start of data array (so it's double-aligned) int no;Some initial observations:
- 'secondaryCallback' is already being removed here: https://charm.cs.illinois.edu/gerrit/#/c/2216/
- 'no' is completely unused, as far as I can tell.
- 'dataStorage' exists only for alignment purposes. Is there another way to get the alignment without always adding 8 bytes?
- The CkSectionInfo is itself 32 bytes, add the other 3 fields for sections and you get 39 bytes in every CkReductionMsg that are only used if that message is for a section.
#6 Updated by Sam White over 1 year ago
This commit reduces the size of CkReductionMsg's from 168 to 136 Bytes and the size of CkCallback's from 72 to 64 Bytes: https://charm.cs.illinois.edu/gerrit/#/c/3866/
This reduces the size of Charm's message envelope from 80 Bytes to 64 Bytes: https://charm.cs.illinois.edu/gerrit/#/c/charm/+/3872/
You can see the size and packing of objects via "clang -cc1 -fdump-record-layouts". The bulk of CkReductionMsg's size is still due to Section support, both in CkReduction's section-related members and in the CkCallback, which is itself bloated by Section support. But I think we can safely mark this 'done' for now.