Feature #1497: Shared memory method to pass data between processes that share the same node
Enable use of pxshm/xpmem on mpi, ofi, and verbs builds
pxshm is currently only supported on the GNI and NetLRTS layers. It should be trivial to allow its use on MPI and Verbs (and OFI when it gets merged), to say nothing of its performance.
- Subject changed from Enable use of pxshm on mpi and verbs builds to Enable use of pxshm/xpmem on mpi and verbs builds
Also, './build charm++ gni-crayxe xpmem' fails to build because it tries to build pxshm and xpmem both. The issue is that we build with pxshm by default for gni-crayx* builds and don't disable that when explicitly building with xpmem. From what I've seen, xpmem offers performance nearly on par with user-space memcpy for Cray MPI, so that could potentially become the default on gni builds instead of pxshm if we implement it correctly. The key is to call xpmem_make() on the entire virtual address space during startup, avoiding the high cost of memory registration/deregistration during runtime.