Project

General

Profile

Feature #1133

Feature #944: AMPI compliance with MPI-2.2 standard

PMPI_ interface for AMPI

Added by Sam White over 2 years ago. Updated 5 months ago.

Status:
Merged
Priority:
Normal
Category:
AMPI
Target version:
Start date:
07/01/2016
Due date:
% Done:

100%


Description

See section 14.2.6 of the MPI-3.1 standard here: https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf

History

#1 Updated by Sam White over 2 years ago

  • Assignee deleted (Edward Hutter)

Full support for the PMPI_ interface in AMPI would require some invasive changes to AMPI:
1. Define each MPI_ routine in its own file. This would mean breaking up ampi.C, ampiOneSided.C, and ampiMisc.C into >300 files. This is necessitated by the non-portability of weak symbols and the fact that users/libs can choose to intercept only some of their MPI_ routines with PMPI_ wrappers and to treat the others as normal MPI_ routines.
2. Instead of using AMPI_ internally and using the preprocessor to rename MPI_, we would need to define everything in AMPI to MPI_ in order to use weak symbols. This would break AMPI's ability to build on the MPI net-layer.

We could potentially support the PMPI_ interface ONLY on platforms supporting weak symbols AND on non-MPI layer builds (multicore, netlrts, verbs, gni, pamilrts), but this needs more thought... In the end, virtualization will break probably all existing profiling libraries too. It might be worth maintaining this in a branch of AMPI, but not in production unless we want to do it properly with full portability + support.

#2 Updated by Sam White over 2 years ago

  • Priority changed from Normal to Low

#3 Updated by Sam White almost 2 years ago

  • Target version changed from 6.8.0 to 6.8.1

#4 Updated by Sam White almost 2 years ago

  • Priority changed from Low to Normal
  • Assignee set to Matthias Diener

I started the implementation for weak symbols and non-MPI communication layers (#2 above) on a branch named 'ampi_with_pmpi' a couple months ago, but it is incomplete.

#5 Updated by Matthias Diener over 1 year ago

  • Target version changed from 6.8.1 to 6.9.0

#6 Updated by Matthias Diener about 1 year ago

My current thinking is to implement something like this (using MPI_Send as an example):

In ampi.h:

#define PMPI_Send AMPI_Send // Support PMPI function
int AMPI_Send(const void *msg, int count, MPI_Datatype type, int dest,
              int tag, MPI_Comm comm); // Actually implemented function
int MPI_Send(const void *msg, int count, MPI_Datatype type, int dest,
              int tag, MPI_Comm comm); // Same definition as before, to prevent compiler warnings and provide type checking for arguments

In ampi.C:

#pragma weak MPI_Send = AMPI_Send // Provide a weak symbol 'MPI_Send' as an alias to 'AMPI_Send'
int AMPI_Send(const void *msg, int count, MPI_Datatype type, int dest, int tag, MPI_Comm comm) {...} // Implementation as before

I have a proof of concept implementation of this that seems to work fine and supports PMPI correctly, even on an MPI machine layer.

Some things to check/discuss:
  • Is this dependent on library ordering on MPI layers?
  • Are there systems without weak symbol support?

#7 Updated by Sam White about 1 year ago

It makes sense to me. If you already have a small test working then I'd say go ahead. Even if it didn't work on the MPI layer it would still be valuable, and I think all compilers that we care about support weak symbols.

#8 Updated by Matthias Diener about 1 year ago

  • Status changed from New to Implemented

The patch is here: https://charm.cs.illinois.edu/gerrit/#/c/2544/

Linux support:
Works on {netlrts,mpi}-linux-x86_64. Tested on CentOS6, CentOS7, Ubuntu17.10 with their default compilers.

Darwin support:
No support. I think weak symbol aliases are not supported on Darwin. MPI applications work as before, but cannot use PMPI.

Windows support:
Untested. Doesn't build at the moment, even without the patch above. As the Win* targets use PE32 as file format for executables (not ELF), it is unlikely that PMPI is supported on Windows.

Note that the actual implementation is slightly different from the one pointed out above.

In ampi.h:

#define MPI_Send AMPI_Send
#define PMPI_Send APMPI_Send // Rename MPI* and PMPI* functions in user code
int AMPI_Send(...); // Actually implemented function
int APMPI_Send(...); // Same definition as before, to prevent compiler warnings and provide type checking for arguments

In ampi.C:

#pragma weak AMPI_Send // Weak symbol, such that it can be overridden in user code 
#pragma weak APMPI_Send = AMPI_Send // Weak APMPI* symbol, such that user code can call our implementation directly
int AMPI_Send(...) // Actual implementation 

#9 Updated by Sam White 12 months ago

  • Status changed from Implemented to Merged

#10 Updated by Sam White 5 months ago

  • % Done changed from 0 to 100

Also available in: Atom PDF