Build target(s) for Intel MIC
We should incorporate the work done to build on MIC with the main codebase. If it's easily backported, it should go on the stable branch as well.
#1 Updated by Eric Bohm about 6 years ago
- Assignee changed from Ronak Buch to Eric Bohm
The initial issues here stem from subtle interactions between the Intel cross compiler scheme for MIC (aka Intel Xeon Phi, KNC, Knight's Corner, plus whatever the, arguably overly fecund, brand name developers from Intel come up with next as a name for these things) and Charm++ compiler wrapper settings.
All libraries which a MIC application binary must link to must go through the cross compiler (icc -mmic, or a more gnarly explicit path call to x86_64-k1om-linux-g++, which should probably be avoided as we have no guarantee about name stability for that thing) and its linker.
All binaries which would be executed on the host (usually charmc and various configure tests) should usually not go through the cross compiler. In the past, we have generally used CMK_NATIVE (aka charmc -seq) for these. However, -seq has crept into a bunch of things which must be linked to by application binaries, such as libckqt.a. So we now have some semantic confusion over what -seq means in this context and how it should be configured.
In the general case, on machines like Stampede, the mic0 host is not always directly accessible, so various tools like charmrun, should not be compiled with -mmic. On experimental machines, like NCSA's Xeon Phi, root access to mic0 is available and they might wish to use charmrun natively compiled for the MIC. IMHO, anyone doing that should be expected to solve their own wacky compilation and linking issues. Our main audience is people trying to leverage Phi on Stampede.
On a related note, the MIC environment has its own issues. Notably, there is no MIC ibverbs library to link to on stampede, so you cannot even attempt to use their verbs proxy within the MIC context.
One can build using intel MPI with one binary for host and one for MIC and use ibrun.symm to have PEs running on both host and MIC. The process currently requires some somewhat arduous hackery. That process will be covered in a different issue.
A proper native layer should be able to use the dual binary launch scheme of ibrun.symm, but the necessary underpinning is not yet developed.
#4 Updated by Michael Robson almost 4 years ago
Just saving some useful changes from knc3 which is being decommissioned today:
# renamed: src/arch/mpi-linux-x86_64/conv-mach-mic.h -> src/arch/common/conv-mach-mic.h # renamed: src/arch/mpi-linux-x86_64/conv-mach-mic.sh -> src/arch/common/conv-mach-mic.sh
#8 Updated by Sam White over 2 years ago
- Category set to Machine Layers
- Assignee changed from Michael Robson to PPL
- Target version changed from 6.8.0 to Unscheduled
KNL is self-hosted and doesn't require the complicated dual binary launch scheme. Since this is off the radar for now, but something similar could appear in the future, reassigning to PPL for now.