Project

General

Profile

Bug #1770

when cross-compiling for KNL, charmc -host uses -xMIC-AVX512 command-line argument to build charmxi

Added by Jim Phillips over 1 year ago. Updated over 1 year ago.

Status:
Merged
Priority:
High
Assignee:
Category:
Machine Layers
Target version:
Start date:
01/08/2018
Due date:
% Done:

0%

Spent time:

Description

Related to https://charm.cs.illinois.edu/redmine/issues/1745 as fixed by https://charm.cs.illinois.edu/gerrit/c/3339/

The following results in a charmxi binary that fails to run on the (non-KNL) host:

./build charm++ multicore-linux64 iccstatic --suffix=knl -xMIC-AVX512

The actual command executed by charmc -host is, e.g.:

icpc -DCMK_GFORTRAN -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512 -D_REENTRANT -I../bin/../include -D__CHARMC__=1 -I../../src/xlat-i/ -I../../src/xlat-i/sdag/ -I. -c ../../src/xlat-i/xi-AstNode.C -o xi-AstNode.o

Later charmc is called with -xMIC-AVX512 passed explicitly:

../bin/charmc  -xMIC-AVX512  -I.   -c -o DummyLB.o DummyLB.C

I'm not sure where it is sneaking in. This is everywhere AVX512 appears:

jim@tulsa$grep -r AVX512 .
./hwloc/include/Makefile:HWLOC_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./hwloc/include/Makefile:HWLOC_EMBEDDED_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./hwloc/src/Makefile:HWLOC_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./hwloc/src/Makefile:HWLOC_EMBEDDED_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./hwloc/Makefile:HWLOC_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./hwloc/Makefile:HWLOC_EMBEDDED_CFLAGS =  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  
./config.log:HWLOC_CFLAGS=' -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  '
./config.log:HWLOC_EMBEDDED_CFLAGS=' -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  '
./config.status:S["HWLOC_EMBEDDED_CFLAGS"]=" -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  " 
./config.status:S["HWLOC_CFLAGS"]=" -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  " 
Binary file ./testlink matches
./buildopts.mk:OPTSATBUILDTIME +=  -xMIC-AVX512
./conv-mach-opt.sh:BUILDOPTS=" -xMIC-AVX512" 
./conv-mach-opt.sh:CMK_CC_FLAGS="$CMK_CC_FLAGS  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  "   # Option added by configure script's hwloc section
./conv-mach-opt.sh:CMK_CXX_FLAGS="$CMK_CXX_FLAGS  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512  "   # Option added by configure script's hwloc section

History

#1 Updated by Jim Phillips over 1 year ago

OK, this seems to be the critical difference between the MPI build (which works) and the multicore build (which doesn't):

< checking machine name... mpi-linux-x86_64-smp-iccstatic
---
> checking machine name... multicore-linux64-iccstatic
120c120
< checking "C++ compiler as"... "icpc     -I/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/../mpi/int
el64/include" 
---
> checking "C++ compiler as"... "icpc   " 
122c122
< checking "C++ linker as"... "icpc -static-intel    -L/opt/intel/compilers_and_libraries_2018.0.128/linux/mkl/../mpi/intel64/lib" 
---
> checking "C++ linker as"... "icpc -static-intel  " 
124c124
< checking "Native C++ compiler as"... "g++" 
---
> checking "Native C++ compiler as"... "icpc " 

It looks like MPI builds explicitly set CMK_NATIVE_CXX to g++ while many non-MPI builds don't:

./mpi-linux-x86_64/conv-mach.sh:CMK_NATIVE_CXX='g++'

This still doesn't explain why the target build options are being applied to host binaries.

#2 Updated by Jim Phillips over 1 year ago

This is where the AVX512 option is showing up in conv-mach-opt.sh:

CMK_CC_FLAGS="$CMK_CC_FLAGS  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512 -O2 -U_FORTIFY_SOURCE  "   # Option added by configure script's hwloc section

CMK_CXX_FLAGS="$CMK_CXX_FLAGS  -DCMK_GFORTRAN -D_REENTRANT -I./../include -D__CHARMC__=1 -xMIC-AVX512 -O2 -U_FORTIFY_SOURCE  "   # Option added by configure script's hwloc section

#3 Updated by Evan Ramos over 1 year ago

  • Assignee set to Evan Ramos

#5 Updated by Sam White over 1 year ago

  • Assignee changed from Evan Ramos to Jim Phillips
  • Status changed from New to Merged

Also available in: Atom PDF