Bug #1404

Support Cray CC on {mpi,gni}-crayxe

Added by Sam White over 2 years ago. Updated over 2 years ago.

Build & Test Automation
Target version:
Start date:
Due date:
% Done:



Currently, Charm will build on CCE 8.5.4+. However, to do so you have to remove the check in our top-level build script that rejects any explicitly chosen compilers on Cray targets, and you have to specify craycc, as in './build charm++ gni-crayxc craycc'.

Remove the need for specifying craycc explicitly, so that all compilers are implicit in the modules loaded when building on Crays.

Also the conv-mach files for gni-crayxc were the only ones modified by Cray's compiler team when they contributed that patch to Charm, so make the same changes to gni-crayxe and mpi cray builds if necessary.

Both Edison and Blue Waters have CCE 8.5.5+ now, though they are not the default for PrgEnv-cray yet.

Related issues

Related to Charm++ - Bug #1149: Cray CC builds are broken Merged 07/22/2016


#1 Updated by Phil Miller over 2 years ago

On Edison with mainline charm, commit v6.7.0-643-g2429278, I was able to successfully run

./build charm++ gni-crayxc

with modules as follows:

Currently Loaded Modulefiles:
  1) modules/                       9) python_base/2.7.9                     17) craype-ivybridge                      25) udreg/2.3.2-1.0502.9889.2.20.ari      33) rca/1.0.0-2.0502.57212.2.56.ari
  2) nsg/1.2.0                             10) numpy/1.9.2                           18) craype/2.5.5                          26) ugni/6.0-1.0502.10245.9.9.ari         34) atp/2.0.2
  3) eswrap/1.3.3-1.020200.1278.0          11) scipy/0.15.1                          19) altd/2.0                              27) pmi/5.0.10-1.0000.11050.0.0.ari       35) PrgEnv-cray/5.2.56
  4) switch/1.0-1.0502.57058.1.58.ari      12) matplotlib/1.4.3                      20) darshan/2.3.1                         28) dmapp/7.0.1-1.0502.10246.8.47.ari     36) gcc/6.1.0
  5) cray-shmem/7.4.1                      13) ipython/3.1.0                         21) cray-hdf5-parallel/1.8.16             29) gni-headers/4.0-1.0502.10317.9.2.ari
  6) cray-mpich/7.4.1                      14) python/2.7.9                          22) git/2.4.6                             30) xpmem/0.1-2.0502.57015.1.15.ari
  7) slurm/edison                          15) craype-network-aries                  23) cce/8.5.6                             31) dvs/2.5_0.9.0-1.0502.1958.2.55.ari
  8) papi/                          16) craype-hugepages8M                    24) cray-libsci/16.07.1                   32) alps/5.2.3-2.0502.9295.14.14.ari

That's CCE 8.5.6 and PrgEnv-cray.

No changes to build, no use of

#2 Updated by Sam White over 2 years ago

I think I ran into that issue on Blue Waters and then assumed it was the same on Edison, but it may have been some problem with my modules.

#3 Updated by Phil Miller over 2 years ago

Same commit builds with the same command using cce/8.5.4 and cce/8.5.7 on Cori.

#4 Updated by Phil Miller over 2 years ago

It sounds like the issue here is much more that the recent Cray-provided changes weren't ported to gni-crayxe and mpi-crayx[ce] than anything else.

Given that what's expected to be working (CCE for gni-craycx) is actually working, I'd suggest moving this off to 6.8.0 and retitling/describing it appropriately.

#5 Updated by Sam White over 2 years ago

  • Target version changed from 6.8.0-beta1 to 6.8.0
  • Subject changed from Fix Cray CC build issues to Support Cray CC on gni-crayxe and mpi-crayx[ce]


#6 Updated by Sam White over 2 years ago

  • Assignee set to Sam White
  • Subject changed from Support Cray CC on gni-crayxe and mpi-crayx[ce] to Support Cray CC on {mpi,gni}-crayxe

The original patch from Cray already includes support for mpi-crayxc.

#7 Updated by Sam White over 2 years ago

  • Status changed from New to Implemented

Fix for mpi- and gni- crayxe is here:

Works on Blue Waters with the following modules loaded:

Currently Loaded Modulefiles:
  1) modules/                           11) gni-headers/4.0-1.0502.10859.7.8.gem       21) moab/9.0.2.TAS2-1477409647_6706986-sles11
  2) eswrap/1.3.3-1.020200.1278.0               12) xpmem/0.1-2.0502.64982.5.3.gem             22) java/jdk1.8.0_51
  3) cce/8.5.5                                  13) dvs/2.5_0.9.0-1.0502.2188.1.113.gem        23) globus/5.2.5
  4) craype-network-gemini                      14) alps/5.2.4-2.0502.9774.31.12.gem           24) gsissh/6.2p2
  5) craype/2.5.4                               15) rca/1.0.0-2.0502.60530.1.63.gem            25) darshan/
  6) cray-libsci/16.03.1                        16) atp/2.0.1                                  26) scripts
  7) udreg/2.3.2-1.0502.10518.2.17.gem          17) PrgEnv-cray/5.2.82                         27) user-paths
  8) ugni/6.0-1.0502.10863.8.28.gem             18) cray-mpich/7.3.3                           28) xalt/0.7.6
  9) pmi/5.0.10-1.0000.11050.179.3.gem          19) craype-interlagos                          29) craype-hugepages8M
 10) dmapp/7.0.1-1.0502.11080.8.74.gem          20) torque/6.0.1

We get an error when building ROMIO on AMPI, but our IO support is poor in AMPI anyways so building with --without-romio is acceptable for now:

CC-1398 craycc: ERROR File = subarrayf.c, Line = 39
  The weak symbol and the strong symbol cannot have the same name.
  #pragma weak mpi_type_create_subarray_ = pmpi_type_create_subarray_

#8 Updated by Sam White over 2 years ago

  • translation missing: en.field_closed_date set to 2017-02-16 15:43:40.593041
  • Status changed from Implemented to Merged

Also available in: Atom PDF