Project

General

Profile

Bug #1325

AMPI programs fail to link with Isomalloc heaps

Added by Sam White almost 2 years ago. Updated 4 months ago.

Status:
Merged
Priority:
Normal
Assignee:
Category:
AMPI
Target version:
Start date:
12/08/2016
Due date:
% Done:

0%

Tags:

Description

This is a known issue for us, but it hurts our testing (we don't actually test Isomalloc on gni-crayxc) and external user's experiences with AMPI.

The solution is to add the link flag '-Wl,--allow-multiple-definition'.
Add that flag to ampicc/charmc when using -memory isomalloc.

History

#1 Updated by Sam White almost 2 years ago

An example of the failed output:

make[3]: Entering directory `/scratch1/scratchdirs/acun/autobuild/gni-crayxc-smp/charm/gni-crayxc-smp/examples/ampi/Cjacobi3D'
../../../bin/ampicxx  -c -DNO_PUP jacobi.C -o jacobi.iso.o
../../../bin/ampicxx  -o jacobi.iso jacobi.iso.o -module CommonLBs -memory isomalloc
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../lib64/libc.a(malloc.o): In function `__libc_free':
/usr/src/packages/BUILD/glibc-2.11.3/malloc/malloc.c:3692: multiple definition of `free'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):libmemory-isomalloc.c:(.text+0x7840): first defined here
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../lib64/libc.a(malloc.o): In function `__libc_malloc':
/usr/src/packages/BUILD/glibc-2.11.3/malloc/malloc.c:3610: multiple definition of `malloc'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):libmemory-isomalloc.c:(.text+0x7410): first defined here
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../lib64/libc.a(malloc.o): In function `__libc_realloc':
/usr/src/packages/BUILD/glibc-2.11.3/malloc/malloc.c:3743: multiple definition of `realloc'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):libmemory-isomalloc.c:(.text+0x7f90): first defined here

#2 Updated by Sam White almost 2 years ago

  • Status changed from New to Implemented

Until we have a real fix for this (see below), link with -Wl,--allow-multiple-definition in tests/examples: https://charm.cs.illinois.edu/gerrit/#/c/2036/

#3 Updated by Sam White almost 2 years ago

  • Status changed from Implemented to In Progress

From Phil's comment on the above patch: If we pass this flag, we have no guarantee which version of the conflicting symbols we're actually getting. The right solution is to figure out what other undefined symbol is pulling in malloc.o from libc.a, and implement that ourself, or avoid referencing it.

From what I can find online, it looks like that other symbol or symbols might be the following: cfree(), pvalloc(), mallinfo(), posix_memalign(), __malloc_fork_lock_parent(), __malloc_fork_unlock_parent(), __malloc_fork_unlock_child().

I took a shot at implementing those in src/conv-core/memory*, but was unsuccessful.

#4 Updated by Sam White almost 2 years ago

Actually abort testing on Isomalloc test failures: https://charm.cs.illinois.edu/gerrit/#/c/2066/

#5 Updated by Sam White almost 2 years ago

  • Target version changed from 6.8.0 to 6.8.1

#6 Updated by Sam White about 1 year ago

  • Target version changed from 6.8.1 to 6.9.0

#7 Updated by Sam White about 1 year ago

  • Target version deleted (6.9.0)
  • Status changed from In Progress to New

#8 Updated by Sam White about 1 year ago

We aren't currently getting Isomalloc tests run on autobuild on Darwin, because the linker there doesn't like -Wl,--allow-multiple-definition. All other autobuild targets work with it, so we should only add that linker option if not on Darwin.

#9 Updated by Sam White 7 months ago

  • Tags set to isomalloc

#10 Updated by Sam White 6 months ago

  • Assignee changed from Sam White to Evan Ramos

#11 Updated by Evan Ramos 6 months ago

$ ../../../bin/ampicxx  -o jacobi.iso jacobi.iso.o -module CommonLBs -memory isomalloc
/usr/lib/../lib64/libc.a(malloc.o): In function `__libc_malloc':
/home/abuild/rpmbuild/BUILD/glibc-2.22/malloc/malloc.c:2900: multiple definition of `malloc'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):/global/homes/e/earamos/charm/gni-crayxc-smp/tmp/libmemory-isomalloc.C:669: first defined here
/usr/lib/../lib64/libc.a(malloc.o): In function `__libc_free':
/home/abuild/rpmbuild/BUILD/glibc-2.22/malloc/malloc.c:2937: multiple definition of `free'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):/global/homes/e/earamos/charm/gni-crayxc-smp/tmp/libmemory-isomalloc.C:677: first defined here
/usr/lib/../lib64/libc.a(malloc.o): In function `__libc_realloc':
/home/abuild/rpmbuild/BUILD/glibc-2.22/malloc/malloc.c:2972: multiple definition of `realloc'
../../../bin/../lib/libmemory-isomalloc.a(libmemory-isomalloc.o):/global/homes/e/earamos/charm/gni-crayxc-smp/tmp/libmemory-isomalloc.C:695: first defined here
../../../bin/../lib/libconv-util.a(sockRoutines.o): In function `skt_lookup_ip':
/global/homes/e/earamos/charm/gni-crayxc-smp/tmp/sockRoutines.c:302: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/bin/ld: link errors found, deleting executable `jacobi.iso'
collect2: error: ld returned 1 exit status
Fatal Error by charmc in directory /global/homes/e/earamos/charm/gni-crayxc-smp/examples/ampi/Cjacobi3D
   Command CC -rdynamic -D_REENTRANT jacobi.iso.o moduleinit14204.o -L../../../bin/../lib -lmoduletcharm -lmoduleCommonLBs -lmoduleampi -lmoduletcharmmain -lampiromio -lampi-compat -lckmain -lck -lmemory-isomalloc -lthreads-default -lconv-machine -lconv-core -ltmgr -lconv-util -lconv-partition -lhwloc_embedded -lm -lmemory-isomalloc -lthreads-default -lldb-rand -lconv-ldb -lckqt -lpthread -lrt -ldl -lmoduletcharmmain -lmoduleampi -lmoduleCommonLBs -lmoduletcharm -ltcharm-compat -lmoduleNDMeshStreamer -lmodulecompletion -lm -o jacobi.iso returned error code 1
charmc exiting...
$ mkdir libc
$ cd libc
$ ar x /usr/lib/../lib64/libc.a
$ rm malloc.o
$ ar cr ../libc.a *
$ cd ..
$ ranlib libc.a
$ ../../../bin/ampicxx  -o jacobi.iso jacobi.iso.o -module CommonLBs -memory isomalloc -L.
../../../bin/../lib/libconv-util.a(sockRoutines.o): In function `skt_lookup_ip':
/global/homes/e/earamos/charm/gni-crayxc-smp/tmp/sockRoutines.c:302: warning: Using 'gethostbyname' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
./libc.a(dl-tls.o): In function `_dl_allocate_tls_storage':
/home/abuild/rpmbuild/BUILD/glibc-2.22/elf/dl-tls.c:367: undefined reference to `__libc_memalign'
/usr/lib/../lib64/libhugetlbfs.a(morecore.o): In function `__lh_hugetlbfs_setup_morecore':
/home/abuild/rpmbuild/BUILD/cray-libhugetlbfs-2.20/morecore.c:351: undefined reference to `__morecore'
/home/abuild/rpmbuild/BUILD/cray-libhugetlbfs-2.20/morecore.c:358: undefined reference to `mallopt'
/home/abuild/rpmbuild/BUILD/cray-libhugetlbfs-2.20/morecore.c:361: undefined reference to `mallopt'
/home/abuild/rpmbuild/BUILD/cray-libhugetlbfs-2.20/morecore.c:365: undefined reference to `mallopt'
/home/abuild/rpmbuild/BUILD/cray-libhugetlbfs-2.20/morecore.c:360: undefined reference to `mallopt'
./libc.a(fork.o): In function `__libc_fork':
/home/abuild/rpmbuild/BUILD/glibc-2.22/posix/../sysdeps/nptl/fork.c:117: undefined reference to `__malloc_fork_lock_parent'
/home/abuild/rpmbuild/BUILD/glibc-2.22/posix/../sysdeps/nptl/fork.c:177: undefined reference to `__malloc_fork_unlock_child'
/home/abuild/rpmbuild/BUILD/glibc-2.22/posix/../sysdeps/nptl/fork.c:222: undefined reference to `__malloc_fork_unlock_parent'
/usr/bin/ld: link errors found, deleting executable `jacobi.iso'
collect2: error: ld returned 1 exit status
Fatal Error by charmc in directory /global/homes/e/earamos/charm/gni-crayxc-smp/examples/ampi/Cjacobi3D
   Command CC -rdynamic -D_REENTRANT -L. jacobi.iso.o moduleinit20740.o -L../../../bin/../lib -lmoduletcharm -lmoduleCommonLBs -lmoduleampi -lmoduletcharmmain -lampiromio -lampi-compat -lckmain -lck -lmemory-isomalloc -lthreads-default -lconv-machine -lconv-core -ltmgr -lconv-util -lconv-partition -lhwloc_embedded -lm -lmemory-isomalloc -lthreads-default -lldb-rand -lconv-ldb -lckqt -lpthread -lrt -ldl -lmoduletcharmmain -lmoduleampi -lmoduleCommonLBs -lmoduletcharm -ltcharm-compat -lmoduleNDMeshStreamer -lmodulecompletion -lm -o jacobi.iso returned error code 1
charmc exiting...

#12 Updated by Sam White 6 months ago

charmc also adds "-Wl,--allow-multiple-definition" to OPTS_LD when using -tlsglobals, I'm not sure why

#13 Updated by Evan Ramos 6 months ago

Adding the following to jacobi.C allows jacobi.iso to link on Cori:

extern "C" int __libc_memalign(void) { return 0; }

extern "C" int __morecore(void) { return 0; }
extern "C" int mallopt(void) { return 0; }

extern "C" int __malloc_fork_lock_parent(void) { return 0; }
extern "C" int __malloc_fork_unlock_child(void) { return 0; }
extern "C" int __malloc_fork_unlock_parent(void) { return 0; }

These are the symbols that we will need to implement to avoid libc.a's malloc.o from being used.

#14 Updated by Evan Ramos 6 months ago

Sam White wrote:

charmc also adds "-Wl,--allow-multiple-definition" to OPTS_LD when using -tlsglobals, I'm not sure why

Testing on Linux and Cray XC, removing it seems to have no effect.

#15 Updated by Evan Ramos 5 months ago

The following should be what we need:

CMI_EXTERNC void * __libc_memalign (size_t alignment, size_t bytes) { return memalign(alignment, bytes); }

CMI_EXTERNC int mallopt (int param_number, int value) { return 1; }

CMI_EXTERNC void * __default_morecore (ptrdiff_t);
CMI_EXTERNC_VARIABLE void *(*__morecore)(ptrdiff_t) = __default_morecore;

CMI_EXTERNC void __malloc_fork_lock_parent (void) { }
CMI_EXTERNC void __malloc_fork_unlock_parent (void) { }
CMI_EXTERNC void __malloc_fork_unlock_child (void) { }

#16 Updated by Evan Ramos 5 months ago

  • Status changed from New to Implemented

With https://charm.cs.illinois.edu/gerrit/4325 jacobi.iso compiles, links, and runs on gni-crayxc without -Wl,--allow-multiple-definition.

#17 Updated by Sam White 4 months ago

  • Status changed from Implemented to Merged
  • Target version set to 6.9.0

Also available in: Atom PDF