Project

General

Profile

Bug #1668

Ensure that all libraries/modules will build as dynamic/shared objects (.so/.dylib vs .a)

Added by Phil Miller 3 months ago. Updated 25 days ago.

Status:
Implemented
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
08/29/2017
Due date:
% Done:

0%


Description

./build LIBS netlrts-darwin-x86_64 --build-shared --suffix=shared -j2 -g

$ ls netlrts-darwin-x86_64-shared/lib{,_so} | grep -v '.dep$' | sort | cut -d. -f1 | uniq -c | sort -n | grep 1 | grep -v /
   1 libampiromio
   1 libirecv
   1 libmoduleCkSparseContiguousReducer
   1 libmodulesearchEngine
   1 libmodulesearchEngine_bound
   1 libqt
   1 libhwloc_embedded

Something is screwy in the build process of these library objects that leaves them without a .so/.dylib built, where every other library has one. The most concerning to me are libqt and libampiromio.

Related issues

Related to Charm++ - Feature #952: Update AMPI's version of ROMIO In Progress 01/17/2016

History

#1 Updated by Phil Miller 3 months ago

So, it turns out that libqt is wrong in the opposite direction - libqt.a gets built, but doesn't get copied to the lib directory, while libqt.dylib (on my mac) gets copied to lib_so

#2 Updated by Phil Miller 3 months ago

A further amendment to that statement - libqt.a is missing because it's only copied as

cp libqt.a ../lib/libckqt.a

Note the change in name from libqt to libckqt. The dynamic version libqt.dylib is copied as both libqt and libckqt. This is a little weird.

#3 Updated by Matthias Diener 3 months ago

ROMIO is currently not able to be built dynamically. Bug #952 tracks an update for ROMIO which should make it easier to enable creating a shared romio library.

#4 Updated by Matthias Diener 3 months ago

Can we merge lib/ and lib_so/ ? I think this separation is not necessary, as shared and static libraries have different file extensions. Normal Linux systems have *.so and *.a in /usr/lib/, for example.

#5 Updated by Phil Miller 3 months ago

  • Related to Feature #952: Update AMPI's version of ROMIO added

#6 Updated by Phil Miller 3 months ago

Merging the two directories would be a separate issue - this is just about whether both static and dynamic objects get built and installed at all.

#7 Updated by Phil Miller 3 months ago

  • Status changed from New to Implemented

remote: https://charm.cs.illinois.edu/gerrit/2962 Bug #1668 irecv Makefile: enable charmc's automatic generation of dynamic ...
remote: https://charm.cs.illinois.edu/gerrit/2963 Bug #1668 Makefile: Don't copy libqt.a under an unused name
remote: https://charm.cs.illinois.edu/gerrit/2964 Bug #1668 state_space_searchengine: Ensure dynamic libraries get built when ...
remote: https://charm.cs.illinois.edu/gerrit/2965 Bug #1668 sparseContiguousReducer: Ensure dynamic libraries get built when ...

AMPI's ROMIO will have to happen with the version update mentioned. I'll add a note over there.

#8 Updated by Phil Miller 3 months ago

  • Assignee set to Phil Miller

#9 Updated by Phil Miller 3 months ago

Ok, I see why we don't merge the two directories - when linking under control of charmc, it addresses user requests for static or dynamic linking of the finished binary by only passing one path or the other to the underlying linker. This means that even if the underlying tool is stupid about respecting the user's preference, we guarantee that it will only see the desired variant of the RTS objects.

#10 Updated by Phil Miller 26 days ago

  • Description updated (diff)

Looks like our hwloc also needs to have shared object compilation enabled. It's missing, probably because it doesn't ultimately use charmc to assemble its .a, which would implicitly generate the .so/.dylib/.dll as well.

#11 Updated by Sam White 26 days ago

libampiromio is only built statically because the version of ROMIO we have doesn't support shared builds.
I believe this commit would get us that, but it isn't working yet. Matthias may have more details on it: https://charm.cs.illinois.edu/gerrit/#/c/2940/

Edit: the above comments confirm this.

#12 Updated by Matthias Diener 25 days ago

Yeah, there is no update regarding romio yet.

#13 Updated by Phil Miller 25 days ago

charmc wants to reference hwloc when linking a shared object for QuickThreads. hwloc doesn't build a corresponding shared object by default, and adding enable_shared='yes' in configure.ac isn't enough to do it. More investigation to follow.

Also available in: Atom PDF