Project

General

Profile

Feature #1875

Test whether hwloc can be used via a git submodule in charm++

Added by Matthias Diener over 1 year ago. Updated about 1 year ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
04/18/2018
Due date:
% Done:

0%


Description

.

History

#1 Updated by Matthias Diener over 1 year ago

  • Target version set to 6.9.0

#2 Updated by Evan Ramos over 1 year ago

  • Target version deleted (6.9.0)
  • Status changed from New to In Progress

This has two blocking issues and one inconvenience to users:

1. Build breaks with the following messages:

configure.ac:2446: error: required file 'hwloc/config/compile' not found
configure.ac:2446:   'automake --add-missing' can install 'compile'
configure.ac:2445: error: required file 'hwloc/config/config.guess' not found
configure.ac:2445:   'automake --add-missing' can install 'config.guess'
configure.ac:2445: error: required file 'hwloc/config/config.sub' not found
configure.ac:2445:   'automake --add-missing' can install 'config.sub'
configure.ac:2449: error: required file 'hwloc/config/install-sh' not found
configure.ac:2449:   'automake --add-missing' can install 'install-sh'
configure.ac:2447: error: required file 'hwloc/config/ltmain.sh' not found
configure.ac:2449: error: required file 'hwloc/config/missing' not found
configure.ac:2449:   'automake --add-missing' can install 'missing'

We seem to need another preprocessing step before autotools are run, but I'm not immediately sure what it is.

2. Submodules break the git-review script Gerrit requires with this message:

$ git review -T
Errors running git rebase -p -i remotes/origin/charm
error: The following untracked working tree files would be overwritten by checkout:
[every file in contrib/hwloc/]
Please move or remove them before you switch branches.
Aborting
could not detach HEAD
It is likely that your change has a merge conflict. You may resolve it
in the working tree now as described above and then run 'git review'
again, or if you do not want to resolve it yet (note that the change
can not merge until the conflict is resolved) you may run 'git rebase
--abort' then 'git review -R' to upload the change without rebasing.

I had to run git review -R to upload the change at all.

3. In order to download the hwloc code, users must either manually initialize and download the submodule, or must git clone charm with the "--recursive" parameter.

#4 Updated by Evan Ramos over 1 year ago

I don't believe moving hwloc to a git submodule gives us anything. Updating hwloc is as simple as downloading the .tar.gz and passing it to my update-hwloc.sh script that can be found in the contrib folder.

As far as knowing when a new release is out, we would still need to manually change which commit our submodule is pointing to upstream. So far, I have known when a new release of hwloc is out because I have the package installed on my work MacBook and I see it as part of "brew upgrade" output.

#5 Updated by Evan Ramos over 1 year ago

  • Status changed from In Progress to Feedback

#6 Updated by Matthias Diener over 1 year ago

1. - Does running autoreconf -ivf (or autogen.sh) solve this issue?
2. - I haven't used git-review, so no idea how to fix that.
3. - Could we do some thing like [ -d contrib/hwloc ] || git submodule init && git submodule update; in the build script?

#7 Updated by Evan Ramos over 1 year ago

1. It gets further, but still fails when the configure script hits hwloc's section:

config.status: error: cannot find input file: `hwloc/include/private/autogen/config.h.in'

3. The closest I've gotten is

[ -d contrib/hwloc/.git ] || git submodule init contrib/hwloc ; git submodule update contrib/hwloc

By default, contrib/hwloc/ exists as an empty folder, so we can't test for it directly. I have hit a case where it was empty but the submodule commands did nothing, which I don't know how to detect.

#8 Updated by Eric Bohm about 1 year ago

  • Assignee set to Evan Ramos

#9 Updated by Evan Ramos about 1 year ago

For the moment, we now include a hotfix for hwloc in order to resolve #1895, something which would have been impossible with submodules.

#10 Updated by Matthias Diener about 1 year ago

  • Status changed from Feedback to Rejected

I think we can close this report for now. As Evan said, if we use an embedded hwloc (as opposed to a system hwloc), shipping it in a git submodule doesn't really improve anything. We should make sure that mixing the embedded hwloc with another hwloc (e.g. if a Charm++ application needs hwloc and links with -lhwloc) is supported (or warn against this in the manual).

#11 Updated by Matthias Diener about 1 year ago

  • Description updated (diff)

Also available in: Atom PDF