Problem 1848

Summary: Path to libraries on OS X broken when setting CMAKE_INSTALL_LIBDIR since version 4.10.2
Product: Geant4 Reporter: Mojca Miklavec <mojca.miklavec.lists>
Component: cmakeAssignee: Ben Morgan <Ben.Morgan>
Status: CLOSED INVALID    
Severity: major CC: ahmedsayeed1982
Priority: P5    
Version: 10.2   
Hardware: Apple   
OS: Mac OS X   
Attachments: Build log of geant4.10.2.p01 installation using CMake
Output of "otool -L" and "otool -l"

Description Mojca Miklavec 2016-03-24 05:55:50 CET
I've been packaging Geant4 on MacPorts since version 4.9.5 with basically the same setup, but since version 4.10.2 something is broken.

I use -DCMAKE_INSTALL_LIBDIR=/opt/local/lib/Geant4/Geant4.10.2 and the libraries are placed to the desired place, but the hardcoded paths to libraries seem to be broken:

Could not open /opt/local/lib/libG4zlib.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4clhep.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4global.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4intercoms.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4materials.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4graphics_reps.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4geometry.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4particles.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4track.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4digits_hits.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4processes.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4tracking.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4event.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4run.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4modeling.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4vis_management.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib)
Could not open /opt/local/lib/libG4FR.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4GMocren.dylib)
Could not open /opt/local/lib/libG4interfaces.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4OpenGL.dylib)
Could not open /opt/local/lib/libG4gl2ps.dylib: Error opening or reading file (referenced from /opt/local/lib/Geant4/Geant4.10.2/libG4OpenGL.dylib)

At the moment I'm still clueless about where to start looking for the problem. Grepping for CMAKE_INSTALL_LIBDIR doesn't seem to indicate any obvious changes since version 4.10.1 that still works as expected.

Here's the complete invocation:

cmake \
	-DCMAKE_INSTALL_PREFIX=/opt/local \
	-DCMAKE_VERBOSE_MAKEFILE=ON \
	-DCMAKE_COLOR_MAKEFILE=ON \
	-DCMAKE_BUILD_TYPE=Release \
	-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
	-DCMAKE_INSTALL_RPATH=/opt/local/lib \
	-DCMAKE_INSTALL_NAME_DIR=/opt/local/lib \
	-DCMAKE_SYSTEM_PREFIX_PATH="/opt/local;/usr" \
	-DCMAKE_MODULE_PATH=/opt/local/share/cmake/Modules \
	-DCMAKE_FIND_FRAMEWORK=LAST \
	-Wno-dev \
	-DGEANT4_USE_SYSTEM_EXPAT=ON \
	-DGEANT4_USE_SYSTEM_CLHEP=OFF \
	-DGEANT4_INSTALL_DATA=OFF \
	-DCMAKE_INSTALL_BINDIR=/opt/local/libexec/Geant4/Geant4.10.2 \
	-DCMAKE_INSTALL_DATAROOTDIR=/opt/local/share/Geant4 \
	-DCMAKE_INSTALL_INCLUDEDIR=/opt/local/include/Geant4/Geant4.10.2 \
	-DCMAKE_INSTALL_LIBDIR=/opt/local/lib/Geant4/Geant4.10.2 \
	-DGEANT4_INSTALL_DATADIR=/opt/local/share/Geant4/Data/Geant4.10.2 \
	-DGEANT4_USE_QT=ON \
	-DGEANT4_USE_GDML=ON \
	-DCMAKE_C_FLAGS_RELEASE="-DNDEBUG" \
	-DCMAKE_CXX_FLAGS_RELEASE="-DNDEBUG" \
	-DCMAKE_OSX_ARCHITECTURES="x86_64" \
	-DCMAKE_OSX_DEPLOYMENT_TARGET="10.7" \
	-DCMAKE_OSX_SYSROOT="/"

(The first ten and the last few flags have been added by the package manager by default.)
Comment 1 Ben Morgan 2016-03-24 22:24:08 CET
Geant4 10.2 updated the install name and RPATH settings so that all libraries are built using the "@rpath@" construct (see https://blog.kitware.com/upcoming-in-cmake-2-8-12-osx-rpath-support/) rather than absolute install names.

I would suspect that this combined with the settings MacPorts is using for packaging (CMAKE_BUILD_WITH_INSTALL_RPATH, CMAKE_INSTALL_RPATH and CMAKE_INSTALL_NAME_DIR) are causing this issue (though I do not yet think this is a bug in Geant4). It's not clear from the report if Geant4 has installed successfully, though assuming it has, could you check (and post) the result of

otool -L /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib 

and 

otool -l /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib 


I'd suspect that the rpaths/install_names aren't getting set correctly.
Comment 2 Mojca Miklavec 2016-03-24 23:20:17 CET
Created attachment 392 [details]
Build log of geant4.10.2.p01 installation using CMake
Comment 3 Mojca Miklavec 2016-03-24 23:23:46 CET
Created attachment 393 [details]
Output of "otool -L" and "otool -l"
Comment 4 Mojca Miklavec 2016-03-24 23:35:15 CET
Yes, Geant4 was built and installed correctly albeit with wrong links to libraries, so it's basically broken (and the package manager would automatically trigger a rebuild two or three more times before giving up).

(The flag -DCMAKE_INSTALL_RPATH has been added to the CMake PortGroup about a year ago and I only noticed that flag when I made this report. I don't yet understand how it works though and I made sure to double-check that Geant4.10.1 still installs as expected despite that flag.)

Earlier I forgot to attach the complete build log, it's there now. I hope you will be able to see all the flags that were used during compilation (-install_name /opt/local/lib/libG4zlib.dylib seems wrong for example given that the library gets installed somewhere else).

I tried to debug problem on my own, but it would take me way too much time to fully understand the build system and the long compilation times were additionally preventing me from doing a lot of testing.
Comment 5 Mojca Miklavec 2016-03-24 23:36:28 CET
The output of otool -L starts with

> otool -L /opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib
/opt/local/lib/Geant4/Geant4.10.2/libG4FR.dylib:
	/opt/local/lib/libG4FR.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libG4vis_management.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/local/lib/libG4modeling.dylib (compatibility version 0.0.0, current version 0.0.0)
	...

The rest is attached.
Comment 6 Ben Morgan 2016-03-25 20:48:22 CET
Looking at the output, what I'd suspect is happening is that in 10.1, the library install_names were hardcoded and thus override any setting from the command line (i.e. -DCMAKE_INSTALL_NAME_DIR=.. has no effect).
In 10.2, this was replaced by the cmake MACOSX_RPATH property, but CMAKE_INSTALL_NAME_DIR overrides this, and sets the install_name incorrectly.

The upshot is that there's a conflict between where the libraries are actually installed, /opt/local/lib/Geant4/Geant4.10.2, and the install names that get set on them, /opt/local/lib/<name>. The solution is probably to override the setting of CMAKE_INSTALL_NAME_DIR in your cmake invocation so that it's the same as CMAKE_INSTALL_LIBDIR. The other variable CMAKE_INSTALL_RPATH can probably be left as absolute install names are being used anyway.
Comment 7 Mojca Miklavec 2016-03-26 01:49:45 CET
Thanks a lot for the pointer. I believe I have enough resources to test now before I come back with further details.

Is the following code from Modules/Geant4MacroLibraryTargets.cmake the change that most likely broke the installation for me then?

  set_target_properties(${G4LIBTARGET_NAME}
-   PROPERTIES INSTALL_NAME_DIR ${CMAKE_INSTALL_FULL_LIBDIR})
+   PROPERTIES MACOSX_RPATH 1
+   )

Thank you,
    Mojca
Comment 8 Mojca Miklavec 2016-03-30 09:24:53 CEST
I did some further investigation. Actually the most offending configuration option was the following
-DCMAKE_INSTALL_NAME_DIR=/opt/local/lib
in the first place.

If I deleted in and replaced it with proper settings, it worked fine (I believe, I stopped the build earlier, but according to the invocation of the compiler, it was setting the proper -install_name).

If I removed that setting altogether, RPATH jumped in with

-install_name @rpath/libG4<something>.dylib -Wl,-rpath,/opt/local/lib

and if I removed -DCMAKE_INSTALL_RPATH=/opt/local/lib, the -rpath setting went away, so I guess the libraries would be linked with rpath, but as expected.

I am running the final compilation now, but it seems that adjusting those settings will lead to a proper linkage. I'll try to figure out what we could do to improve the default behaviour in the package manager for cases when relocation of libraries is desired. I'm still baffled why it works properly in previous versions of Geant4 (now I could argue that it shouldn't with that weird -DCMAKE_INSTALL_NAME_DIR setting), but this definitely looks like a "buggy invocation" of CMake on our side.
Comment 9 Mojca Miklavec 2016-03-30 10:52:12 CEST
After the build finished I can finally confirm that different CMake options lead to a properly functional installation. Thank you very much for the pointers and sorry for taking your time. I believe this bug report can be closed as invalid.