Export DYLD_LIBRARY_PATH in modulefiles for mac builds

mfasel · October 19, 2022, 12:37pm

Dear experts,

seems the support for LD_LIBRARY_PATH was removed in dlopen in macOS Monterey and only the DYLD_LIBRARY_PATH is further supported, for which i.e. the autoloader of ROOT which relies on dlopen no longer works when exporting only the LD_LIBRARY_PATH (see i.e. discussion in 'dlopen' ignore LD_LIBRARY_PATH on… | Apple Developer Forums). Indeed when exporting manually the DYLD_LIBRARY_PATH to the LD_LIBRARY_PATH the libraries get loaded again. Wouldn’t it make sense to export the DYLD_LIBRARY_PATH in modulefiles besides the LD_LIBRARY_PATH (which I assume will be needed for compatibility) in case of mac builds?

Cheers

Markus

laphecet · October 19, 2022, 1:03pm

Hi Markus,

I would have thought that this is taken care of by the ROOT_DYN_PATH env. variable that is set by the alibuild module for Root and used in the $ROOTSYS/etc/system.rootrc (see https://github.com/alisw/alidist/blob/61e83524ce4c63b1494f07fcea8347f0e5601110/root.sh#L206), isn’t it ?

Could you give an example of what’s not working for you ? Thanks.

mfasel · October 19, 2022, 1:26pm

Hi Laurant,

Simple reproducer:

alienv enter AliPhysics/latest
Loading AliPhysics/latest
  Loading requirement: BASE/1.0 protobuf/v3.15.8-local1 utf8proc/v2.6.1-local1 boost/v1.75.0-local1 xsimd/8.1.0-local1 arrow/v9.0.0-alice1-local1 lzma/v5.2.3-local1 AliEn-Runtime/v2-19-le-local1
    XRootD/v5.5.0-local1 TBB/v2021.5.0-local1 ROOT/v6-26-04-patches-alice1-local1 DPMJET/v19.1.2-alice3-local1 cgal/4.12.2-local1 fastjet/v3.4.0_1.045-alice1-local1 VMC/v2-0-local1 GEANT3/v4-1-local1
    ZeroMQ/v4.3.3-local1 GEANT4/v11.0.3-local1 vgm/v5-0-local1 GEANT4_VMC/v6-1-p1-local1 Vc/1.4.1-local1 xjalienfs/1.4.5-local1 JAliEn-ROOT/0.6.8-local1 AliRoot/aliceo2-local1
    RooUnfold/V02-00-01-alice5-local1 treelite/8498081-local1 KFParticle/v1.1-4-local1
Currently Loaded Modulefiles:
 1) BASE/1.0                     7) lzma/v5.2.3-local1                   13) cgal/4.12.2-local1                  19) vgm/v5-0-local1            25) RooUnfold/V02-00-01-alice5-local1
 2) protobuf/v3.15.8-local1      8) AliEn-Runtime/v2-19-le-local1        14) fastjet/v3.4.0_1.045-alice1-local1  20) GEANT4_VMC/v6-1-p1-local1  26) treelite/8498081-local1
 3) utf8proc/v2.6.1-local1       9) XRootD/v5.5.0-local1                 15) VMC/v2-0-local1                     21) Vc/1.4.1-local1            27) KFParticle/v1.1-4-local1
 4) boost/v1.75.0-local1        10) TBB/v2021.5.0-local1                 16) GEANT3/v4-1-local1                  22) xjalienfs/1.4.5-local1     28) AliPhysics/latest
 5) xsimd/8.1.0-local1          11) ROOT/v6-26-04-patches-alice1-local1  17) ZeroMQ/v4.3.3-local1                23) JAliEn-ROOT/0.6.8-local1
 6) arrow/v9.0.0-alice1-local1  12) DPMJET/v19.1.2-alice3-local1         18) GEANT4/v11.0.3-local1               24) AliRoot/aliceo2-local1

Key:
auto-loaded
Use alienv list to list loaded modules. Use exit to exit this environment.
[AliPhysics/latest] ~ %> root -l
root [0] AliEmcalList *test = new AliEmcalList()
cling::DynamicLibraryManager::loadLibrary(): dlopen(/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/libPWGEMCALbase.so, 0x0009): Library not loaded: '@rpath/libboost_unit_test_framework.dylib'
  Referenced from: '/Users/markusfasel/alice/sw/osx_arm64/RooUnfold/V02-00-01-alice5-local1/lib/libRooUnfold.dylib'
  Reason: tried: '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/bin/../lib/libboost_unit_test_framework.dylib' (no such file), '/usr/local/lib/libboost_unit_test_framework.dylib' (no such file), '/usr/lib/libboost_unit_test_framework.dylib' (no such file)
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libPWGEMCALbase for AliEmcalList
cling::DynamicLibraryManager::loadLibrary(): dlopen(/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/libPWGEMCALbase.so, 0x0009): Library not loaded: '@rpath/libboost_unit_test_framework.dylib'
  Referenced from: '/Users/markusfasel/alice/sw/osx_arm64/RooUnfold/V02-00-01-alice5-local1/lib/libRooUnfold.dylib'
  Reason: tried: '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/bin/../lib/libboost_unit_test_framework.dylib' (no such file), '/usr/local/lib/libboost_unit_test_framework.dylib' (no such file), '/usr/lib/libboost_unit_test_framework.dylib' (no such file)
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libPWGEMCALbase for AliEmcalList
cling::DynamicLibraryManager::loadLibrary(): dlopen(/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/libPWGEMCALbase.so, 0x0009): Library not loaded: '@rpath/libboost_unit_test_framework.dylib'
  Referenced from: '/Users/markusfasel/alice/sw/osx_arm64/RooUnfold/V02-00-01-alice5-local1/lib/libRooUnfold.dylib'
  Reason: tried: '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/bin/../lib/libboost_unit_test_framework.dylib' (no such file), '/usr/local/lib/libboost_unit_test_framework.dylib' (no such file), '/usr/lib/libboost_unit_test_framework.dylib' (no such file)
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libPWGEMCALbase for AliEmcalList
cling::DynamicLibraryManager::loadLibrary(): dlopen(/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/libPWGEMCALbase.so, 0x0009): Library not loaded: '@rpath/libboost_unit_test_framework.dylib'
  Referenced from: '/Users/markusfasel/alice/sw/osx_arm64/RooUnfold/V02-00-01-alice5-local1/lib/libRooUnfold.dylib'
  Reason: tried: '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliPhysics/aliceo2-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliRoot/aliceo2-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/lib/../lib/libboost_unit_test_framework.dylib' (no such file), '/Users/markusfasel/alice/sw/osx_arm64/ROOT/v6-26-04-patches-alice1-local1/bin/../lib/libboost_unit_test_framework.dylib' (no such file), '/usr/local/lib/libboost_unit_test_framework.dylib' (no such file), '/usr/lib/libboost_unit_test_framework.dylib' (no such file)
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libPWGEMCALbase for AliEmcalList
ROOT_prompt_0:1:26: error: unknown type name 'AliEmcalList'
AliEmcalList *test = new AliEmcalList()
                         ^
root [1] .q
[AliPhysics/latest] ~ %> export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH
[AliPhysics/latest] ~ %> root -l
root [0] AliEmcalList *test = new AliEmcalList()
(AliEmcalList *) 0x600001c3c000

I know several other people having the same problem, where exporting DYLD_LIBRARY_PATH helped in the end. However it might be that the issue is related to a few libraries in the dependency chain not included in the ROOT_DYN_PATH, where probably just a few recipes need to be fixed (boost in this example).

Cheers

Markus