O2 build problems with macOS Ventura 13.1

Dear colleagues,
I’m trying to build O2/O2Physics after a forced update of the macOS to Ventura 13.1.
I already faced several difficulties in the compilation, but now I’m stuck with XRootD.
I tried to look for a possible reason but I couldn’t find the right one.
The most relevant error is:

  ld: library not found for -ldl
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/Library/Developer/CommandLineTools/usr/bin/c++' failed with exit code 1
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

The full log is attached.
Here is the list of my building tools and their version:

alibuild 1.13.1
autoconf 2.69 2.71
automake 1.16.5
berkeley-db 18.1.40_1
boost 1.80.0
ca-certificates 2022-10-11
cmake 3.25.1
coreutils 9.1
docbook 5.1_1
docbook-xsl 1.79.2_1
fmt 9.1.0
freetype 2.12.1
gcc 12.2.0
gcc@7 7.5.0_4
gdbm 1.18.1_1 1.23 1.20
gettext 0.21.1
glfw 3.3.8
gmp 6.2.1_1
gsl 2.7.1
gtk-doc 1.33.2_1
hub 2.14.2
icu4c 71.1 67.1
isl 0.25
libidn2 2.3.4_1
libmpc 1.3.1
libomp 15.0.6
libpng 1.6.39
libsodium 1.0.18_1
libtool 2.4.7
libunistring 1.1
libxml2 2.10.3_1
libyaml 0.2.5
llvm 15.0.6
llvm@14 14.0.6
lz4 1.9.4
m4 1.4.18 1.4.19
make 4.4
modules 4.6.1 5.2.0 4.8.0
mpdecimal 2.5.1
mpfr 4.2.0
msgpack 4.0.0
o2-full-deps 21.06-1
openssl@1.1 1.1.1s 1.1.1i 1.1.1l 1.1.1k
pcre 8.45 8.44
perl 5.36.0
pigz 2.7_1 2.6 2.5
pkg-config 0.29.2_3
pygments 2.14.0
python@3.11 3.11.1
python@3.8 3.8.16
python@3.9 3.9.16
readline 8.2.1 8.1
s3cmd 2.3.0 2.1.0_3
six 1.16.0_1 1.16.0_3
sqlite 3.40.1 3.36.0 3.34.0
tcl-tk 8.6.11_1 8.6.11 8.6.13
texinfo 7.0.1
utf8proc 2.8.0
xz 5.4.0 5.2.5
yaml-cpp 0.7.0
z3 4.11.2_1
zeromq 4.3.4
zopfli 1.0.3_1
zstd 1.5.2 1.5.0

Any help would be really appreciated since I am runnning out of ideas.
Thanks in advance,
Stefano

log.log (107.9 KB)

It looks like yet another issue in XRootd where its cmake thinks you are on a Linux box. Maybe @asevcenc has some ideas on how to fix it?

well, i have no idea … you could try in your environment to do python3 -m pip install --user --upgrade xrootd and if it breaks, report to upstream the problem (Issues · xrootd/xrootd · GitHub)

if it is successful, please report here, as it would mean that the settings that we use must be tweaked, and maybe an macos expert can find then a solution…
(personally i strongly believe macos to be a bad choice for any kind of usage, so not only that i have 0 experience with it but also i do not plan to gain any)

Dear @eulisse, @asevcenc,
I tried to do python3 -m pip install --user --upgrade xrootd but it broke:

Failed to build xrootd
Installing collected packages: xrootd
  Running setup.py install for xrootd ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for xrootd did not run successfully.
  │ exit code: 1
  ╰─> [23 lines of output]
      /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages/setuptools/dist.py:487: UserWarning: Normalizing '5.5.1
      ' to '5.5.1'
        warnings.warn(tmpl.format(**locals()))
      running install
      Some dependencies are missing:
          openssl development package is missing!
          python3 development package is missing!
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/2w/6pqhy56s3gq1q6t_6s0p2hw00000gn/T/pip-install-dl4eysdr/xrootd_dae8f5736cda4044b6c0545263e5527c/setup.py", line 205, in <module>
          setup(
        File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
          dist.run_commands()
        File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
          self.run_command(cmd)
        File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
          cmd_obj.run()
        File "/private/var/folders/2w/6pqhy56s3gq1q6t_6s0p2hw00000gn/T/pip-install-dl4eysdr/xrootd_dae8f5736cda4044b6c0545263e5527c/setup.py", line 165, in run
          raise Exception( 'Dependencies missing!' )
      Exception: Dependencies missing!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> xrootd

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Hence, as suggested by Adrian, I reported the issue in the github of xrootd. I hope to have a solution as soon as possible.

@strogolo unfortunately these error come from lacking of dependencies: python3 and openssl
beside, these are also required: json-c, libxml2, libuuid, zlib

I do not understand, is this a new behaviour? We have everything you mention in alidist, how come they are not picked up?

Hi @asevcenc, @eulisse,
indeed is a bit strange. If I do brew list --versions | sort I get the following list:

alibuild 1.13.1
autoconf 2.71
automake 1.16.5
berkeley-db 18.1.40_1
boost 1.80.0
ca-certificates 2022-10-11
cmake 3.25.1
coreutils 9.1
docbook 5.1_1
docbook-xsl 1.79.2_1
fmt 9.1.0
freetype 2.12.1
gcc 12.2.0
gcc@7 7.5.0_4
gdbm 1.23
gettext 0.21.1
git 2.39.0
glfw 3.3.8
gmp 6.2.1_1
gsl 2.7.1
gtk-doc 1.33.2_1
hub 2.14.2
icu4c 71.1
isl 0.25
libidn2 2.3.4_1
libmpc 1.3.1
libomp 15.0.6
libpng 1.6.39
libsodium 1.0.18_1
libtool 2.4.7
libunistring 1.1
libxml2 2.10.3_1
libyaml 0.2.5
llvm 15.0.6
llvm@14 14.0.6
lz4 1.9.4
m4 1.4.19
make 4.4
modules 5.2.0
mpdecimal 2.5.1
mpfr 4.2.0
msgpack 4.0.0
o2-full-deps 21.06-1
openssl@1.1 1.1.1s
pcre 8.45
pcre2 10.42
perl 5.36.0
pigz 2.7_1
pkg-config 0.29.2_3
pygments 2.14.0
python@3.11 3.11.1
python@3.8 3.8.16
python@3.9 3.9.16
readline 8.2.1
s3cmd 2.3.0
six 1.16.0_3
sqlite 3.40.1
tcl-tk 8.6.13
texinfo 7.0.1
utf8proc 2.8.0
xz 5.4.0
yaml-cpp 0.7.0
z3 4.11.2_1
zeromq 4.3.4
zopfli 1.0.3_1
zstd 1.5.2

Here python3, openssl and libxml2 are reported, while are missing json-c, libuuid and zlib.

Hi @eulisse ! See my above message where i said

well, i have no idea … you could try in your environment to do python3 -m pip install --user --upgrade xrootd and if it breaks, report to upstream the problem (Issues · xrootd/xrootd · GitHub 1)

if it is successful, please report here, as it would mean that the settings that we use must be tweaked, and maybe an macos expert can find then a solution…

so, these missing dependencies comes from tryout of installing xrootd directly in as the user (not within ALICE framework), and the purpose of exercise was to validate the xrootd installation process as native procedure (so it can be reported to upstream as is, without any of our customizations that could be hard to debug by upstream devs)

Dear @eulisse, @asevcenc,
I have received a reply from XRootD people (XRootD compilation failing on macOS (13.1) · Issue #1878 · xrootd/xrootd · GitHub).
From his comment it seems that openssl and python3 dependencies are missing and moreover he suggests to install XRootD from Homebrew.
Nevertheless, if I check (brew list --version | sort or directly tyiping openssl version or python3) I can see that they are present in my laptop. Hence, do you have any suggestion on how to fix these dependencies?
Thanks,
Stefano

Dear all,
I am experiencing a very similar problem with macos Monterey, openssl version
LibreSSL 2.8.3 and python@3.11 3.11.1.

Stuck since few days, don’t know what else to do…
Any help would be appreciated.
Thanks in advance, Chiara

Dear all,
I am also experiencing this same issue on Monterey for AliPhysics. Thank you for any help!
Caitie

Dear @eulisse,
this is just to let you know that by doing

brew install python@3.10
brew link --overwrite python@3.10

the compilation of both O2Physics and AliPhysics went until the end without any problem.

Dear all,
this doesn’t work anymore with MacOS Monterey 12.6.3, XCode 14.2.
It seems there is a mismatch between installed and linked python versions. Many users are reporting it via mails and, so far, no help nor hints were suggested by anyone. Is there a way to help people overcome this issue and try to work?

Thanks in advance,
Chiara

1 Like