Alibuild fails on ninja on ubuntu 20.04

O²
1

Hi,
Just tried to rebuild O2 with updated alidist master, it fails on

ERROR: Error while executing /home/shahoian/alice/sw/SPECS/ubuntu2004_x86-64/ninja/fortran-v1.8.2.g3b-1/build.sh on `alicrs02'.

less /home/shahoian/alice/sw/BUILD/ninja-latest/log

++ /home/shahoian/alice/sw/SOURCES/ninja/fortran-v1.8.2.g3b/v1.8.2.g3bbbe.kitware.dyndep-1.jobserver-1/configure.py --bootstrap
/usr/bin/env: 'python': No such file or directory

@ktf, are we now forced to used ninja? Any idea what is wrong with configuration?
I paste below the aliDoctor output.

Cheers,
Ruben

shahoian@alicrs02:~/alice$ aliDoctor  O2 --defaults o2
SUCCESS: Required package openmp will be picked up from the system.
WARNING: Package libuv cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; printf "#include <uv.h>" | c++ -I$(brew --prefix libuv)/include -xc++ - -c -o /dev/null 2>&1
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: libuv: <stdin>:1:10: fatal error: uv.h: No such file or directory
WARNING: libuv: compilation terminated.
WARNING: libuv: 
WARNING: 
WARNING: Package lz4 cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; printf "#include <lz4.h>\n" | cc -xc -I$(brew --prefix lz4)/include - -c -M 2>&1
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: lz4: <stdin>:1:10: fatal error: lz4.h: No such file or directory
WARNING: lz4: compilation terminated.
WARNING: lz4: 
WARNING: 
SUCCESS: Package zlib will be picked up from the system.
WARNING: Package flatbuffers cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; which flatc && printf "#include \"flatbuffers/flatbuffers.h\"\nint main(){}" | c++ -I$(brew --prefix flatbuffers)/include -xc++ -std=c++11 - -o /dev/null
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: flatbuffers: 
WARNING: 
WARNING: Package CMake cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; verge() { [[  "$1" = "`echo -e "$1\n$2" | sort -V | head -n1`" ]]; }
WARNING: type cmake && verge 3.19.2 `cmake --version | sed -e 's/.* //' | cut -d. -f1,2,3`
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: CMake: /bin/bash: line 1: type: cmake: not found
WARNING: CMake: 
WARNING: 
WARNING: Package yaml-cpp cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; pkg-config --atleast-version=0.6.2 yaml-cpp && printf "#include \"yaml-cpp/yaml.h\"\n" | c++ -std=c++17 -I`brew --prefix yaml-cpp`/include -I`brew --prefix boost`/include -xc++ - -c -o /dev/null
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: yaml-cpp: 
WARNING: 
SUCCESS: Package GCC-Toolchain will be picked up from the system.
WARNING: Package ZeroMQ cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; printf "#include <zmq.h>\n#if(ZMQ_VERSION < 40105)\n#error \"zmq version >= 4.1.5 needed\"\n#endif\n int main(){}" | c++ -I$(brew --prefix zeromq)/include -xc++ - -o /dev/null 2>&1
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: ZeroMQ: <stdin>:1:10: fatal error: zmq.h: No such file or directory
WARNING: ZeroMQ: compilation terminated.
WARNING: ZeroMQ: 
WARNING: 
WARNING: Package OpenSSL cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; if [ `uname` = Darwin ]; then test -d `brew --prefix openssl || echo /dev/nope` || exit 1; fi; echo '#include <openssl/bio.h>' | c++ -x c++ - -I`brew --prefix openssl`/include -c -o /dev/null || exit 1; echo -e "#include <openssl/opensslv.h>\n#if OPENSSL_VERSION_NUMBER >= 0x10100000L\n#error \"System's GCC cannot be used: we need OpenSSL 1.0.x to build XrootD. We are going to compile our own version.\"\n#endif\nint main() { }" | cc -x c++ - -I`brew --prefix openssl`/include -c -o /dev/null || exit 1
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: OpenSSL: <stdin>:3:2: error: #error "System's GCC cannot be used: we need OpenSSL 1.0.x to build XrootD. We are going to compile our own version."
WARNING: OpenSSL: 
WARNING: 
SUCCESS: Package GLFW will be picked up from the system.
WARNING: Package FreeType cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; printf "#include <ft2build.h>\n" | c++ -xc++ - `freetype-config --cflags` -c -M 2>&1;
WARNING: if [ $? -ne 0 ]; then printf "FreeType is missing on your system.\n * On RHEL-compatible systems you probably need: freetype freetype-devel\n * On Ubuntu-compatible systems you probably need: libfreetype6 libfreetype6-dev\n"; exit 1; fi
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: FreeType: /bin/bash: freetype-config: command not found
WARNING: FreeType: <stdin>:1:10: fatal error: ft2build.h: No such file or directory
WARNING: FreeType: compilation terminated.
WARNING: FreeType: FreeType is missing on your system.
WARNING: FreeType:  * On RHEL-compatible systems you probably need: freetype freetype-devel
WARNING: FreeType:  * On Ubuntu-compatible systems you probably need: libfreetype6 libfreetype6-dev
WARNING: FreeType: 
WARNING: 
SUCCESS: Required package system-curl will be picked up from the system.
SUCCESS: Package libpng will be picked up from the system.
SUCCESS: Package lzma will be picked up from the system.
SUCCESS: Package bz2 will be picked up from the system.
SUCCESS: Required package Python-system will be picked up from the system.
SUCCESS: Required package make will be picked up from the system.
WARNING: Package GSL cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; printf "#include \"gsl/gsl_version.h\"\n#define GSL_V GSL_MAJOR_VERSION * 100 + GSL_MINOR_VERSION\n# if (GSL_V < 116)\n#error \"Cannot use system's gsl. Notice we only support versions from 1.16 (included)\"\n#endif\nint main(){}" | c++  -I$(brew --prefix gsl)/include -xc++ - -o /dev/null
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: GSL: <stdin>:1:10: fatal error: gsl/gsl_version.h: No such file or directory
WARNING: GSL: compilation terminated.
WARNING: GSL: 
WARNING: 
SUCCESS: Required package opengl will be picked up from the system.
SUCCESS: Required package Xdevel will be picked up from the system.
SUCCESS: Package libxml2 will be picked up from the system.
WARNING: Package ofi cannot be picked up from the system and will be built by aliBuild.
WARNING: This is due to the fact the following script fails:
WARNING: 
WARNING: brew() { true; }; pkg-config --atleast-version=1.6.0 libfabric 2>&1 && printf "#include \"rdma/fabric.h\"\nint main(){}" | c++ -xc - -o /dev/null
WARNING: 
WARNING: with the following output:
WARNING: 
WARNING: ofi: 
WARNING: 

==> The following packages will be built by aliBuild because
     usage of a system version of it is not allowed or supported, by policy:
    
    - FairMQ 
    - curl 
    - asiofi 
    - FLUKA 
    - O2 
    - double-conversion 
    - libtirpc 
    - capstone 
    - libInfoLogger 
    - Vc 
    - ApMon-CPP 
    - fmt 
    - GEANT4_VMC 
    - Python-modules 
    - libjalienO2 
    - GEANT3 
    - fastjet 
    - ninja 
    - cgal 
    - vgm 
    - googlebenchmark 
    - FairLogger 
    - simulation 
    - libwebsockets 
    - HepMC3 
    - pythia6 
    - Python-modules-list 
    - generators 
    - UUID 
    - Configuration 
    - Common-O2 
    - JAliEn-ROOT 
    - Clang 
    - re2 
    - DDS 
    - xjalienfs 
    - MCStepLogger 
    - AliEn-CAs 
    - lhapdf 
    - GMP 
    - GEANT4 
    - MPFR 
    - Ppconsul 
    - ROOT 
    - arrow 
    - XRootD 
    - DebugGUI 
    - pythia 
    - FLUKA_VMC 
    - KFParticle 
    - defaults-release 
    - Monitoring 
    - RapidJSON 
    - AliEn-Runtime 
    - json-c 
    - FairRoot 
    - Alice-GRID-Utils 
    - O2-customization 
    - protobuf 
    - alibuild-recipe-tools 
    - boost 
    - VecGeom 
    - ms_gsl

==> The following packages will be picked up from the system:
    
    - lzma
    - zlib
    - libpng
    - GLFW
    - GCC-Toolchain
    - bz2
    - libxml2
    
    If this is not you want, you have to uninstall / unload them.

==> The following packages will be built by aliBuild because they couldn't be picked up from the system:
    
    - yaml-cpp
    - GSL
    - flatbuffers
    - ZeroMQ
    - lz4
    - CMake
    - libuv
    - ofi
    - FreeType
    - OpenSSL
    
    This is not a real issue, but it might take longer the first time you invoke aliBuild.
    Look at the error messages above to get hints on what packages you need to install separately.
2

Hi Ruben,

just as a workaround you can check the content of the file
/home/shahoian/alice/sw/SOURCES/ninja/fortran-v1.8.2.g3b/v1.8.2.g3bbbe.kitware.dyndep-1.jobserver-1/configure.py which is mentioned in the build LOG and in there you can change the first line from
#!/usr/bin/env python
to
#!/usr/bin/env python3

For me the full O2 build with ninja worked with that change.

Cheers,
Ole

Edit: made a small correction for anyone who might stumble upon this issue until the ninja build recipe is fixed for Ubuntu

3

Thanks @oschmidt !

Actually, @eulisse has already pushed a fix Update ninja.sh by ktf · Pull Request #3024 · alisw/alidist · GitHub which makes it to work on ubuntu20 (though CI fails on CentOS, so it is not final solution).

Cheers,
Ruben