Cannot build O2 failing Python-modules@1.0

Hi

I am trying to build O2 on MAC OS (Catalina 10.15.4)
and it fails in Python-modules@1.0

ERROR: Error while executing /Users/germain/alicesw/sw/SPECS/osx_x86-64/Python-modules/1.0-1/build.sh on `nanpc511.in2p3.fr’.
ERROR: Log can be found in /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log
ERROR: Please upload it to CERNBox/Dropbox if you intend to request support.
ERROR: Build directory is /Users/germain/alicesw/sw/BUILD/Python-modules-latest/Python-modules.
germain@nanpc511 alisw %

the logs indicates error

Collecting Keras==2.2.4
Using cached Keras-2.2.4-py2.py3-none-any.whl (312 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow==1.13.1 (from -r requirements.txt (line 18)) (from versions: 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1)
ERROR: No matching distribution found for tensorflow==1.13.1 (from -r requirements.txt (line 18))

I use python 3.8 and it’s the first trial of installation I do on this labtop (no previous version);

Thanks in advance for any help
Best Marie

Hi,

I have checked on my setup and I don’t have the module tensorflow installed by pip. However I use the defaults o2-dataflow and that might be the reason why.

Cheers,
Barth

Hi,

I had a similar problem quite a while ago. Back then there was a list of python modules in alidist/python-modules.sh, which now seems to be in alidist/python-modules-list.sh.

Apparently tensorflow is only required for PIP36 and not PIP.

PIP_REQUIREMENTS: |
    requests==2.21.0
    ipykernel==5.1.0
    ipython==7.4.0
    ipywidgets==7.4.2
    metakernel==0.20.14
    mock==2.0.0
    notebook==5.7.8
    numpy==1.16.2
    pandas==0.24.2
    PyYAML==5.1
    scikit-learn==0.20.3
    scipy==1.2.1
    uproot==3.4.18
  PIP36_REQUIREMENTS: |
    Cython==0.29.16
    seaborn==0.9.0
    sklearn-evaluation==0.4
    Keras==2.2.4
    tensorflow==1.13.1
    xgboost==0.82
    dryable==1.0.3
    responses==0.10.6
    RootInteractive==0.0.10

What I did (again: a while ago) was to simply remove tensorflow from the list. For me it worked because this package didn’t seem to be crucial.

No guarantees but maybe it works.

Cheers,
Thomas

Dear Thomas,

I just removed the tensorflow but then get an other error:

==> Building Python-modules@1.0
==> Python-modules is being built (use --debug for full output): failed
ERROR: Error while executing /Users/germain/alicesw/sw/SPECS/osx_x86-64/Python-modules/1.0-2/build.sh on `nanpc511.in2p3.fr’.
ERROR: Log can be found in /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log
ERROR: Please upload it to CERNBox/Dropbox if you intend to request support.
ERROR: Build directory is /Users/germain/alicesw/sw/BUILD/Python-modules-latest/Python-modules.
germain@MacBook-Pro-de-Service-5 alisw % more /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log

then

Collecting qgrid
Downloading qgrid-1.3.1.tar.gz (889 kB)
ERROR: Could not find a version that satisfies the requirement request (from RootInteractive==0.0.10->-r requirements.txt (line 21)) (from versions: none)
ERROR: No matching distribution found for request (from RootInteractive==0.0.10->-r requirements.txt (line 21))

So it failed further seems something is missing.

Best regards
Marie

Ok so the tensorflow problem seems to be “solved”. I would also remove RootInteractive from the list (last item) and check if everything else compiles. I assume RootInteractive as well as tensorflow is not needed at the moment for most users.

No guarantee again, that it will solve the issue but it is for sure worth a try.

Cheers,
Thomas

Hi all,

well removing the RootInteractive still not building properly. Latest log error still while building Python-modules@1.0

+++ python3 -c ‘import distutils.sysconfig; print(distutils.sysconfig.get_python_version())’
++ export PYVER=3.8
++ PYVER=3.8
++ unset PYTHONPATH
++ echo requests==2.21.0 ipykernel==5.1.0 ipython==7.4.0 ipywidgets==7.4.2 metakernel==0.20.14 mock==2.0.0 notebook==5.7.8 numpy==1.16.2 pandas==0.24.2 PyYAML==5.1 scikit-learn==0.20.3 scipy==1.2.1 uproot==3.4.18
++ tr ’ ’ ‘\n’
++ python3 -c ‘import sys; exit(0 if 1000sys.version_info.major + sys.version_info.minor >= 3006 else 1)’
++ [[ osx_x86-64 != slc6
]]
++ echo Cython==0.29.16 seaborn==0.9.0 sklearn-evaluation==0.4 Keras==2.2.4 xgboost==0.82 dryable==1.0.3 responses==0.10.6
++ tr ’ ’ ‘\n’
++ PYTHON_MODULES_INSTALLROOT=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules
++ mkdir -p /Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules
++ env PYTHONUSERBASE=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules pip3 install --user -IU setuptools
Collecting setuptools
Using cached setuptools-49.2.0-py3-none-any.whl (789 kB)
Installing collected packages: setuptools
WARNING: The scripts easy_install and easy_install-3.8 are installed in ‘/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules/bin’ which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed setuptools-49.2.0
++ grep RootInteractive requirements.txt
++ grep RootInteractive requirements.txt
++ env PYTHONUSERBASE=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules pip3 install --user -IU -r requirements.txt
Collecting requests==2.21.0
Using cached requests-2.21.0-py2.py3-none-any.whl (57 kB)
Collecting ipykernel==5.1.0
Using cached ipykernel-5.1.0-py3-none-any.whl (113 kB)
Collecting ipython==7.4.0
Using cached ipython-7.4.0-py3-none-any.whl (769 kB)
Collecting ipywidgets==7.4.2
Using cached ipywidgets-7.4.2-py2.py3-none-any.whl (111 kB)
Collecting metakernel==0.20.14
Using cached metakernel-0.20.14-py2.py3-none-any.whl (100 kB)
Collecting mock==2.0.0
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting notebook==5.7.8
Using cached notebook-5.7.8-py2.py3-none-any.whl (9.0 MB)
Collecting numpy==1.16.2
Using cached numpy-1.16.2.zip (5.1 MB)
Collecting pandas==0.24.2
Using cached pandas-0.24.2.tar.gz (11.8 MB)
Collecting PyYAML==5.1
Using cached PyYAML-5.1.tar.gz (274 kB)
Collecting scikit-learn==0.20.3
Using cached scikit-learn-0.20.3.tar.gz (11.8 MB)
Collecting scipy==1.2.1
Using cached scipy-1.2.1.tar.gz (23.1 MB)
Collecting uproot==3.4.18
Using cached uproot-3.4.18-py2.py3-none-any.whl (124 kB)
Collecting Cython==0.29.16
Using cached Cython-0.29.16-cp38-cp38-macosx_10_9_x86_64.whl (2.0 MB)
Collecting seaborn==0.9.0
Using cached seaborn-0.9.0-py3-none-any.whl (208 kB)
Collecting sklearn-evaluation==0.4
Using cached sklearn-evaluation-0.4.tar.gz (18 kB)
Collecting Keras==2.2.4
Using cached Keras-2.2.4-py2.py3-none-any.whl (312 kB)
Collecting xgboost==0.82
Using cached xgboost-0.82.tar.gz (665 kB)
Collecting dryable==1.0.3
Using cached dryable-1.0.3-py3-none-any.whl (1.8 kB)
Collecting responses==0.10.6
Using cached responses-0.10.6-py2.py3-none-any.whl (13 kB)
Collecting chardet<3.1.0,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3<1.25,>=1.21.1
Using cached urllib3-1.24.3-py2.py3-none-any.whl (118 kB)
Collecting idna<2.9,>=2.5
Using cached idna-2.8-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting tornado>=4.2
Using cached tornado-6.0.4.tar.gz (496 kB)
Collecting jupyter-client
Using cached jupyter_client-6.1.6-py3-none-any.whl (108 kB)
Collecting traitlets>=4.1.0
Using cached traitlets-4.3.3-py2.py3-none-any.whl (75 kB)
Collecting pexpect; sys_platform != “win32”
Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting appnope; sys_platform == “darwin”
Using cached appnope-0.1.0-py2.py3-none-any.whl (4.0 kB)
Collecting jedi>=0.10
Using cached jedi-0.17.1-py2.py3-none-any.whl (1.4 MB)
Collecting prompt-toolkit<2.1.0,>=2.0.0
Using cached prompt_toolkit-2.0.10-py3-none-any.whl (340 kB)
Collecting pygments
Using cached Pygments-2.6.1-py3-none-any.whl (914 kB)
Collecting setuptools>=18.5
Using cached setuptools-49.2.0-py3-none-any.whl (789 kB)
Collecting backcall
Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting decorator
Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting widgetsnbextension~=3.4.0
Using cached widgetsnbextension-3.4.2-py2.py3-none-any.whl (2.2 MB)
Collecting nbformat>=4.2.0
Using cached nbformat-5.0.7-py3-none-any.whl (170 kB)
Collecting pbr>=0.11
Using cached pbr-5.4.5-py2.py3-none-any.whl (110 kB)
Collecting six>=1.9
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting nbconvert
Using cached nbconvert-5.6.1-py2.py3-none-any.whl (455 kB)
Collecting pyzmq>=17
Using cached pyzmq-19.0.1-cp38-cp38-macosx_10_9_x86_64.whl (820 kB)
Collecting jupyter-core>=4.4.0
Using cached jupyter_core-4.6.3-py2.py3-none-any.whl (83 kB)
Collecting ipython-genutils
Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting jinja2
Using cached Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting prometheus-client
Using cached prometheus_client-0.8.0-py2.py3-none-any.whl (53 kB)
Collecting terminado>=0.8.1
Using cached terminado-0.8.3-py2.py3-none-any.whl (33 kB)
Collecting Send2Trash
Using cached Send2Trash-1.5.0-py3-none-any.whl (12 kB)
Collecting python-dateutil>=2.5.0
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pytz>=2011k
Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB)
Collecting uproot-methods>=0.4.0
Using cached uproot_methods-0.7.4-py2.py3-none-any.whl (32 kB)
Collecting awkward>=0.8.0
Using cached awkward-0.12.22-py2.py3-none-any.whl (87 kB)
Collecting cachetools
Using cached cachetools-4.1.1-py3-none-any.whl (10 kB)
Collecting matplotlib>=1.4.3
Using cached matplotlib-3.3.0-cp38-cp38-macosx_10_9_x86_64.whl (11.4 MB)
Collecting keras-preprocessing>=1.0.5
Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting keras-applications>=1.0.6
Using cached Keras_Applications-1.0.8-py3-none-any.whl (50 kB)
Collecting h5py
Using cached h5py-2.10.0-cp38-cp38-macosx_10_9_x86_64.whl (3.0 MB)
Collecting ptyprocess>=0.5
Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting parso<0.8.0,>=0.7.0
Downloading parso-0.7.0-py2.py3-none-any.whl (100 kB)
Collecting wcwidth
Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting jsonschema!=2.5.0,>=2.4
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting defusedxml
Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Collecting bleach
Downloading bleach-3.1.5-py2.py3-none-any.whl (151 kB)
Collecting mistune<2,>=0.8.1
Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting testpath
Downloading testpath-0.4.4-py2.py3-none-any.whl (163 kB)
Collecting entrypoints>=0.2.2
Downloading entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting pandocfilters>=1.4.1
Downloading pandocfilters-1.4.2.tar.gz (14 kB)
Collecting MarkupSafe>=0.23
Downloading MarkupSafe-1.1.1-cp38-cp38-macosx_10_9_x86_64.whl (16 kB)
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.2.0-cp38-cp38-macosx_10_9_x86_64.whl (60 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pillow>=6.2.0
Downloading Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl (2.2 MB)
Collecting cycler>=0.10
Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting attrs>=17.4.0
Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting pyrsistent>=0.14.0
Downloading pyrsistent-0.16.0.tar.gz (108 kB)
Collecting packaging
Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting webencodings
Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: numpy, pandas, PyYAML, scikit-learn, scipy, sklearn-evaluation, xgboost, tornado, pandocfilters, pyrsistent
Building wheel for numpy (setup.py): started
Building wheel for numpy (setup.py): still running…
Building wheel for numpy (setup.py): finished with status ‘done’
Created wheel for numpy: filename=numpy-1.16.2-cp38-cp38-macosx_10_15_x86_64.whl size=3454827 sha256=22a01129b333109e147abec72ac86a38d171a4a06ebbb5153b70c2d60d7f8622
Stored in directory: /Users/germain/Library/Caches/pip/wheels/c7/1a/26/d24add95f4ea99e790c3a8b339fb5b46b66f9849142d1b93c5
Building wheel for pandas (setup.py): started
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): finished with status ‘error’
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python@3.8/bin/python3.8 -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/setup.py’"’"’; file=’"’"’/private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/setup.py’"’"’;f=getattr(tokenize, ‘"’"‘open’"’"’, open)(file);code=f.read().replace(’"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, file, ‘"’"‘exec’"’"’))’ bdist_wheel -d /private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-wheel-59_o3o0v
cwd: /private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/
Complete output (3786 lines):
running bdist_wheel
running build
running build_py
creating build

I followed the doc for installation ans prerequesite but it sounds still something not OK in the installation.
Thanks a lot for your patience and inputs
Best Marie

Looks like the pandas package cannot be installed for some reason.
I am out of ideas. We need to wait for an expert opinion, I’m afraid.

Cheers,
Thomas

Thanks a lot Thomas for your help.
Lets hope some suggestions.
Best Marie