Cannot build O2 failing Python-modules@1.0

O²
1

Hi

I am trying to build O2 on MAC OS (Catalina 10.15.4)
and it fails in Python-modules@1.0

ERROR: Error while executing /Users/germain/alicesw/sw/SPECS/osx_x86-64/Python-modules/1.0-1/build.sh on `nanpc511.in2p3.fr’.
ERROR: Log can be found in /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log
ERROR: Please upload it to CERNBox/Dropbox if you intend to request support.
ERROR: Build directory is /Users/germain/alicesw/sw/BUILD/Python-modules-latest/Python-modules.
germain@nanpc511 alisw %

the logs indicates error

Collecting Keras==2.2.4
Using cached Keras-2.2.4-py2.py3-none-any.whl (312 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow==1.13.1 (from -r requirements.txt (line 18)) (from versions: 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1)
ERROR: No matching distribution found for tensorflow==1.13.1 (from -r requirements.txt (line 18))

I use python 3.8 and it’s the first trial of installation I do on this labtop (no previous version);

Thanks in advance for any help
Best Marie

2

Hi,

I have checked on my setup and I don’t have the module tensorflow installed by pip. However I use the defaults o2-dataflow and that might be the reason why.

Cheers,
Barth

3

Hi,

I had a similar problem quite a while ago. Back then there was a list of python modules in alidist/python-modules.sh, which now seems to be in alidist/python-modules-list.sh.

Apparently tensorflow is only required for PIP36 and not PIP.

PIP_REQUIREMENTS: |
    requests==2.21.0
    ipykernel==5.1.0
    ipython==7.4.0
    ipywidgets==7.4.2
    metakernel==0.20.14
    mock==2.0.0
    notebook==5.7.8
    numpy==1.16.2
    pandas==0.24.2
    PyYAML==5.1
    scikit-learn==0.20.3
    scipy==1.2.1
    uproot==3.4.18
  PIP36_REQUIREMENTS: |
    Cython==0.29.16
    seaborn==0.9.0
    sklearn-evaluation==0.4
    Keras==2.2.4
    tensorflow==1.13.1
    xgboost==0.82
    dryable==1.0.3
    responses==0.10.6
    RootInteractive==0.0.10

What I did (again: a while ago) was to simply remove tensorflow from the list. For me it worked because this package didn’t seem to be crucial.

No guarantees but maybe it works.

Cheers,
Thomas

4

Dear Thomas,

I just removed the tensorflow but then get an other error:

==> Building Python-modules@1.0
==> Python-modules is being built (use --debug for full output): failed
ERROR: Error while executing /Users/germain/alicesw/sw/SPECS/osx_x86-64/Python-modules/1.0-2/build.sh on `nanpc511.in2p3.fr’.
ERROR: Log can be found in /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log
ERROR: Please upload it to CERNBox/Dropbox if you intend to request support.
ERROR: Build directory is /Users/germain/alicesw/sw/BUILD/Python-modules-latest/Python-modules.
germain@MacBook-Pro-de-Service-5 alisw % more /Users/germain/alicesw/sw/BUILD/Python-modules-latest/log

then

Collecting qgrid
Downloading qgrid-1.3.1.tar.gz (889 kB)
ERROR: Could not find a version that satisfies the requirement request (from RootInteractive==0.0.10->-r requirements.txt (line 21)) (from versions: none)
ERROR: No matching distribution found for request (from RootInteractive==0.0.10->-r requirements.txt (line 21))

So it failed further seems something is missing.

Best regards
Marie

5

Ok so the tensorflow problem seems to be “solved”. I would also remove RootInteractive from the list (last item) and check if everything else compiles. I assume RootInteractive as well as tensorflow is not needed at the moment for most users.

No guarantee again, that it will solve the issue but it is for sure worth a try.

Cheers,
Thomas

6

Hi all,

well removing the RootInteractive still not building properly. Latest log error still while building Python-modules@1.0

+++ python3 -c ‘import distutils.sysconfig; print(distutils.sysconfig.get_python_version())’
++ export PYVER=3.8
++ PYVER=3.8
++ unset PYTHONPATH
++ echo requests==2.21.0 ipykernel==5.1.0 ipython==7.4.0 ipywidgets==7.4.2 metakernel==0.20.14 mock==2.0.0 notebook==5.7.8 numpy==1.16.2 pandas==0.24.2 PyYAML==5.1 scikit-learn==0.20.3 scipy==1.2.1 uproot==3.4.18
++ tr ’ ’ ‘\n’
++ python3 -c ‘import sys; exit(0 if 1000sys.version_info.major + sys.version_info.minor >= 3006 else 1)’
++ [[ osx_x86-64 != slc6 ]]
++ echo Cython==0.29.16 seaborn==0.9.0 sklearn-evaluation==0.4 Keras==2.2.4 xgboost==0.82 dryable==1.0.3 responses==0.10.6
++ tr ’ ’ ‘\n’
++ PYTHON_MODULES_INSTALLROOT=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules
++ mkdir -p /Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules
++ env PYTHONUSERBASE=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules pip3 install --user -IU setuptools
Collecting setuptools
Using cached setuptools-49.2.0-py3-none-any.whl (789 kB)
Installing collected packages: setuptools
WARNING: The scripts easy_install and easy_install-3.8 are installed in ‘/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules/bin’ which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed setuptools-49.2.0
++ grep RootInteractive requirements.txt
++ grep RootInteractive requirements.txt
++ env PYTHONUSERBASE=/Users/germain/alicesw/sw/INSTALLROOT/502b8008f2986ddb35aeb72bf29cb907c3686d35/osx_x86-64/Python-modules/1.0-2/share/python-modules pip3 install --user -IU -r requirements.txt
Collecting requests==2.21.0
Using cached requests-2.21.0-py2.py3-none-any.whl (57 kB)
Collecting ipykernel==5.1.0
Using cached ipykernel-5.1.0-py3-none-any.whl (113 kB)
Collecting ipython==7.4.0
Using cached ipython-7.4.0-py3-none-any.whl (769 kB)
Collecting ipywidgets==7.4.2
Using cached ipywidgets-7.4.2-py2.py3-none-any.whl (111 kB)
Collecting metakernel==0.20.14
Using cached metakernel-0.20.14-py2.py3-none-any.whl (100 kB)
Collecting mock==2.0.0
Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB)
Collecting notebook==5.7.8
Using cached notebook-5.7.8-py2.py3-none-any.whl (9.0 MB)
Collecting numpy==1.16.2
Using cached numpy-1.16.2.zip (5.1 MB)
Collecting pandas==0.24.2
Using cached pandas-0.24.2.tar.gz (11.8 MB)
Collecting PyYAML==5.1
Using cached PyYAML-5.1.tar.gz (274 kB)
Collecting scikit-learn==0.20.3
Using cached scikit-learn-0.20.3.tar.gz (11.8 MB)
Collecting scipy==1.2.1
Using cached scipy-1.2.1.tar.gz (23.1 MB)
Collecting uproot==3.4.18
Using cached uproot-3.4.18-py2.py3-none-any.whl (124 kB)
Collecting Cython==0.29.16
Using cached Cython-0.29.16-cp38-cp38-macosx_10_9_x86_64.whl (2.0 MB)
Collecting seaborn==0.9.0
Using cached seaborn-0.9.0-py3-none-any.whl (208 kB)
Collecting sklearn-evaluation==0.4
Using cached sklearn-evaluation-0.4.tar.gz (18 kB)
Collecting Keras==2.2.4
Using cached Keras-2.2.4-py2.py3-none-any.whl (312 kB)
Collecting xgboost==0.82
Using cached xgboost-0.82.tar.gz (665 kB)
Collecting dryable==1.0.3
Using cached dryable-1.0.3-py3-none-any.whl (1.8 kB)
Collecting responses==0.10.6
Using cached responses-0.10.6-py2.py3-none-any.whl (13 kB)
Collecting chardet<3.1.0,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3<1.25,>=1.21.1
Using cached urllib3-1.24.3-py2.py3-none-any.whl (118 kB)
Collecting idna<2.9,>=2.5
Using cached idna-2.8-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting tornado>=4.2
Using cached tornado-6.0.4.tar.gz (496 kB)
Collecting jupyter-client
Using cached jupyter_client-6.1.6-py3-none-any.whl (108 kB)
Collecting traitlets>=4.1.0
Using cached traitlets-4.3.3-py2.py3-none-any.whl (75 kB)
Collecting pexpect; sys_platform != “win32”
Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting appnope; sys_platform == “darwin”
Using cached appnope-0.1.0-py2.py3-none-any.whl (4.0 kB)
Collecting jedi>=0.10
Using cached jedi-0.17.1-py2.py3-none-any.whl (1.4 MB)
Collecting prompt-toolkit<2.1.0,>=2.0.0
Using cached prompt_toolkit-2.0.10-py3-none-any.whl (340 kB)
Collecting pygments
Using cached Pygments-2.6.1-py3-none-any.whl (914 kB)
Collecting setuptools>=18.5
Using cached setuptools-49.2.0-py3-none-any.whl (789 kB)
Collecting backcall
Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting decorator
Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting widgetsnbextension~=3.4.0
Using cached widgetsnbextension-3.4.2-py2.py3-none-any.whl (2.2 MB)
Collecting nbformat>=4.2.0
Using cached nbformat-5.0.7-py3-none-any.whl (170 kB)
Collecting pbr>=0.11
Using cached pbr-5.4.5-py2.py3-none-any.whl (110 kB)
Collecting six>=1.9
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting nbconvert
Using cached nbconvert-5.6.1-py2.py3-none-any.whl (455 kB)
Collecting pyzmq>=17
Using cached pyzmq-19.0.1-cp38-cp38-macosx_10_9_x86_64.whl (820 kB)
Collecting jupyter-core>=4.4.0
Using cached jupyter_core-4.6.3-py2.py3-none-any.whl (83 kB)
Collecting ipython-genutils
Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting jinja2
Using cached Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting prometheus-client
Using cached prometheus_client-0.8.0-py2.py3-none-any.whl (53 kB)
Collecting terminado>=0.8.1
Using cached terminado-0.8.3-py2.py3-none-any.whl (33 kB)
Collecting Send2Trash
Using cached Send2Trash-1.5.0-py3-none-any.whl (12 kB)
Collecting python-dateutil>=2.5.0
Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting pytz>=2011k
Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB)
Collecting uproot-methods>=0.4.0
Using cached uproot_methods-0.7.4-py2.py3-none-any.whl (32 kB)
Collecting awkward>=0.8.0
Using cached awkward-0.12.22-py2.py3-none-any.whl (87 kB)
Collecting cachetools
Using cached cachetools-4.1.1-py3-none-any.whl (10 kB)
Collecting matplotlib>=1.4.3
Using cached matplotlib-3.3.0-cp38-cp38-macosx_10_9_x86_64.whl (11.4 MB)
Collecting keras-preprocessing>=1.0.5
Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting keras-applications>=1.0.6
Using cached Keras_Applications-1.0.8-py3-none-any.whl (50 kB)
Collecting h5py
Using cached h5py-2.10.0-cp38-cp38-macosx_10_9_x86_64.whl (3.0 MB)
Collecting ptyprocess>=0.5
Downloading ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting parso<0.8.0,>=0.7.0
Downloading parso-0.7.0-py2.py3-none-any.whl (100 kB)
Collecting wcwidth
Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting jsonschema!=2.5.0,>=2.4
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting defusedxml
Downloading defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Collecting bleach
Downloading bleach-3.1.5-py2.py3-none-any.whl (151 kB)
Collecting mistune<2,>=0.8.1
Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting testpath
Downloading testpath-0.4.4-py2.py3-none-any.whl (163 kB)
Collecting entrypoints>=0.2.2
Downloading entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting pandocfilters>=1.4.1
Downloading pandocfilters-1.4.2.tar.gz (14 kB)
Collecting MarkupSafe>=0.23
Downloading MarkupSafe-1.1.1-cp38-cp38-macosx_10_9_x86_64.whl (16 kB)
Collecting kiwisolver>=1.0.1
Downloading kiwisolver-1.2.0-cp38-cp38-macosx_10_9_x86_64.whl (60 kB)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pillow>=6.2.0
Downloading Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl (2.2 MB)
Collecting cycler>=0.10
Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting attrs>=17.4.0
Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting pyrsistent>=0.14.0
Downloading pyrsistent-0.16.0.tar.gz (108 kB)
Collecting packaging
Downloading packaging-20.4-py2.py3-none-any.whl (37 kB)
Collecting webencodings
Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: numpy, pandas, PyYAML, scikit-learn, scipy, sklearn-evaluation, xgboost, tornado, pandocfilters, pyrsistent
Building wheel for numpy (setup.py): started
Building wheel for numpy (setup.py): still running…
Building wheel for numpy (setup.py): finished with status ‘done’
Created wheel for numpy: filename=numpy-1.16.2-cp38-cp38-macosx_10_15_x86_64.whl size=3454827 sha256=22a01129b333109e147abec72ac86a38d171a4a06ebbb5153b70c2d60d7f8622
Stored in directory: /Users/germain/Library/Caches/pip/wheels/c7/1a/26/d24add95f4ea99e790c3a8b339fb5b46b66f9849142d1b93c5
Building wheel for pandas (setup.py): started
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): still running…
Building wheel for pandas (setup.py): finished with status ‘error’
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python@3.8/bin/python3.8 -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/setup.py’“'”‘; file=’“'”‘/private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/setup.py’“'”‘;f=getattr(tokenize, ‘"’“‘open’”’“‘, open)(file);code=f.read().replace(’”‘"’\r\n’“'”‘, ‘"’"’\n’“'”‘);f.close();exec(compile(code, file, ‘"’“‘exec’”’"‘))’ bdist_wheel -d /private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-wheel-59_o3o0v
cwd: /private/var/folders/f5/7gzg0ws9097fvdn5h7q62p2h0000gp/T/pip-install-vbdcgjr0/pandas/
Complete output (3786 lines):
running bdist_wheel
running build
running build_py
creating build

I followed the doc for installation ans prerequesite but it sounds still something not OK in the installation.
Thanks a lot for your patience and inputs
Best Marie

7

Looks like the pandas package cannot be installed for some reason.
I am out of ideas. We need to wait for an expert opinion, I’m afraid.

Cheers,
Thomas

8

Thanks a lot Thomas for your help.
Lets hope some suggestions.
Best Marie