Alien.py fails

I see the alien.py fails. The operating system is MAC MONTEREY, M1-chip. Any help is appreciated.
Thanks in advance.

Enter PEM pass phrase:
Could NOT establish connection (WebSocket) to ::ffff:137.138.99.139:8097
SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)’)
Check the logfile: /Users/arvindkhuntia/alien_py.log
Enter PEM pass phrase:
Could NOT establish connection (WebSocket) to ::ffff:137.138.99.140:8097
SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)’)

Hi! it would seem that you have a problem with your certificate and/or the available collection of authority certificates.
Send to me the following:

  1. alien.py version
  2. ALIENPY_DEBUG=1 ALIENPY_DEBUG_FILE=log.txt alien.py pwd and send me the log.txt
  3. alien.py cert-info
  4. alien.py cert-verify

log.txt (3.2 KB)

1: alien.py version

alien.py version: 1.4.5

alien.py version date: 20220927_120245

alien.py version hash: cd50178

alien.py location: /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/xjalienfs/1.4.5-local1/lib/python/site-packages/alienpy/alien.py

script location: /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/xjalienfs/1.4.5-local1/bin/alien.py

Interpreter: /opt/homebrew/Cellar/python@3.10/3.10.7/Frameworks/Python.framework/Versions/3.10/bin/python3.10

Python version: 3.10.7 (main, Sep 14 2022, 22:38:23) [Clang 14.0.0 (clang-1400.0.29.102)]

XRootD version: 5.5.0

XRootD path: /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/XRootD/v5.5.0-local1/lib/python/site-packages/XRootD/client/init.py

2: ALIENPY_DEBUG=1 ALIENPY_DEBUG_FILE=log.txt alien.py pwd

(log.txt attached)

3: alien.py cert-info
DN >>> DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=akhuntia/CN=788311/CN=Arvind Khuntia

ISSUER >>> DC=ch/DC=cern/CN=CERN Grid Certification Authority

BEGIN >>> 2022-04-11 08:49:41

EXPIRE >>> 2023-04-22 11:20:16

4: alien.py cert-verify

SSL Verification failed for /Users/arvindkhuntia/.globus/usercert.pem

@asevcenc Please have a look.

@akhuntia could you check if CA CERN certs are present:
ls -1 /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA*

/Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA.crl_url
/Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA.info
/Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA.pem
/Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA.signing_policy

@akhuntia so, at this point i suspect that you have a network related problem…
if on lxplus with the same user certificates you can do:
/cvmfs/alice.cern.ch/bin/alienv enter xjalienfs
then
alien.py pwd
then both your user certificates and alien.py are ok
and the problem is with your local connection … i suspect that a transparent proxy or vpn interfere with your connection towards CERN…
Could you please confirm that on lxplus things work?
Thanks a lot!

also try by hand:

openssl verify -CApath /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates $HOME/.globus/usercert.pem

yes, it is working on lxplus

openssl verify -CApath /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates $HOME/.globus/usercert.pem

/AliEn-Runtime/v2-19-le-local1/globus/share/certificates $HOME/.globus/usercert.pem
/Users/arvindkhuntia/.globus/usercert.pem: DC = ch, DC = cern, OU = Organic Units, OU = Users, CN = akhuntia, CN = 788311, CN = Arvind Khuntia
error 20 at 0 depth lookup:unable to get local issuer certificate
4372891180:error:09FFF06C:PEM routines:CRYPTO_internal:no start line:/AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/libressl/libressl-2.8/crypto/pem/pem_lib.c:684:Expecting: TRUSTED CERTIFICATE
4372891180:error:0BFFF009:x509 certificate routines:CRYPTO_internal:PEM lib:/AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/libressl/libressl-2.8/crypto/x509/by_file.c:146:

could you check that the same certificates are on both machines?
just run on lxplus and on your machine:

sha256sum $HOME/.globus/user{cert,key}.pem

and compare the outputs

On lxplus:
[xjalienfs] FDDEPNDATA > sha256sum $HOME/.globus/user{cert,key}.pem
b66eb01a1e38deeaf70534763433dc748b4b8e245de0e91d721a9f7eeb988610 /afs/cern.ch/user/a/akhuntia/.globus/usercert.pem
8c5e77769def35a9905504fa9e0802e9f2aa44726c3295610d10476481e49f66 /afs/cern.ch/user/a/akhuntia/.globus/userkey.pem

On my laptop:
[O2/latest] ~/alice/alisw/TestO2/EPN_data_may2022/lhcBackground %> sha256sum $HOME/.globus/user{cert,key}.pem
b66eb01a1e38deeaf70534763433dc748b4b8e245de0e91d721a9f7eeb988610 /Users/arvindkhuntia/.globus/usercert.pem
8c5e77769def35a9905504fa9e0802e9f2aa44726c3295610d10476481e49f66 /Users/arvindkhuntia/.globus/userkey.pem

then compare the hash of CERN-GridCA.pem, in both environments (on your O2/latest on mac and in xjalienfs on lxplus) check:
sha256sum $X509_CERT_DIR/CERN-GridCA.pem
other than this i ran out of ideas… but it’s clear that something is corrupted on your laptop,
and it might be a solution to delete everything under $ALIBUILD_WORK_DIR (except MIRROR) and try again the build

lxplus:
[xjalienfs] FDDEPNDATA > sha256sum $X509_CERT_DIR/CERN-GridCA.pem
b192dfb7e84a523441c2a10dce555f5e2577973883d944cd5d14a6f4c76b166c /cvmfs/alice.cern.ch/el7-x86_64/Packages/AliEn-Runtime/v2-19-le-113/globus/share/certificates/CERN-GridCA.pem

Local:
[O2/latest] ~/alice/alisw/TestO2/EPN_data_may2022/lhcBackground %> sha256sum $X509_CERT_DIR/CERN-GridCA.pem
b192dfb7e84a523441c2a10dce555f5e2577973883d944cd5d14a6f4c76b166c /Users/arvindkhuntia/alice/alisw/sw/osx_arm64/AliEn-Runtime/v2-19-le-local1/globus/share/certificates/CERN-GridCA.pem

I’m sorry, i’m out of ideas…

  1. the most basic test for cert verification should work (within the respective environment where X509_CERT_DIR is defined and have the collection of CA certificates):
    openssl verify -CApath $X509_CERT_DIR $HOME/.globus/usercert.pem

  2. this assumes that the involved files (both user{cert,key}.pem and CERN-GridCA.pem) are valid, the the hash shown that the files are the same

  3. the basic validation tests on lxplus were successful

so, given that the files are the same, everything works on lxplus, but not on the laptop, i can only blame the network somehow… is the network CERN? if yes, then it cannot be the network (unless is an un-registered device), if no, then try to have the network access through another ISP and see if this change anything

Now, I am building O2 from scratch. Let’s hope, this will solve the problem.

Thank you very much for your input.

Still have the same issues with the new build.

Unfortunately I don’t know what to say… i’m out of ideas …

@akhuntia could you check if your machine have the time synchronized? try to set and enable network synchronization for time like this: If the date or time is wrong on your Mac - Apple Support

@asevcenc Yes, the time is synchronised.