When I run O2 workflows in a shell script, it works as usual. However, when I submit a batch job with the same script on HTCondor we maintain for my lab, it fails to connect to CCDB. It should be technically possible to run O2 with HTCondor as we do on the grid.
Is there anyone using HTCondor locally to execute O2 workflows? If any, would you share tips?
Do you pass the proper environment to HTCondor? I remember struggling a lot with this when I tried setting this up. HTCondor does not simply transfer the environment of alienv when you set getenv = true in your submit file. I am unsure as to why it does not, but I found a workaround.
First I print the environment of alienv into a .sh file which I then tell HTCondor to transfer. This file should be sourced in your executable.
I named the file that prints the environment “prepare_env.sh” and it looks like this
To unblock you with this, I think that one of the following approaches might work:
You initialize a GRID token on a machine where your GRID certificate is installed (typically you should have a file ~/.globus/usercert.pem). This can be done by typing alien.py ls which demands a password. Upon successful execution, there will be files tokencert_XXX.pem and tokenkey_XXX.pem somewhere under /tmp/.
(Option 1) – You simply copy/sync those files to the HTCondor nodes, where the job will be run. This may be possible via a shared file system or not.
(Option 2) – You export the content of these files as environment variables within your script submitted to HTCondor (not discussing security here):
export JALIEN_TOKEN_CERT='CONTENT OF TOKENCERT FILE'
export JALIEN_TOKEN_KEY='CONTENT OF TOKENKEY FILE'
With these methods, a simple call to alien-token-info should succeed and show the token information.
Sandro is right, with your full Grid certificate any alien.py connection will first create these token*.pem files in your tmp directory.
In addition to that you can also call a token alien.py command to print the same content on screen. To enhance the security, try something like alien.py token -v 2 to only give a 2 day validity to the generated identity. Better have a very limited validity for them and if needed overwrite them tomorrow with another fresh, short lived, pair.
Then indeed, either take the two respective contents and export them in the environment of the job, or set those environment variables to the path to those token files somewhere in your own space on the shared file system (ideally all nodes would see the same location).