How to get O2 and QC on Ubuntu without the QCG?

O²
1

Dear all,

we currently have issues getting the newest versions of the software compiled on Ubuntu 18.04. We had found some time ago, that building with

aliBuild build O2Suite --defaults o2

does not work any longer due to incompatibilities with some of the readout packages. I also tried just now, and it failed in ReadoutCard. As we for the QC development do not need those, we switched to using

aliBuild build qcg --defaults o2

which was working fine for quite some time now. Unfortunately, now the QCG has been separated from the O2 software packages and the qcg.sh completely removed from alidist. Running the above command now leads just to

ERROR: [Errno 2] No such file or directory: u'alidist/qcg.sh'

which is obvious, as QCG is no longer there. As we want to be able to run some simulations for testing purposes, building with the defaults o2-dataflow is not an option.

Building only O2 with

aliBuild build O2 --defaults o2

is still running fine, but does not help, as we need QualityControl.

If this is of any relevance, my current aliBuild version is 1.8.0.

In conclusion, I have the question, how to get the full software including QualityControl and the simulation packages (but not necessarily the readout packages) on Ubuntu 18.

Thanks a lot for any help!

Cheers,
Stefan

PS: I am aware, that Ubuntu is not an officially supported platform, but from my experience and the topics here in ALICE talk, I think it is widely used, so would be really nice to get it working! :slightly_smiling_face:

2

Hi,

We are not 100% sure if you mean QualityControl or QCG.

If you want to install QC you can do aliBuild build QualityControl --defaults o2 and you will get O2 as well because it is a dependency of QC. When you load with alienv enter QualityControl/latest you will see it.

If you want to install QCG as well, then you are correct that the recipe has disappeared because it does not depend anymore on QC. The instructions to install are here: WebUi/QualityControl at dev · AliceO2Group/WebUi · GitHub
Do you think that it is ok to install this way ? or is it a problem ? We are eager to hear your opinion.

Cheers,
Barth

PS: I realize now that the instructions in the QC repository still mention the recipe for qcg and I will fix it.

3

Hi Barth,

thanks a lot for your quick reply!

The more important point is to get QualityControl, but for some tests we also have used a local instance of the QCG already, so in the end it would be good to have both.

I was not aware of the possibility to build QualityControl instead of qcg, it indeed would be good, if you could add this to the instructions. I tried it now and it worked, so we have again a running and up-to-date version of QualityControl on Ubuntu!

I will try the installation of the QCG later and report back here.

Cheers,
Stefan

4

Hi Barth,

I tried the new way to install the QCG. The installation went fine, and I am also able to run in parallel a local instance of the CCDB, the QCG and then a QC task. After fixing the path of the plots in the config.js I also get the correct tree (e.g. qc/TPC/MO/Tracks ) in my browser on localhost:8081 (I use port 8080 for the CCDB and 8081 for the QCG).

But unfortunately, the histograms cannot be displayed. It seems, that the QCG always looks for an object with a timestamp at the exact moment of looking up this object - but as the objects have been produced before, it cannot find anything.

E.g. in /tmp/QC/qc/TPC/MO/Tracks/hEta I have the following dirs:

1618852668163  1618852678065  1618852688071  1618852698079  1618852698918

And when I try to open this histogram in the browser, with few attempts in a row, I get:

2021-04-19T17:18:43.737Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1618852723726
2021-04-19T17:18:44.178Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1618852724173
2021-04-19T17:18:44.477Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1618852724469
2021-04-19T17:18:44.764Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1618852724758

So it seems, that it always looks for an object with the timestamp at the moment of clicking.

Am I doing something wrong or is this a bug?

If it helps: I am not running in online mode, and besides the warning

2021-04-19T17:17:10.484Z  [QualityControl/QCModel] warn: Consul Service: No Configuration Found

which is mentioned on the QCG installation guide, I get an error:

2021-04-19T17:18:35.241Z  [QualityControl/Api] error: Online mode is not enabled due to missing Consul configuration

But as far as I understand, this should not be an issue.

Cheers,
Stefan

5

Hi Stefan,

I am calling my colleague @graduta for help as I am not sure how the QCG queries the CCDB. I would have thought that it uses the command latest on the REST api but I might be mistaken.

Cheers,
Barth

6

Hi @sheckel,

First lets start with the Consul error. As Consul is an optional dependency the error message is there just to let us know that it was not configured. This will not impact in any way the QCG. I shall consider making it a warning.

Secondly, let me explain a little bit the behavior of QCG. @bvonhall is partially correct. We used to use latest when retrieving an object but this changed when we added the functionality to look back at other versions of the object. (Navigating the history of the object).

Now, in order to retrieve a certain version of an object (based on a timestamp), CCDB requires us to always send a timestamp. Moreover, CCDB also expects objects to be defined with a certain availability in the future. (Usually 10 years ahead). That is why we can simply pass the current timestamp and CCDB will retrieve us the latest version of the object unless the user selects a different timestamp from the dropdown list.

Moving on to the issue you described. You mention that you configured the path of the plots in config.js. Since the new version of QCG 2.x there is another field plotUrl that has to be configured. More info here: WebUi/QualityControl at dev · AliceO2Group/WebUi · GitHub

Could you please double check if you have done so? In your case, as you host CCDB locally it would be as simple as http://localhost:8080

Thank you,
George

7

Dear @graduta,

thanks a lot for your reply and explanations! I did change the path in plotUrl, this is actually what I meant with the changed path. Do I have to change this somewhere else?

I now rechecked, and indeed, the objects are correctly available in my local CCDB, e.g. for the hEta histogram this is

http://localhost:8080/browse/qc/TPC/MO/Tracks/hEta

Here I have an entry with the timestamp

1619199758546

Now, trying a moment later to access this file in the local QCG on

http://localhost:8081/?page=objectTree

I get Object 'qc/TPC/MO/Tracks/hEta' could not be loaded and in the terminal the qcg tells me (trying to access a few times):

2021-04-23T17:43:06.710Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1619199786705
2021-04-23T17:43:07.511Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1619199787505
2021-04-23T17:43:09.030Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1619199789025
2021-04-23T17:43:10.462Z  [Framework] debug: [HTTP] Page was not found: /tmp/QC/qc/TPC/MO/Tracks/hEta/1619199790457

So I would conclude, that the qcg always looks for the timestamp at the moment in time, when I am clicking, while the object has been created before, and thus the qcg does not find anything.

Cheers,
Stefan

8

Hi @sheckel ,

QCG is developed in such way so that the latest version of the plot is retrieved from CCDB. Thus, the current timestamp is sent on purpose. CCDB will take the request and send back the latest version available.
Would you mind posting the value of the ccdb: and http fields from your configuration file? I was not able to replicate the issue you are encountering even though I have a local QC and CCDB running.

Thank you,
George

9

Hi George,

sure, here they are:

  http: {
    port: 8081,
    // portSecure: 8443,
    hostname: 'localhost',
    // key: './cert/key.pem',
    // cert: './cert/cert.pem',
    tls: false
  },

  ccdb: {
    hostname: 'localhost',
    port: 8080,
    prefix: 'qc',
    // plotUrl: 'localhost/ccdb'
    plotUrl: 'tmp/QC'
  },
10

Hi @sheckel ,

Thank you for providing the configuration. Now it is clear why your objects were not found.

Please have a quick look on the instructions: WebUi/QualityControl at dev · AliceO2Group/WebUi · GitHub
The plotUrl has to be a valid URL. As per the example and based on the configuration you pasted, yours should look something like: http://localhost:8080
In this way, JSROOT will know on what machine and port to look for objects.

Please give it a try and let me know if it works. Moreover, please do let me know if the installation/setup instructions from the repository are not clear so that we can improve them.

Have a nice day,
George

11

Hi @graduta,

thanks a lot for your suggestion. I tried it, but unfortunately still do not get the histograms displayed. In the browser, I get the same error as before (“could not be loaded”), but in the terminal, the qcg itself does not give any errors anymore (except the consul ones). I tried for the plotUrl with http://localhost:8080 and just localhost:8080. Both cases show the same behaviour. If it maybe helps, the full output of the qcg is now:

2021-04-27T10:37:18.745Z  debug: Created default instance of console logger
2021-04-27T10:37:18.839Z  [QualityControl/ConfigProvider] info: Read config file "/home/sheckel/WebUi/QualityControl/config.js"
2021-04-27T10:37:18.841Z  [QualityControl/QCModel] warn: Consul Service: No Configuration Found
2021-04-27T10:37:18.841Z  [QualityControl/QCModel] info: Object listing: CCDB
2021-04-27T10:37:18.844Z  [QualityControl/Index] info: HTTP endpoint: http://localhost:8081
2021-04-27T10:37:18.848Z  [Framework] info: [WebSocket] Server started
2021-04-27T10:37:18.850Z  [Framework] info: [HTTP] Server listening on port 8081
2021-04-27T10:37:18.850Z  debug: [ILSender] InfoLogger executable not found
2021-04-27T10:37:18.852Z  [QualityControl/JsonFileConnector] info: DB file updated
2021-04-27T10:37:18.852Z  [QualityControl/JsonFileConnector] info: Preferences will be saved in /home/sheckel/WebUi/QualityControl/db.json
2021-04-27T10:37:18.890Z  [QualityControl/CCDBConnector] info: Successfully connected to CCDB
2021-04-27T10:38:38.184Z  [Framework] debug: [WebSocket] ID 0 Processing "filter"
2021-04-27T10:38:38.184Z  [Framework] debug: [WebSocket] ID 0 Sent filter/200
2021-04-27T10:38:38.191Z  [QualityControl/Api] error: Online mode is not enabled due to missing Consul configuration

I think, what is strange is, that the qcg obviously in all cases I tried so far is able to find the CCDB, otherwise it would not be able to recognise the correct directory structure (qc/TPC/MO/Tracks) and the histograms therein. It is just not able to get the latest version of those.

EDIT: Maybe I just found a hint: the qcg has the wrong time! In the output above, it claime 10:37, but its 12:37. But the system time should be fine, the CCDB started a moment before correctly tells me 12:37…

Cheers,
Stefan

12

Hi @sheckel ,

If the errors in the terminal are gone than it means now JSROOT is successfully calling the URL provided in the plotURL.
As I have already replicated your environment and could not replicate the problem, I believe it has to be a system specific issue. Thus, I would propose we have a quick meeting on ZOOM so that we can have a look together.
Feel free to mail me george.raduta@cern.ch your availability.

While for the time, QCG uses the browser’s date and time.

Thank you,
George

1 Like
13

Dear all,

coming back to the issue of the local QCG installation after quite a while. The issue is solved by an update of the local CCDB (i.e. download and use the most recent local.jar). My bad, this I really should have tried before…

In any case, the time difference in the QCG terminal output and all other times (system time, browser time) is still observed, but apparently not an issue.

Thanks a lot for all the help, esp. to @graduta for all the time and patience!

Cheers,
Stefan

PS: I marked the first reply by Barth as solution, as that one had solved the original issue of this thread.

1 Like