TPC QC task does not publish anything anymore

O²
1

Dear all,

when running our TPC QC PID task, it does not publish anything anymore, neither with the default settings on the qcg-test webpage, nor when setting up our own local CCDB and QCG. In the latter case, also nothing is created in /tmp/QC/. When running the basic QC task, it works in both environments as expected, the histograms are published and in the local case also stored.

We are investigating this since a while and found some error messages related to consul, but these are also present when running the basic task, so they seem not to be the issue here (although probably have to be solved as well - as soon as this topic is solved, I will have a closer lock and open another one for that issue).

Looking at the DPL GUI, we see the following difference between the basic task and our own: Both start with the internal-dpl-clock, then follow the producer-0 (basic) or tpc-track-reader (tpc), in both cases the Dispatcher and then the QC-TASK-RUNNER. The basic task continues with the QC-CHECK-RUNNER and the printer, while our task ends with a internal-dpl-global-binary-file-sink.

We compared the tpcQCPID.json file to the basic.json file and found two differences:

  • In the taskParameters, "nothing": "rien" was changed in the basic.json to "myOwnKey": "myOwnValue". Although this both sounds just like a placeholder, we changed this, but with no effect.
  • In the basic.json, a checker was added. This we have not done, yet. We think, the problem may be here. It has been changed within a larger commit for the Checkers 2.0 from the 16th of December, and since then we did not have our task running, e.g. because of some general issues in O2/QC. After that commit, there was one change to be done in the run executable, but this was changed centrally for the PID task and should be fine.

Some further information:

  • We are running on Ubuntu 18.04
  • We use a custom installation of O2 and QualityControl (i.e. no docker), and O2, QualityControl and alidist are up-to-date.
  • As input we use the output of an O2 TPC simulation, which was working before. We have even re-run the simulation with the most recent version of O2 and QC.
  • In the DPL GUI we do see data going through all the steps.
  • We added an InfoLogger message in the initialization of the publishing in the PID.cxx and we get the corresponding messages in the terminal.

Could anybody confirm that the checker (or CheckRunner) now is mandatory to run a QC task? And if so, before we look at each file, what exactly has to be changed in order to get this running? Thanks a lot for any information and help!

Cheers,
Thomas and Stefan

2

Dear Thomas and Stefan,
I am sorry about that.
I can confirm that there is a bug (it is not the expected behaviour) that makes it mandatory to have a checker to store the data coming from the task. It was introduced in the last version along with the new Checkers as you have noted.
Thank you, and congrats, for the extensive investigation.

To work around the problem, please add this piece of json to your config after the tasks :

    "tasks": {   // this is just to show where to put the next block of json
      "QcTask": { // this is the name of your task, take note of it
         ...           
      }             // end of QcTask definition
    },               // end of tasks, don't forget the comma !
    "checks": {
      "QcCheck": {
        "active": "true",
        "className": "o2::quality_control_modules::skeleton::SkeletonCheck",
        "moduleName": "QcSkeleton",
        "policy": "OnAny",
        "dataSource": [{
          "type": "Task",
          "name": "QcTask",                  // put the name of your task 
          "MOs": ["example"]
        }]
      }
    }

We are working on a proper fix.

I hope the workaround is enough for you, in case it is not just let us know and we will help you out.
cheers,
Barth

3

Dear Barth,

thanks a lot for your quick reply and suggested workaround! I just implemented it, now the data is going from the QC-TASK-RUNNER to the QC-CHECK-RUNNER. Unfortunately, from there it again goes to the file-sink and not to the printer. Do we in addition have to change anything anywhere else, e.g. in the source file of the QC workflow (i.e. in our case: QualityControl/Modules/TPC/src/PID.cxx)?

Cheers,
Stefan

4

Dear Stefan,

If it goes to the checker, it should at least go to the database and the QCG. Can you confirm this ?
I would say this is the most important.

Concerning the printer, it is just an example device to display the QC objects without reaching to the CCDB and to show how to send the QC objects to a different device. I am not sure whether you really want it. For sure it is not necessary.

Now, if you want to add the printer then you can add the printer to runTPCQCPID.cxx (your workflow).
Simply copy lines 100-108 from runBasic.cxx.

Cheers,

5

Dear Barth,

yes, I can confirm that it goes to the database and QCG. I now also checked within the setup of a local CCDB and QCG and this works as well. We will not use the printer, at least not for the moment.

Thanks a lot for your help!

Cheers,
Stefan

1 Like