O2-qc --config new format

O²
1

Dear experts,

apparently there was a change in formatting the --config option when running

o2-qc

With the line copied from the QuickStart documentation on the QC github

o2-qc-run-producer | o2-qc --config json://${QUALITYCONTROL_ROOT}/etc/basic.json

I get the following error

[5930:Dispatcher]: [ERROR] Unhandled exception reached the top of main, device shutting down. Details follow: 
[5930:Dispatcher]: Throw location unknown (consider using BOOST_THROW_EXCEPTION)
[5930:Dispatcher]: Dynamic exception type: boost::wrapexcept<boost::program_options::ambiguous_option>
[5930:Dispatcher]: std::exception::what: option '--config' is ambiguous and matches '--config-key', and '--configuration'

in the Dispatcher, QC-CHECK-RUNNER-QcCheck and QC-TASK-RUNNER-QcTask devices.

Simply changeing --config to either --config-key or --configuration did lead to

2020-05-20 11:25:10.529016     QC infologger initialized
2020-05-20 11:25:10.529027     Using config file ''
2020-05-20 11:25:10.529047     Creating a standalone QC topology.
[ERROR] error while setting up workflow: Ill-formed URI
[INFO] Process 6010 is exiting.

I guess simply the command I need to use changed a bit.
Can someone please give me some insight?

Thanks a lot!
Cheers,
Thomas

2

Hi
I think this is the same problem as reported here https://github.com/AliceO2Group/AliceO2/pull/3605 by Ruben.
Long story short, there was a PR adding a --configuration parameter to any DPL workflow, which collides with some other parameters due to the way boost handles those. I know that @eulisse is already aware of the problem, but seeing that it practically destroyed QC, I would vote for increasing priority on this.

3

Ok, thanks for the info. Then I go back to an older tag.

4

Hi, the greedy boost option parsing ( allow_guessing) should be eliminated in https://github.com/AliceO2Group/AliceO2/pull/3625 (but I did not check if it is)

5

Ok, very good.

Out of curiosity I checked if, as Giulio suggested in https://github.com/AliceO2Group/AliceO2/pull/3605, renaming configuration to dpl-configuration helps. And as far as I can judge it does. But I guess getting rid of allow_guessing goes more to the root of the problem.

6

Yes https://github.com/AliceO2Group/AliceO2/pull/3625 should remove the greedy parsing. it requires a new qc tag though.

7

https://github.com/AliceO2Group/AliceO2/pull/3633 should also work, without requiring a new tag.

8

Hi @eulisse,

FYI: I still have the same error with https://github.com/AliceO2Group/AliceO2/pull/3633

Best,
Thomas

9

You mean using --config-key?

10

Yes I get the exact same behavior as described in the initial post.

11

Ok, let me try, but first I need to find out why QualityControl does not like my current version of Monitoring…

12

@eulisse @tklemenz, let me mention that I am getting the same kind of problem with the current O2 dev and QC master branches. I would also vote for rising the priority of this issue, since I am practically unable to continue the QC development as we need an up-to-date O2 dev branch for the MCH-specific parts…

I am available to do testing if this can help you debugging the problem.

13

It is at the highest priority. Notice you need to use --config-key (in full now). I am surprised it does not work, did recompile everything with alibuild?

14

Latest alidist/master, O2/dev, and QC/master rebuilt with alibuild still gives

2020-05-23 10:21:47.135997     QC infologger initialized
2020-05-23 10:21:47.136009     Using config file ''
2020-05-23 10:21:47.136027     Creating a standalone QC topology.
[ERROR] error while setting up workflow: Ill-formed URI
[INFO] Process 14110 is exiting.

upon

o2-qc-run-producer | o2-qc --config-key json://${QUALITYCONTROL_ROOT}/etc/basic.json

Cheers,
Thomas

15

can you drop the second / in json://?

16

--config-key does not work for me either. Using --config I get the following at the beginning:

[O2Suite/latest-o2-dataflow] ~/VS-test $> o2-qc-run-producer -b | o2-qc --config json://${QUALITYCONTROL_ROOT}/etc/basic.json -b
2020-05-23 10:59:43.273713     QC infologger initialized
2020-05-23 10:59:43.273788     Using config file 'json:///home/flp/alibuild-test/alice/sw/slc7_x86-64/QualityControl/master-1/etc/basic.json'
2020-05-23 10:59:43.273866     Creating a standalone QC topology.
2020-05-23 10:59:43.274489     QC version 0.25.4

If I am using --config-key the file is not recongnized:

[O2Suite/latest-o2-dataflow] ~/VS-test $> o2-qc-run-producer -b | o2-qc --config-key json://${QUALITYCONTROL_ROOT}/etc/basic.json -b
2020-05-23 11:01:36.088991     QC infologger initialized
2020-05-23 11:01:36.089052     Using config file ''
2020-05-23 11:01:36.089106     Creating a standalone QC topology.
[ERROR] error while setting up workflow: Ill-formed URI
[INFO] Process 170894 is exiting.

Same when dropping the second /in json://:

[O2Suite/latest-o2-dataflow] ~/VS-test $> o2-qc-run-producer -b | o2-qc --config-key json:/${QUALITYCONTROL_ROOT}/etc/basic.json -b
2020-05-23 11:02:19.874789     QC infologger initialized
2020-05-23 11:02:19.875166     Using config file ''
2020-05-23 11:02:19.875234     Creating a standalone QC topology.
[ERROR] error while setting up workflow: Ill-formed URI
[INFO] Process 171091 is exiting.

My self-compiled software is up-to-date as of yesterday afternoon…

17

Also for me dropping the second / gives Ill-formed URI.

18

So, the problem is that FairMQ does not allow greedy parsing of options so everything worked so far basically by accident since --config from QC was prepended to --config-key from FairMQ… I will enable back the old behaviour and rename --configuration to --dpl-config,--cfg.

19

https://github.com/AliceO2Group/AliceO2/pull/3646 should workaround the issue by:

  • Reverting back to the allow_guessing style when parsing the options
  • Use --dpl-configuration or --cfg to specify DPL configuration overrides

I will contact the FairMQ people to see how we can proceed, since this is a ticking bomb IMHO.

2 Likes
20

I confirm that the changes in PR 3646 allow to run the QC correctly.
Thanks!