FLP-specific subspecifications

O² O² FLP
1

Dear experts,

In order to port our reconstruction workflow for data compression for EMCAL to the FLP we need to set a FLP-specific subspecification to the DPL channels as we have two FLPs and both FLPs would write into the same channel in case the subspecification is the same. The reconstruction workflow can take any subspecification as parameter. A possible candidate would be the ID of the FLP. Is there any way to handle this with the control workflows so that a dedicated subspecification value can be passed to the DPL workflow during configuration, which is different on the two FLPs.

Thanks in advance!

Cheers

Markus

2

Hi, we discussed with @teo about this a bit.
I assume you need it to have unique IDs for processed data. If this is something you need to run QC, then perhaps we have to understand better what you want to do, because you shouldn’t need it.

In a workflow template you can get e.g. it which is the host name that DPL workflow is going to run on. This is an example how it is used to create another variable: ControlWorkflows/readout-dataflow.yaml at 34be6a399c3919d26ae3a392d795bf380df8c0c2 · AliceO2Group/ControlWorkflows · GitHub

In your case, I assume you could pass it as argument to your DPL workflow. See an example of PHOS compressor, where we allow to set certain parameter from adv. config. in GUI:
Default value in workflow template: ControlWorkflows/phos-compressor.yaml at 34be6a399c3919d26ae3a392d795bf380df8c0c2 · AliceO2Group/ControlWorkflows · GitHub
Then an argument in Task template: ControlWorkflows/phos-compressor-PHOSRawToCellConverterSpec.yaml at 34be6a399c3919d26ae3a392d795bf380df8c0c2 · AliceO2Group/ControlWorkflows · GitHub

However, I would advise that you keep the same Data Processor names, so you can use one workflow template for both FLPs.

If needed, @teo offered to add a function which extracts the ID from the host name.

Alternatively you can also use task_id, but this one will change each time the workflow is started.

3

Hi @pkonopka , Hi @teo,

Not sure my point was clear, let me try to explain again:

Our problem is that we 2 FLPs (one for EMCAL and one for DCAL), and if we run the reconstruction workflow on both simultaneously they will both write EMC/CELLS/0 and EMC/CELLSTRGR/0. According to discussions with @shahoian this will create problems in timeframe building on the EPN as we have 2 processors sending channels with exactly the same channel configuration, and we must define the subspecification differently for the 2 FLPs. In [EMCAL-703] Add optional settable subspecification for raw to digits output by mfasDa · Pull Request #6725 · AliceO2Group/AliceO2 · GitHub I added the option to make the subspecification settable. What we need now is a way via the control workflows to be able to configure different subspecifications for the 2 FLPs. This is a feature only needed on the FLP - when we run offline or on the EPN there is no issue in timeframe building and the subspecification can be always 0.

Hope the issue is clearer now!

Cheers

Markus

4

Then you could use what i proposed above with {{ it }} and a function proposed by @teo which would strip the FLP name to the FLP ID only (alio2-cr1-flp146 → 146).

Just for the record, privately i proposed you also an alternative, where you use outputs for any subspec (EMC/CELLS/*) on both FLPs and you determine the output subspec based on link id.

5

@mfasel I think the device on the FLP can do this w/o any option passing or interaction with external world: all what you need to do is to define your outputspec via ConcreteDataTypeMatcher (wildcarded subspec) and tell to your device to push the output with the subspec = e.g. the FEEID of the 1st encountered RDH (which will happen to be the same as the subspec of the corresponding incoming raw data). The combination of your DataDescription and such subspec is guaranteed to be unique and this is all what you need. Obviously, the consumer device should also subscribe with ConcreteDataTypeMatcher

PS just noticed that @pkonopka proposed the same (as private communication)

6

Dear @pkonopka , Dear @shahoian,

Thanks a lot for your suggestions! I will go ahead with the implementation based on wildcard subpsecification using ConcreateDataTypeMatcher.

Cheers

Markus

7

Dear @mfasel, @pkonopka, @shahoian
I would like to implement same functionality for PHOS, but I see one problem in automatic reading of subspec number from the data header: If the very first TF will be empty, we do not have raw data to calculate subspec and have to send output to some default subspec. Then, once data will arrive, subspec will change. Probably, if all consumers will take wildcarded inputs is does not matter, but I am not sure, Is it OK for the framework if subspec will change in the process?

8

Hi @prsnko

Perhaps you can use as a subspec a hash generated from the flp hostname? Something like

char hostname[HOST_NAME_MAX];
gethostname(hostname, HOST_NAME_MAX);
std::string ss(hostname);
uint32_t subSpec = (uint32_t)std::hash<std::string>{}(ss)
9

As long as you use a wildcard, you should be fine.

10

Dear Giulio @eulisse, all, I implemented this approach in PR Subevent merger implemented by peressounko · Pull Request #7674 · AliceO2Group/AliceO2 · GitHub, but it does not work as EventBuilder produces and consumes same messages: PHOSEventBuilder depends on itself. Do you plan to implement something like “wildcard excluding zero subspec” or we need another approach?
Best regards, Dmitri

11

Dear Giulio @eulisse,
do you expect any progress in implementation of “excluded wildcards”? Or should we invent another way how to separate input and output messages? Best regards, Dmitri