Modify CPV ControlWorkflows

sevdokim · October 4, 2021, 6:15am

Dear all,

I want to modify CPV ControlWorkflows, namely:

https://github.com/AliceO2Group/ControlWorkflows/blob/master/scripts/cpv-compressor.sh
add more outputs to o2-dpl-output-proxy: --dataspec 'A:CPV/DIGITS;dd:FLP/DISTSUBTIMEFRAME/0' → --dataspec 'x0:CPV/RAWDATA;A:CPV/DIGITS/0;A:CPV/DIGITTRIGREC/0;A:CPV/RAWHWERRORS/0;dd:FLP/DISTSUBTIMEFRAME/0'
https://github.com/AliceO2Group/ControlWorkflows/blob/master/scripts/cpv-qc-compressor.sh
add flag --pedestal to o2-cpv-reco-workflow.

For that I want to fork ControlWorkflows to my github, modify myself and create pull request. But first it needs to be tested. Can I add it to aliECS with coconut command issued from cpv flp? So that I will be able to push something to https://github.com/sevdokim/ControlWorkflows, select this repo on aliecs page and run environments for testing purposes.
Is it even allowed for ordinary users to do such operations?
If not then how to proceed in order to test workflows?

Thank you & regards,
Sergey

bvonhall · October 4, 2021, 7:23am

Dear Sergey,

We can call coconut to add your repo on aliecs. You should tell us what is your fork.

Cheers,
Barth

sevdokim · October 4, 2021, 7:27am

Hi Barth,

my fork is https://github.com/sevdokim/ControlWorkflows
I didn’t pushed anything there yet.
Do I need to specify branch/anything else?

Cheers,
Sergey

vmcb · October 4, 2021, 11:58am

Hi Sergey,

I have added your fork to the AliECS instance at P2. You should select it and the branch you want to use from the AliECS GUI. Please remember to click on the reload button in the AliECS GUI whenever you do any change.

Let us know if you have any issues.

Cheers,
Vasco

sevdokim · October 4, 2021, 12:02pm

Hi Vasco,

thank you very much! I see my repo in aliECS.

Regards,
Sergey

sevdokim · October 4, 2021, 1:13pm

Hi @vmcb , @bvonhall, all,

how do I choose relevant branch in my repo?
I see that I can only select the repo, not the branch.

To explain what I did: modified cpv-compressor workflow, pushed changes to fix-cpv-workflows branch: https://github.com/sevdokim/ControlWorkflows/tree/fix-cpv-workflows

Cheers,
Sergey

bvonhall · October 4, 2021, 2:56pm

Hi,

You should select the branch in the Revision field:

sevdokim · October 4, 2021, 3:04pm

Oh, thank you very much, Barth! I was temporary blind))

sevdokim · October 5, 2021, 9:18am

Hello @vmcb @bvonhall all,

I want to produce a production workflow which does not do anything with raw data from readout but reads simulated raw file. A working workflow is based on command

o2-dpl-raw-proxy -b --session default --dataspec 'x0:CPV/RAWDATA;dd:FLP/DISTSUBTIMEFRAME/0' --readout-proxy '--channel-config "name=readout-proxy,type=pull,method=connect,address=ipc:///tmp/stf-builder-dpl-pipe-0,transport=shmem,rateLogging=1"' \ 
| o2-cpv-reco-workflow -b --session default --input-type raw --output-type digits --disable-root-input --disable-root-output --disable-mc \
| o2-dpl-output-proxy -b --session default --dataspec 'x0:CPV/RAWDATA;A:CPV/DIGITS;A:CPV/DIGITTRIGREC/0;A:CPV/RAWHWERRORS/0;dd:FLP/DISTSUBTIMEFRAME/0' --dpl-output-proxy '--channel-config "name=downstream,type=push,method=bind,address=ipc:///tmp/stf-pipe-0,rateLogging=1,transport=shmem"'

I was thinking about removing o2-dpl-raw-proxy and replacing it by o2-raw-file-reader-workflow however run cannot be configured with such command:

gRPC client can't dial command="source /etc/profile.d/o2.sh && o2-raw-file-reader-workflow -b --input-conf /home/cpv/O2tests/evd/pythia8hi/raw/CPV/CPVraw.cfg --detect-tf0 --nocheck-tf-start-mismatch --configKeyValues HBFUtils.nHBFPerTF=128 --session default | o2-cpv-reco-workflow -b --session default --input-type raw --output-type digits --disable-root-input --disable-root-output --disable-mc | o2-dpl-output-proxy -b --session default --dataspec 'x0:CPV/RAWDATA;A:CPV/DIGITS/0;A:CPV/DIGITTRIGREC/0;A:CPV/RAWHWERRORS/0;dd:FLP/DISTSUBTIMEFRAME/0' --dpl-output-proxy '--channel-config "name=downstream,type=push,method=bind,address=ipc:///tmp/stf-pipe-0,rateLogging=1,transport=shmem"' | o2-dpl-output-proxy" endpoint="127.0.0.1:31005" error="context deadline exceeded" id="2TphsoSZ8ca" task="github.com/sevdokim/ControlWorkflows/tasks/cpv-simulated-compressor-dpl-output-proxy@1434cd60c50fb0a0bf3369ebe8d62fe2bd5c3064?2TphsoSZ8ca" transport="JSON"

To query relevant today’s logs:

I think such fail happened because of no consumers which pick up data from readout such as o2-dpl-raw-proxy. Any suggestions how to trick the system?

Thanks, regards,
Sergey

bvonhall · October 6, 2021, 2:32pm

Good question !

coudl you plug an extra device to the proxy to discard the data ?

sevdokim · October 6, 2021, 7:28pm

Hi Barth,

that’s exactly what I’m thinking about.
Trying following command:

o2-dpl-raw-proxy --session default --dataspec 'x0:CPV/RAWDATA;dd:FLP/DISTSUBTIMEFRAME/0' --readout-proxy '--channel-config "name=readout-proxy,type=pull,method=connect,address=ipc:///tmp/stf-builder-dpl-pipe-0,transport=shmem,rateLogging=1"'   \
  | o2-dpl-null-sink --inputs 'x0:CPV/RAWDATA;dd:FLP/DISTSUBTIMEFRAME/0'   \
  | o2-raw-file-reader-workflow --loop 1 --nocheck-tf-start-mismatch --session default --input-conf CPVraw.cfg   \
  | o2-cpv-reco-workflow --session default --input-type raw --output-type digits --disable-root-input --disable-root-output --disable-mc   \
  | o2-dpl-output-proxy --session default --dataspec 'x0:CPV/RAWDATA;A:CPV/DIGITS/0;A:CPV/DIGITTRIGREC/0;A:CPV/RAWHWERRORS/0;dd:FLP/DISTSUBTIMEFRAME/0' --dpl-output-proxy '--channel-config "name=downstream,type=push,method=bind,address=ipc:///tmp/stf-pipe-0,rateLogging=1,transport=shmem"' \
  | o2-qc --config json://pedestal-task-no-sampling.json

But the problem is that dpl-null-sink is a bit smarter that it looks: it is sinking raw data not only from o2-dpl-raw-proxy but from all possible sources which match the query including o2-raw-file-reader-workflow which also outputting data. See debug gui:

Do you have any ideas how to arrange the sinking of only raw data coming from o2-dpl-raw-proxy and not from o2-raw-file-reader-workflow?

Thanks,
Sergey

bvonhall · October 7, 2021, 7:17am

Hi,

Could you rename the output of either the proxy or the reader ?

Cheers,

sevdokim · October 7, 2021, 8:12am

Hi Barth,
for renaming outputs I need a bit of explanation and suggestion.
Here what raw-file-reader outputs:

            "name": "raw-file-reader", 
            "outputs": [
                {
                    "binding": "CPV/RAWDATA",
                    "origin": "CPV",
                    "description": "RAWDATA",
                    "lifetime": 0
                },
                {
                    "binding": "stfDist",
                    "origin": "FLP",
                    "description": "DISTSUBTIMEFRAME",
                    "subspec": 0,
                    "lifetime": 0
                }
            ],

Here what raw-proxy outputs:

            "name": "readout-proxy",
            "outputs": [
                {
                    "binding": "x0",
                    "origin": "CPV",
                    "description": "RAWDATA",
                    "lifetime": 0
                },
                {
                    "binding": "dd",
                    "origin": "FLP",
                    "description": "DISTSUBTIMEFRAME",
                    "subspec": 0,
                    "lifetime": 0
                }

Here what Raw2DigitConverter expexts as input:

            "name": "CPVRawToDigitConverterSpec",
            "inputs": [
                {
                    "binding": "RAWDATA",
                    "origin": "CPV",
                    "description": "RAWDATA",
                    "lifetime": 7
                },
                {
                    "binding": "STFDist",
                    "origin": "FLP",
                    "description": "DISTSUBTIMEFRAME",
                    "subspec": 0,
                    "lifetime": 0
                }
            ],

What and how to re-name it? Here I most probably need a written command form you.
Another possibility I see is just do not proxying raw data. But I’m sure if readout will be happy with that. Let me check.

Cheers,
Sergey

sevdokim · October 7, 2021, 8:19am

Also inportant to say is that Raw2DigitConverter expects input which is hardcoded, namely

inputs.emplace_back("RAWDATA", o2::framework::ConcreteDataTypeMatcher{"CPV", "RAWDATA"}, o2::framework::Lifetime::Optional);

And I’m not sure how to change name of outputs within raw reader? In the file CPVraw.cfg?
In it I have only following conf:

[input-CPV-0]
dataOrigin = CPV
dataDescription = RAWDATA
readoutCard = CRU
filePath = /data/cpv-raw-data/data_p2_21_036.raw

Seems not a best idea to change anything in it…

Cheers,
Sergey

shahoian · October 7, 2021, 9:15am

@sevdokim you have both input-proxy and raw-reader sending the same output. If the workflow must start by the input-proxy, can you define its output to something not used downstream, e.g. “xx:FT0/RAWDATA”, so the input you consume comes from the reader only

sevdokim · October 8, 2021, 11:24am

Hello @bvonhall @shahoian

I didn’t manage to insert o2-raw-file-reader-workflow by renaming outputs and other tricks. Everything is running locally but aliECS just refuse the workflows. Solution is provided by @sy-c : in the readout config it is possible to replay raw data file. I post here the readout config which works well for cpv just for future track

Thank you all & cheers,
Sergey

# Example o2-readout-exe configuration file for replay

[readout]
flushEquipmentTimeout=2
aggregatorStfTimeout=0.5
aggregatorSliceTimeout=1

# timeframe rate limit, to throttle output
# real rate may be lower if limited by CPU
tfRateLimit=10

[consumer-stats]
enabled=1
consumerType=stats
monitoringUpdatePeriod=1
consoleUpdate=0
monitoringEnabled=1
monitoringURI=influxdb-unix:///tmp/telegraf.sock

[consumer-fmq-stfb]
enabled=1
consumerType=FairMQChannel
# fmq-name should be 'readout'
# to allow OCC to overwrite params for StfBuilder connection
fmq-name=readout
fmq-address=ipc://@readout-fmq-stfb
fmq-type=push
fmq-transport=shmem
sessionName=default
unmanagedMemorySize=57026M
memoryPoolNumberOfPages=39858
memoryPoolPageSize=244k
disableSending=0

[rx-fmq-stfb]
decodingMode=stfHbf
dumpRDH=0
dumpTF=1
channelAddress=ipc://@readout-fmq-stfb
channelType=pull
transportType=shmem

[equipment-player-1]
equipmentType=player
memoryPoolPageSize=400k
memoryPoolNumberOfPages=10000
#filePath=/home/cpv/O2tests/evd/pedestal_raw/data_p2_21_042.raw
filePath=/tmp/data_p2_21_042.raw
rdhUseFirstInPageEnabled=1
TFperiod=128
autoChunk=1
autoChunkLoop=-1