Import csv file into DPL

Dear all,

I am working on exporting DCS config file to CCDB.
We are thinking that one csv file will be prepared and then export to DPL via o2-config-proxy workflow.

After importing csv file into DPL, I can handle it (I hope), but I don’t know how to do it.
Dod someone succeed to import csv file into DPL?
If possible, could you please tell me the example?

Satoshi

Hi Satoshi,

I did the other way around, send via DPL a buffer to be converted to CSV in DCS. To do this, I use a vector of chars, see https://github.com/AliceO2Group/AliceO2/blob/dev/DataFormats/Detectors/DCS/include/DataFormatsDCS/DCSConfigObject.h. If you want to use this approach, DCS should send this type of data, and then you can get it with DPL.

Chiara

Dear Chiara and O2 experts

Thank you for your reply.
The bellow message has been sent to you by e-mail, but I would like to share the message because I need the support from O2 experts to smooth development.

I don’t get how to use DCSConfigDataFormat.h in DPL.
I am preparing a new workflow that receives a CSV file, converts it into a binary file, and then packed it into an object to store in CCDB.

I would like to know when the CSV file is converted into a binary file?
I think o2-dcs-proxy receives CSV file from DCS and the file is imported into DPL. I think the next workflow receives the CSV and converts the binary file. Is it right?
Or just after o2-dcs-proxy receives CSV file, some build-in function converts it into binary file and imports in DPL?

I followed the explanation below to understand DCSConfigDataFormat.h, but I got the error message and I didn’t succeed to use it…
The error message is,
/home/syano/test.C:10:3: error: no matching function for call to ‘addConfigItem’
o2::dcs::addConfigItem(&config, “pippo”, “pluto”);
^~~~~~~~~~~~~~~~~~~~~~
/home/syano/alice/sw/slc7_x86-64/O2/mft-dcs-config-local1/include/DataFormatsDCS/DCSConfigObject.h:31:13: note: candidate function template not viable: no known conversion from ‘o2::dcs::DCSconfigObject_t *’ (aka ‘vector *’) to ‘o2::dcs::DCSconfigObject_t &’ (aka ‘vector &’) for 1st argument; remove &
inline void addConfigItem(DCSconfigObject_t& configVector, std::string key, const T value)
^
In file included from input_line_8:1:
/home/syano/test.C:11:3: error: no matching function for call to ‘addConfigItem’
o2::dcs::addConfigItem(&config, “pippo_float”, 12.f);
^~~~~~~~~~~~~~~~~~~~~~
/home/syano/alice/sw/slc7_x86-64/O2/mft-dcs-config-local1/include/DataFormatsDCS/DCSConfigObject.h:31:13: note: candidate function template not viable: no known conversion from ‘o2::dcs::DCSconfigObject_t *’ (aka ‘vector *’) to ‘o2::dcs::DCSconfigObject_t &’ (aka ‘vector &’) for 1st argument; remove &
inline void addConfigItem(DCSconfigObject_t& configVector, std::string key, const T value)

I think some mismatch with the arguments.

Thank you for your help.

Satoshi

Hello Satoshi,

Sorry, I think I overlooked your mail…
I don’t know if DPL can receive directly a CSV file, maybe Giulio can comment about it. My idea was that you convert it before sending it to DPL. So DPL would have already a binary blob.
The interface to the format DCSConfigObject changed in the PR, so you need to modify what was in the PR DCS config data format by chiarazampolli · Pull Request #6446 · AliceO2Group/AliceO2 · GitHub (I guess you used that one). I am attaching the updated version.
test.C (656 Bytes)
Cheers,
Chiara

Dear Chiara,

Thank you so much for your reply.

I am still confused about the timing of conversion from CSV into binary.

My idea was that you convert it before sending it to DPL. So DPL would have already a binary blob

My understanding is that csv file from DCS is received by o2-dcs-config-proxy first on O2 facility. O2-dcs-config-proxy can give an output file into DPL.
Is there a time to convert to binary just after receiving the file with o2-dcs-config-proxy without any workflows? All workflows use DPL for output. Your code to convert binary file uses O2 classes, so it is impossible to use it on DCS facility.

Hi @zampolli ,

@syano came to us to confirm on the DCS side. Here, we (DCS and PDP) had already agreed that detectors would send a CSV, which would be converted on O2 side to a format and then written to CCDB. There is no way to prepare a suitable file on the DCS side due to missing O2 software packages.

Cheers,
Pete

Hi @bond , @syano ,

Maybe I was misunderstood: the conversion is not to any O2 format, but plain C/C++. I don’t know if we can receive pure csv files - I let @eulisse comment. If so, good for me, otherwise you should “convert” it to a plain C format like the class I indicated (which is just a vector of chars, nothing else, you can have the same outside O2).

Chiara

I am not sure what you mean about CSV files. Do you mean sending to DPL a string which is actually CSV encoded? I am not sure what kind of support do you expect from DPL for it. To create a vector of strings for each row of the CSV file? Can we move the discussion to JIRA?

Dear Giulio,

Thank you for your reply.
Let’s have a discussion on Cern Authentication

Satoshi