Hi all,
in the InputRecord in DPL, I see that there are options to get:
Messageable types
std container of type with ROOT dictionary
Is there also a way to get a std container of messageable types? That would be pretty nice, since most of my objects are std containers of trivially copiable struct.
Also, what would be the best way to serialise a struct containing a std::vector? (it is not trivially copiable). Should I resort to more refined serialisation methods (boost? protobuf?) for this?
you can get a gsl::span, which is basically a view which has the same API as an std::vector (but does not own the data). With @richterm and Mikolay (which I don’t know how to tag on discourse) we are also discussing how to integrate polymorphic resources so that would allow for the struct of vectors.
Another possibility would be to indeed use one of the other serialisation mechanism, maybe you could even have a look at the arrow one, which provides columns of basic types. Have a look for the TableBuilder examples and then we can iterate more.
Hi @dstocco , i’d avoid non trivially copyable types input/output data since that introduces the need to serialise. I imagine you have pointer members in your struct/vector?
Personally i’d stay away from using arrow directly to construct data if we are talking about trivial types since we always can use arrow to adopt a trivial struct/vec of structs without copying data (not yet implemented, i’ll do it when i get around to it) - best to just use “normal” universal types (like vector) since they allow for all the optimizations we need using e.g. the pmr business, which is already implemented and supported. Integration of that stuff into DPL is a WIP, but we don’t seem to be able to coverge, see e.g. https://github.com/AliceO2Group/AliceO2/pull/1593
Hi @mkrzewic,
actually I don’t have pointer members.
Most of my struct only contains trivial types. So the output is simply a vector of such struct (trivially copiable). I stick to these struct as much as possible, so that I do not really need to serialise.
If I get it correctly, I can simply send this output with a outputs().snapshot, and then I recover them as a gsl::span and that’s it (no serialisation involved).
However, in some cases (like the pre-clusters) my struct contains a vector of digits. This is unfortunately needed since I do not know a priori the number of contiguous fired digits. So my output is a vector of struct that contain a vector data member. In this case, either I serialise, or I stream the memory as a byte series and then re-interpret cast it. The latter case is of course faster than a serialisation, but I need to compute the size of my output and decode it manually. Not sure which of the two methods is best.
Hi @eulisse and @mkrzewic,
I’ve managed in transferring my trivially copiable struct.
In particular, I send them as a std::vector and get them as a gsl::span. This seems to work just fine.
However, I have an issue with one struct that has a Point3D and Vector3D as data members.
These are rather trivial and common objects, so I’d expect them to be messageable, but it turns out they are not. It seems that the problem is that: std::is_forced_non_messageable<Point3D<float>> is evaluated to true.
Is this a bug or a known feature? It would be very useful if Point3D and Vector3D are trivially copiable since they will be probably widely used in O2…
Hi @shahoian,
I see that the Point3D and Vector3D are not messageable since: std::is_forced_non_messageable<Point3D<float>> is evaluated to true.
Do you know if this is a bug or a known feature?
Since they’re both just an array of 3 float (or double), it would be good if they can be trivially messageable in order to avoid an explicit serialisation.
Hi @shahoian,
indeed I retested it and is_forced_non_messageable was not working properly in my script. Not sure why.
However, the Point3D is not trivially copiable: I modified your script to show it (see here ).
Probably it is working fine in your case since the Point3D is serialised through the root dictionary…but it would be nice if this would be trivially copiable as well, so that one does not need root serialisation.
This class formally is not POD type since it does have a non-default copy constructor. Still, it is, in reality, copyable since it compiles to nothing more but 3 floats. Since it is a ROOT class, we have to either live with this it (and use it in the messaging as trivially copyable) or write our own equivalent of formally POD type. The 2nd choice would also imply that we have to write our own transformation class instead of using GenVector/Transform3D of ROOT.
This is doable but before we start inventing a bicycle, I would like to hear the opinion of experts (@mkrzewic, @richterm, @eulisse ) if we need to be so picky in what we allow to message.
Hi @shahoian
Given the special nature of this guy i’d propose to make an exception for this particular type - for now it looks like it is indeed messageable, but there is no API guarantee it will stay this way (although i’m pretty sure ROOT guys will not just change this into e.g. something polymorphic). I would propose to adapt the is_messageable trait to allow this exception with a few constexpr checks (like size) and possibly add a test for casting behaviour or something (@eulisse). In case something does change, we just have to cook our own implementation of Point3D.
(Btw, why can I only mention up to 2 users in a post?)