Non-intrusive boost serialisation


(Diego Stocco) #1

Hi all, and especially Framework’s gurus (@eulisse),
I’m currently using boost for the serialisation of reconstruction related data.
However, I know that root objects will be used in the end for simulations. This is not an issue except at the boundary between simulation and reconstruction (digits).
Indeed, the digits should be streamed to root files in simulation, while we can stream them with boost in data.
An easy way to be able to choose if using the ROOT or Boost serialisation would be to use simple struct for the data format and move the serialisation-related part outside of the definition.
In this way we would be able to chose a different serialization for the simulation and reconstruction workflows.
For ROOT, it’s enough to build the dictionary.
For Boost, there is a non-intrusive implementation which reads as:

struct myStruct {
  int myInt;
};

namespace boost {
namespace serialization {
template <class Archive>
void serialize(Archive& ar, myStruct& data, const unsigned int version)
{
  ar& data.myInt;
}
} // namespace serialization
} // namespace boost

So far so good. The problem is that the test in Framework/TypeTraits.h:

template <class Type, typename Archive>
struct is_boost_serializable_base<Type, Archive, void_t<decltype(std::declval<Type&>().serialize(std::declval<Archive&>(), 0))>> : std::true_type {};

clearly fails if the serialisation is non intrusive (i.e. defined as a global function in the boost::serialization namespace).
Is there a way to modify the test so that is_boost_serializable returns true for a non-intrusive boost serialisation as well?

Thanks in advance,
best regards,
Diego


(Giulio Eulisse) #2

I think what you need is to add something like:

template <class Type, typename Archive>
struct is_boost_serializable_base<Type, Archive, void_t<(boost::serialisation::serialize<Archive>(std::declval<Archive&>(), 0))>> : std::true_type {};

to the Framework/TypeTraits.h so that if the method is not present it will check if the free function is there. Because of the way templates work, substitutions are not an error so in case neither one is true, it will fallback to the generic, lower priority, version which results in std::false_type. I might have gotten some bracket wrong, but the logic should be valid, if you have problems and you are at CERN let’s sit together later today.


(Giulio Eulisse) #3

@richterm might have more suggestions.


(Diego Stocco) #4

Hi @eulisse,
thanks a lot for the tip.
I already tried with a similar expression:
template <class Type, typename Archive>
struct is_boost_serializable_base<Type, Archive, void_t<decltype(boost::serialization::serialize(std::declval<Archive&>(), std::declval<Type&>, 0))>>: std::true_type {};
But the compiler complains that this function was already defined (for some reason it considers this expression the same as the one currently in aliroot).
I tried by removing the decltype as in your example, but the template expects a type, so it’s not compiling either.

For testing purposes, I made a small example:
https://cernbox.cern.ch/index.php/s/Qu1KaA78NI59Vta
that can be compiled with:
c++ -std=c++11 -isystem /usr/local/include test_boostSerializable.cxx

Unfortunately I’m not at CERN, but I can connect with vidyo.
Thanks!
Cheers,
Diego


(Giulio Eulisse) #5

right… Can you try adding an extra template parameter “int” in one and “long” in the other and put a 0 as its value? That should deduplication of the two and give the preference to the int one. I can try myself later today.


(Diego Stocco) #6

Hi Giulio,
I’ve been trying w/o success.
Anyways, there is also another issue that will probably pop up at some point. When I use: template <class Type, typename Archive>

struct is_boost_serializable_base<Type, Archive, void_t<decltype(boost::serialization::serialize(std::declval<Archive&>(), std::declval<Type&>, 0))>>: std::true_type {};

alone in my example, is_boost_serializable becomes always true. I guess the problem is that boost::serialization::serialize is always defined as a template, even if a specialisation is not provided for Type.


(Sebastien Binet) #7

I know meta-template programming is a C++ developer pride, but…
wouldn’t it be as readable from casual developers (or dare I say, physicists?) to have either a simple (static?) method:

class MyType {
public:
static void is_boost_serializable() {}
static void is_ROOT_serializable()  {}
static void is_Arrow_serializable() {}
};

or, alternatively:

class MyType {
public:
static bool is_serializable_with(const SerializableTech tech) {
    switch (tech) {
    case SerializableTech::Boost: return true;
    case SerializableTech::ROOT: return true;
    }
    return false;
}
};

surely, the amount of characters to type to implement this little method dwarfs that of implementing the actual serialization work :slight_smile:


(Diego Stocco) #8

Hi @sbinet,
I tend to disagree with your solution. It is better to have a clean struct format, and check if the struct is serialisable by looking at the serialisation code that in any case needs to be written.
In any case, what we’re trying to do here is to modify only one templetized check in order to check for the existence of any of the two possible methods proposed.
But there is also a simpler solution where we create two separate templetized tests and ask for the logical OR of both tests in DataAllocator.h and InputRecord.h, where the tests are actually applied. It ends up in modifying just 3 files instead of 1, while, your solution requires a virtual structure for all data formats.

Cheers,
Diego


(Diego Stocco) #9

Hi @eulisse,
I think I understand why the method above implemented in a standalone is always true, whether or not the boost serialization is implemented.
The point is that, in boost, it exists a template named:

template<class Archive, class T> inline void serialize( Archive & ar, T & t, const unsigned int file_version ){ access::serialize(ar, t, static_cast<unsigned int>(file_version)); }

In the non-intrusive version, we simply override this function.
This means that the function always exists: that’s why the check written here is always true.
Now, this function calls access::serialize, which in turns calls t.serialize, i.e. the serialize function in the intrusive method. If this is not defined, the function will crash badly when called.
So, what one would need to test (at compilation time), is if boost::serialization::serialize is valid, rather than if it exists.
Does anyone know how to do it?

Thanks!
Cheers,
Diego


(Giulio Eulisse) #10

I came to the same conclusion, but I don’t have an answer yet.


(Diego Stocco) #11

Ciao @eulisse
I’ve kept looking for a solution but couldn’t find one yet (all of the solutions that I found are meant to search for a data member of a class, not a global function).
I was wondering if you had time to look at this in the meantime…

Thanks in advance,
Ciao,
Diego


(Giulio Eulisse) #12

Ciao @dstocco,

I missed your message, sorry about that.

I could not find a solution either, apart from wrapping the object to serialize in something like ImplicitlyBoostSerialised<YourClass> like we do for RootSerialized<>.

Maybe @richterm has better ideas?