Trying to solve some issue with the testing of one Root macro in AliceO2, I’m revisiting how we deal with macros in our testing. We currently do 2 tests for each macro : one “loading” test (equivalent to .L macro.C) and one “compilation” test (equivalent to .L macro.C+).
But with Root6 the .L macro.C is also “compiling” (just in time compiling) the macro, right ?
According to this answer to a question in the root forum, the only differences between cling (.L macro.C) and ACliC (.L macro.C+) are the compiler optimization level (not really relevant for testing I would argue) and the fact that ACliC makes a dictionary. For the latter I suspect we don’t have objects to be serialized that are only defined in macros, do we ?
So, if I’m not missing something, I would then conclude that we could simply drop the “compilation” tests altogether (keeping only the “loading” tests) and thus cut the macro testing time by 2 (at least) … Would that be correct ?
@laphecet Usually, if the macro test fails in the compiled mode but passes in the interpreted one, it is due to the dictionary generation (because of some dependencies or some template related problems). So, if the macro is supposed to run in the compiled (i.e. optimized) mode, i.e. requires a perfomance, I would keep the test just to be sure the macro can be compiled.
On the other hand, one could argue that we use macros much less than in the past and that most of our performance critical stuff is in devices (or in library code).
Since the main advantage of macros is the interpreted mode, I think we could concentrate on testing this one and potentially slash the compiled one … or make the latter optional for the ones that are performance critical. This would also simplify our CMake (as far as I understand.) and boost the CI performance.
What we could do is change the default behaviour we currently have.
Currently, the o2_add_test_root_macro function will, unless instructed otherwise (by the LOAD_ONLY or COMPILE_ONLY options), generate tests for both the compile and load versions. We could change that to generate only the loading test by default, unless compilation is explicitly required (e.g. with a COMPILE option) ?
That way most of the macros will be tested (more) quickly, and only the ones that actually requires some more strict testing (i.e. test of dictionaries) would incur the extra compilation overhead.
I agree on the last point. Something maybe more or less related: Could we then also get rid of having to declare each macro individually via o2_add_test_macro? For me, this is a big inconvenience. Macros are scripts and should just be tested implicitly unless someone needs special requirements (not to test or to compile). There is no big advantage over executables, If I have to declare them.
Sandro knows that I’m kind of biased against macros (as we already exchanged briefly on the subject in a private email). At least against what I’m considering an abuse of those macros in our current usage (we do have 183 currently in AliceO2 !). So I’m probably not the best person to facilitate too much their usage
That being said, automatically testing all macros would not be completely trivial anyway. That would mean using a glob (to find e.g. all *.C files) but that would probably not give what you want. Globs are evaluated at configure time : so if you add a macro afterwards, CMake will not notice automatically… Hence the requirement to explicitly declare them (plus the current o2_add_test_root_macro forces to think about each macro dependencies, which is a good thing in my book).
There is no big advantage over executables, If I have to declare them
To which I would answer that in many cases we should indeed use executables instead of macros.
