is there a strict limit at the moment concerning the maximum number of events which can be simulated, digitized and reconstructed in on Go via O2 at the moment? For the development of some TRD calibration code we require a rather large event sample (maybe around 100k pp collisions). These would probably span over multiple TFs. Is this already supported?
I tried to start with 20k pp events. The simulation and digitization worked fine. But the TPC reco workflow started with o2-tpc-reco-workflow -b --tpc-digit-reader '--infile tpcdigits.root' --input-type digits --output-type clusters,tracks --configKeyValues "GPU_proc.ompThreads=4;" --disable-mc --shm-segment-size 100000000000 --run | tee recTPC.log
Did not finish because of the error: [2930038:tpc-tracker]: [11:25:31][ERROR] Input data has invalid time bin 1498048 > 114050
I am not sure if this is a bug or if I went over the limit of events which can be simulated in one go…
Has anyone tried to create such a large event sample?
The commands I used for simulation and digitization are the following: o2-sim -n 20000 -g pythia8 --skipModules ZDC -j 64 --seed 1234 --noemptyevents o2-sim-digitizer-workflow -b --onlyDet ITS,TPC,TRD,TOF --disable-mc --run --shm-segment-size 200000000000 |& tee out.log
Only TPC has strict limit of the number of time-bins corresponding to 256 orbits.
You can increase the --interactionRate to fit more collisions to this period, but 100k is too much. If you don’t need mc samples, you can convert the digits to row and process multiple TFs (once the raw decoder of TRD will be operational).
Perhaps @drohr can add an option to increase the TBs limit.
That said, there are other limitations coming from the available memory on the PC and on the root output TBuffer: single branch cannot exceed 1GB.
Unfortunately, digitization does not automatically produce multiple timeframes (in any of the detectors). Such feature can be implemented in the future if there is need. For now, there are alternative ways:
I guess you’d like to produce a number of timeframes equivalent to ~100K pp events.
So for the current processing mechanism one would
a) derive the number of timeframes → 1 timeframe 11ms → ~4400 pp events (at 400kHz) → ~22 timeframes. ( Take more and smaller timeframes to be safe).
b) use the O2DPG scripts with the -tf + -ns options to ask for the simulation and reconstruction of this amount of timeframes. Each timeframe data will be stored in a separate folder. [ This is even powerful since now different timeframes are produced in parallel on a big machine]. A downside is that edge-effects across timeframes are not covered.
We have decided that 256 orbits is the limit, since 8 bit counters are used in some places of the CTP.
For the TPC, there are no 8 bit orbit counters, but the scaled integers for z will overflow somewhere between 256 to 512 orbits (don’t remember exactly where). I could in theory add an option to relax the orbit limit of 256 slightly, but then we would only get ~50% margin, so I do not believe this would be of much help.
I also think changing the TB limit is not necessary (at least for my problem it would not help).
I tried the O2DPG scripts and I managed to process 4 TFs with 100 pp events each. Now I will try to create a larger sample. Just to be sure since I don’t know Pythia very well… @swenzel when I do o2dpg_sim_workflow.py -proc "none" is this equivalent to minimum bias?
Because first I missed that option and then I saw the error from the generator: mkpy8cfg.py: error: argument --process: invalid choice: '' (choose from 'none', 'inel', 'ccbar', 'bbbar', 'heavy', 'jets', 'dirgamma')
@oschmidt : Not sure, but it sounds reasonable. This part is maintained by @rpreghen and G. Conesa (@gconesab) or the DPG team in general. Maybe they can comment.