TPC RST- and SYNC-trigger

O² O² CRU
1

Dear CRU Team,

i have a few questions concerning the sending of the RST- and SYNC-Trigger in CTP emulator mode with the PatPlayer.

Before data taking i send the RST-trigger to the FECs via the PatPlayer. The command line is shown below

python ./patplayer.py -i 4:0.0 --idle 0xffffffffffff --sync 0xff0000fffff --reset 0xfffffff00000 --sync-length=32 --reset-length=32 --sync-delay=0  --trg-rst

According to the description, this should manually send the reset pattern via the GBT downlink to the FECs.

Next, i set up the PatPlayer to send the SYNC pattern:

python ./cru-sw/COMMON/patplayer.py -i 04:00.0 --idle 0xffffffffffff --sync 0xff0000fffff --reset 0xfffffff00000 --sync-length=32 --reset-length=32 --sync-delay=0  --sync-trg-sel 9

Since we use the CTP Emulator, it will create the SOT trigger (Bit 9). This is used as the input to the PatPlayer for sending the SYNC, “–sync-trg-sel 9”.

So far this seems to work, we can take data and we see the SYNC pattern in the data stream. However, the SYNCs are miss-aligned, not only by one clock cycle between links but also within a link (which contains 5 SYNC patterns from the 5 SAMPA streams per link). The only explanation for this is that the SAMPAs have not been reset before, so the data streams are miss-aligned. Below is the output from the decoder with the SYNC positions:

Links present
1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 
Processing data for link 0
Num packets : 248
4 : SYNC found at P : 0 F : 163 P : 2656 H : 2
0 : SYNC found at P : 0 F : 166 P : 2704 H : 3
1 : SYNC found at P : 0 F : 166 P : 2704 H : 3
2 : SYNC found at P : 0 F : 166 P : 2704 H : 1
3 : SYNC found at P : 0 F : 166 P : 2704 H : 1
Processing data for link 1
Num packets : 248
0 : SYNC found at P : 0 F : 163 P : 1018464 H : 3
1 : SYNC found at P : 0 F : 163 P : 1018464 H : 3
2 : SYNC found at P : 0 F : 164 P : 1018480 H : 3
3 : SYNC found at P : 0 F : 164 P : 1018480 H : 3
4 : SYNC found at P : 0 F : 165 P : 1018496 H : 3
Processing data for link 2
Num packets : 248
2 : SYNC found at P : 0 F : 161 P : 2034240 H : 2
3 : SYNC found at P : 0 F : 161 P : 2034240 H : 2
4 : SYNC found at P : 0 F : 161 P : 2034240 H : 2
0 : SYNC found at P : 0 F : 168 P : 2034352 H : 2
1 : SYNC found at P : 0 F : 168 P : 2034352 H : 2
Processing data for link 3
Num packets : 248
0 : SYNC found at P : 0 F : 162 P : 3050064 H : 1
1 : SYNC found at P : 0 F : 162 P : 3050064 H : 1
2 : SYNC found at P : 0 F : 162 P : 3050064 H : 1
3 : SYNC found at P : 0 F : 162 P : 3050064 H : 1
4 : SYNC found at P : 0 F : 162 P : 3050064 H : 1
Processing data for link 4
Num packets : 248
0 : SYNC found at P : 0 F : 168 P : 4065968 H : 1
1 : SYNC found at P : 0 F : 168 P : 4065968 H : 1
2 : SYNC found at P : 0 F : 168 P : 4065968 H : 1
3 : SYNC found at P : 0 F : 168 P : 4065968 H : 1
4 : SYNC found at P : 0 F : 168 P : 4065968 H : 1
Processing data for link 5
Num packets : 124
0 : SYNC found at P : 0 F : 162 P : 5081680 H : 1
1 : SYNC found at P : 0 F : 162 P : 5081680 H : 1
2 : SYNC found at P : 0 F : 162 P : 5081680 H : 2
3 : SYNC found at P : 0 F : 162 P : 5081680 H : 2
4 : SYNC found at P : 0 F : 162 P : 5081680 H : 1
Processing data for link 6
Num packets : 248
0 : SYNC found at P : 0 F : 166 P : 6097552 H : 1
1 : SYNC found at P : 0 F : 166 P : 6097552 H : 1
2 : SYNC found at P : 0 F : 166 P : 6097552 H : 1
3 : SYNC found at P : 0 F : 166 P : 6097552 H : 1
4 : SYNC found at P : 0 F : 166 P : 6097552 H : 1
Processing data for link 7
Num packets : 124
2 : SYNC found at P : 0 F : 160 P : 8129072 H : 3
3 : SYNC found at P : 0 F : 160 P : 8129072 H : 3
0 : SYNC found at P : 0 F : 161 P : 8129088 H : 1
1 : SYNC found at P : 0 F : 161 P : 8129088 H : 1
4 : SYNC found at P : 0 F : 161 P : 8129088 H : 1
Processing data for link 8
Num packets : 124
2 : SYNC found at P : 0 F : 164 P : 7113328 H : 1
3 : SYNC found at P : 0 F : 164 P : 7113328 H : 1
4 : SYNC found at P : 0 F : 164 P : 7113328 H : 3
0 : SYNC found at P : 0 F : 165 P : 7113344 H : 0
1 : SYNC found at P : 0 F : 165 P : 7113344 H : 0
Processing data for link 9
Num packets : 124
2 : SYNC found at P : 0 F : 163 P : 9144928 H : 1
3 : SYNC found at P : 0 F : 163 P : 9144928 H : 1
0 : SYNC found at P : 0 F : 164 P : 9144944 H : 1
1 : SYNC found at P : 0 F : 164 P : 9144944 H : 1
4 : SYNC found at P : 0 F : 164 P : 9144944 H : 1

The interesting columns are the “F - Frame” and the “H - Halfword”. It indicates for each stream the position of the SYNC, so in which frame and where in the frame the SYNC was found. The deviation in the frame position between different links could vary, due to some latency differences in the upstream path. However, they are quite large. But i also see a deviation within a link in both, the frame and the half-word. This means the SAMPAs have not been reset (aligned) before.

What i am missing? Do i use the PatPlayer correctly for sending the reset manually?

Cheers,
Torsten

2

Hello Torsten,

1.) This is how you should send the reset manually. I checked and using this command I could see all the 32 reset patterns. However, when the scipt is updating the sync and reset patterns, (before the reset is sent) the pattern player is sending all-zero patterns, could this be an issue for you? (although it happens before the reset)

2.) If you expect the SOT you should use --sync-trg-sel 7, --sync-trg-sel 9 is for SOC (also make sure that --rst-trg-sel is set to a trigger bit that doesn’t toggle. By default it is set to 3, which is the Health check bit)

Cheers,
Tuan

3

Actually, the all-zero pattern is sent again when you issue the

python ./cru-sw/COMMON/patplayer.py -i 04:00.0 --idle 0xffffffffffff --sync 0xff0000fffff --reset 0xfffffff00000 --sync-length=32 --reset-length=32 --sync-delay=0  --sync-trg-sel 9

command. So maybe that is the time when the error occurs, is it possible?

4

Hello Tuan,

thanks for the information!

1). If no SYNC or RST is send then always the IDLE pattern should be send downstream. If you send all “0”, it might create some side-effects. Not sure about it, we never checked that in the lab, we always did send the IDLE pattern. Since i see that the chips are not in sync, this might be the case. I will investigate a bit more and let you know.

  1. I actually meant SOC, the CTP emulator is set up to send the SOC, which is bit 9. I didn’t change the -rst-trg-sel. I assumed that by omitting this option the correct default value will be used. The CTP/LTU uses Bit 30 (TPCrst) for this. So i assume that the CTPEmulator does the same and the default should then be 30, no?

Cheers,
Torsten

5

1.) Ok, let me know. The reason for sending this during configuration was to prevent sending intermediate random values. But if needed the configuration (possible zero pattern) can be separated from the manual trigger (only predefined patterns) in the script

2.) Ok, I’ll update the default values to bit 29 and 30, (TPC sync and reset)

6

Hi Tuan,

i did a quick check by setting the programmable SYNC pattern to be the combined SYNC + RST pattern.

Sync-Pattern looks like this 0xFF00000FFFFF
RST Pattern looks like this 0xFFFFFFF00000

Comined it looks then 0xFF0000000000 (so pretty close to what is send out when the patplayer is reconfigured). This pattern i used as the new SYNC pattern, so when triggering the SYNC is actually does a SYNC & Reset at the same time. And i see side-effects.

How it works at the moment is the following.

  1. Trigger RST manually => SAMPAs are reset
  2. Reconfigure PatPlayer => “Zero Pattern to SAMPA = combined SYNC + RST”
  3. Trigger SYNC during SOC => SYNC in data but SAMPAs missaligned.

Will check some more.

Cheers,
Torsten

7

Hello Torsten,

Both the firmware (send IDLE instead of zeros during configuration) and software (default trigger bits are 29 and 30) changes have been merged to cru-fw/develop and cru-sw/develop, respectively.

Also note that you don’t need to use the “-ss/no-sync-at-start” option anymore to prevent run_enable to trigger a SYNC, it is disabled by default. Now it is the other way around: if for any reason you want to trigger a SYNC when asserting run_enable (when running “./DETECTORS/daq_start PCIE_ID 1”) then you need to specify “-ss/–sync-at-start”

Cheers,
Tuan

8

Hi Tuan,

thanks a lot! Did you also compile the firmware or shall i check out the lasted changes and compile it myself? I don’t mind but if there is a compiled version already it will save some hours.

Cheers,
Torsten

9

We are going to release a new firmware version with the latest feature asked by TPC … it should be tagged before the end of the week.
Cheers

10

I have compiled it, but it is only our core logic, without any userlogic. If it’s not a problem then I can upload it to cernbox.

11

Hi Tuan,

that is fine for the moment, no need for the user logic at this point. First checking the rest of the data flow, then putting the UL in. Does it support 24 links? If yes, then it would be cool to get it :slight_smile:

Cheers,
Torsten

12

Hello Torsten,

Yes it has 24 links. I uploaded it to cernbox (CRU_FW_RELEASE/dev/cru-v2-idlefix.sof.zip)

Cheers,
Tuan

13

Cool, thanks!

T.