CRU test with clock from LTU

O² O² CRU
1

Hi Tuan,
I just received all of the hardware to connect the LTU to the CRU, and was trying to do a loopback test with that setup to our Readout Unit to verify the clock quality in the GBT link.
First of all, Marian mentioned that I had to assign an “ONU number” to the CRU, but if I understood this correctly, when I am not using the upstream link, this is not necessary, correct?
I simply configured the CRU as follows:

python standalone-startup.py -i 3b:0.0 -c ttc

I then used the linkstat.py script to verify the GBT connection, and that one also seems to indicate the ttc frequency:

[root@gemini COMMON]# python linkstat.py -i 3b:0.0 -l0-3 -v
globalgen: | ttc240freq 240.47 MHz | lcl240freq 240.47 MHz | ref240freq 240.47 MHz | clk not ok cnt 0 |
Wrapper 0: | ref freq0 240.47 MHz | ref freq1 0.00 MHz | ref freq2 0.00 MHz | ref freq3 0.00 MHz |

            Link #0: Wrapper 0 - Bank 0 - Link 0

            Status  |                     Counters
PLL locked: YES     |          Not ready: 3575538523

Locked to data: YES | Not locked to data: 2353903492
TX clock frequency: 240.47 MHz
RX clock frequency: 240.47 MHz
….

I then configured the 8bit counter loopback test and seem to see no errors on the connected link (0):
[root@gemini py]# ./testbench.py cru gbt stat
seconds RX EC/FEC #0 RX EC/FEC #1 RX EC/FEC #2 RX EC/FEC #3 RX EC/FEC #4 RX EC/FEC #5 RX EC/FEC #6 RX EC/FEC #7
1 0/ 394 lockedtodata/ 0 lockedtodata/ 3 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0
2 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 4 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0
3 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0
4 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 125 lockedtodata/ 4 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0
5 0/ 394 lockedtodata/ 0 lockedtodata/ 10 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0

Just as a counter test, I then disconnected the ONU fiber, and verified that the ttc clock went away:

[root@gemini COMMON]# python linkstat.py -i 3b:0.0 -l0-3 -v
globalgen: | ttc240freq 0.00 MHz | lcl240freq 240.47 MHz | ref240freq 240.47 MHz | clk not ok cnt 0 |
Wrapper 0: | ref freq0 240.47 MHz | ref freq1 0.00 MHz | ref freq2 0.00 MHz | ref freq3 0.00 MHz |

            Link #0: Wrapper 0 - Bank 0 - Link 0

            Status  |                     Counters
PLL locked: YES     |          Not ready: 3575538523

Locked to data: YES | Not locked to data: 2353903492
TX clock frequency: 240.47 MHz
RX clock frequency: 240.47 MHz

However, as you can see, link 0 still claims to be locked, even though the ttc clock is gone? Does that make sense? Also, the error counters in the loopback test don’t seem to affected either:

[root@gemini py]# ./testbench.py cru gbt stat
seconds RX EC/FEC #0 RX EC/FEC #1 RX EC/FEC #2 RX EC/FEC #3 RX EC/FEC #4 RX EC/FEC #5 RX EC/FEC #6 RX EC/FEC #7
1 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 7 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0
2 0/ 394 lockedtodata/ 15 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 1 lockedtodata/ 19 lockedtodata/ 0 lockedtodata/ 0
3 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 65 lockedtodata/ 3 lockedtodata/ 16 lockedtodata/ 0 lockedtodata/ 0
4 0/ 394 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 4 lockedtodata/ 16 lockedtodata/ 0 lockedtodata/ 0
5 0/ 394 lockedtodata/ 25 lockedtodata/ 0 lockedtodata/ 131 lockedtodata/ 1 lockedtodata/ 0 lockedtodata/ 0 lockedtodata/ 0

So what clock is the CRU using at this point?

And how can I verify that it is using the clock coming from the LTU?

Cheers,
Jo

2

Hello Jo,

Which version of cru-sw are you using?

Best regards,
Tuan

3

Yes, you don’t need to set the ONU number if you don’t use the PON TX

4

So how do I verify that the CRU is actually using the PON clock?

Also, Anton had me run this “calibration” script for the LTU, which does require the ONU number to be set in CRU, so with the LTU not “calibrated” what kind of clock does the CRU actually receive then?

5

Hello Jo,

If you run

python report.py -i PCIEID

it will display which clock is used:

Firmware short hash=0x5e6916ac
Firmware is dirty status= False
Firmware build date and time=20190220 135037
Altera chip ID=0x00540186-0x2858060a
=========================================================================================
LOCAL clock is selected
…
globalgen: | ttc240freq 240.47 MHz | lcl240freq 240.47 MHz | ref240freq 240.47 MHz | clk not ok cnt 0 |
…

and if the clock sources have the required frequency (it’s true that if there’s no clock at all, the linkstat error counters won’t change, and that is interpreted as OK, we will fix this). The LTU can used for downstream only, the CRU will anyway get the proper clock without the upstream calibration. (check with Anton how to omit the upstream calibration)

6

First of all, when I ran “report.py”, it did not report the clock used:

[root@gemini COMMON]# python report.py -i 3b:0.0

7

This is the latest develop branch, which one do you use? Can you please upgrade your cru-sw?

8

Is the software compatible with older version of the fw?
We should not mix new features with old firmware otherwise it can be difficult to debug.
Jo which firmware are you using?

9

I am trying to be in sync with the setup in our detector tests, which is using v2.6.0. Hopefully, we can soon upgrade to 2.7.0. As I am using 2.6.0 in fw, I am also using 2.6.0 in software