Problem, Root Cause, Tests
& planned GTRC7 Resolution.
The charge is given in Appendix A, the committee and other attendees in Appendix B, and the agenda in Appendix C.
Materials submitted are at htttp://scipp.ucsc.edu/~hartmut/GLAST/GTRC.
The committee was impressed by the cooperation of the TKR and ELEX groups and the time spent on the review, given the many other pressing duties.
Following are findings and conclusions related to the charge, followed by a list of recommendations.
A Findings:
During the TKR MCM and mini-tower testing, two problems were found:
· A logical error in the handling of the time-over-threshold (which is used to estimate the charge deposited in the SSDs). This error occurs when the falling edge of the TOT coincides with the raising edge of a consecutive pulse, and leads to a time out.
· A race condition in the inter-chip communications of the GTRCs, which severely limits the operational range in frequency and/or voltage.
The TOT problem has been verified in VHDL and on bare die, after the test program was modified. It showed up in mini-tower testing with a random source in fairly large data sets. The data taken with the mini-tower seem to confirm the findings, but need confirmation.
à Recommendation 1, 2.
The timing problem, including observed delays etc., is understood and has been corrected on 6 GTRC6 with Focused Ion Beam (FIB) surgery, after which the GTRC6 worked well.
The TKR and ELEX group have submitted ASICs with the following changes:
· GTRC6B: fix the timing problem without re-route.
· GTRC7: fix timing problem, AND change TOT logic, complete re- route.
· GTCC1_TOT1 TEM ASIC containing the TOT logic.
The GTRC6B has minimal risk, because the existing GTRC layout will be used with only two local modifications, which have been proven to work after FIB surgery on 6 ASICs. These modifications restored frequency and voltage margins, but will not run with TOT enabled.
GTRC7 will restore frequency and voltage margins, and will run with TOT enabled. The corrected TOT code has been run successfully in the TEM FPGA attached to the mini-tower with 1M triggers at 300 Hz without problem.
The modified GLAST Tracker Cable-Controller ASIC GTCC1_TOT1
would replace the ordinary GTCC1 residing in the TEM and would be able to run
the TOT code in conjunction with the GTRC6B.
Operation within the frequency and power budget should be possible with either the GTRC7 + GTCC1 combination or the GTRC6B + GTCC1_TOT1 combination.
The LAT TKR readout controller ASIC GTRC is designed, simulated, laid out by the SLAC LAT electronics group. The testing is done in the LAT TKR group, mainly at UCSC, both on probe card and on MCM, and on the mini-TKR tower at INFN Pisa and at SLAC, with support from the LAT I&T group. The test vectors used by the designer during design verification are different from those used in the testing. High rate testing with random realisticly long pulses, which could help identifying further problems with the GTRC, has not been performed yet.
à Recommendations 3, 4, 5, 6, 7, 8, 9
Past GTRC6
testing could have benefited from better coordination of design, layout and
testing, and from realistic system tests including high rate random triggering.
This should be lessons learned for future GTRC6 and GTRC7 testing.
The TKR and ELEX systems assume the same voltage both at the TEM power supply and at the TKR ASIC pad. This does not take into account the voltage drop on cables and polyswitches.
Up to now, the proper workings of protection resistors on ASICs and the polyswitches in case of a damaged ASIC has not been ascertained.
The design and verification tools used are not entirely at industry standards.
à Recommendations 10, 11, 12
1. Resolve GTRC6 Mini-Tower test results (No TOT timeout with Van der Graaf). Estimate the rate of TOT overlap and compare with observed time-out rate.
2. Test GTRC7 VHDL code in FPGA with a long TREQ (GTFE pulse) and many TACK. The length of the TREQ should be longer than the longest pulse you expect from heavy ions in the space.
3. Improve handling of Test Vectors for GTRC6 & GTRC7:
Designer should propose test vectors used in testing.
Designer should verify the % of coverage in VHDL
4. Provide documentation of GTRC – describe the different states of the state machine.
5. Perform high rate triggering tests with single tray and radioactive source.
6. Continue system tests with multiple tray/multiple MCM using TEM, preferable with ELEX/DAQ group involvement.
7. Continue to perform coordinated, but independent test with present mini-tower (at SLAC) and the new mini-tower (at INFN)
8. Test results vs. specifications: Provide a matrix comparing specifications described in LAT-SS-00152-02 and the test results.
9. Produce new schedule for GTRC / MCM production and testing, taking in account downtime of foundry.
10. Review requirements for normal / worst cases operations (temp, voltage, freq), together with ELEX group.
11. Perform tests of interplay of Polyswitches and protection resistors.
12. For future ASIC programs at SLAC improve robustness of ASIC development flow
- e.g., digital simulation instead of SPICE
- test vector generation should be done from simulations
Charge to Committee (as understood by the chair):
1. Review if the root cause for these issues has been identified
2. Review if the design solution is appropriate
3. Review the completed and planned test & simulation verification activities
4. Identify any recommendations for future verification activities
5. Add any other insights and recommendations
Hartmut Sadrozinski - chair
Neil Johnson - Cal Subsystem Manager (by phone)
Helmuth Spieler - LBL Off Project Support
Lou Fetter - GSFC Consultant
Elliott Bloom - LAT I&T Test Manager
Jim Martin – LAT IPO
In Attendance:
Robert Johnson – TKR Subsystem Manager
Gunther Haller – Elex-DAQ Subsystem Manager
Sandro Brez – TKR Subsystem, INFN (by phone)
Luca Latronico – TKR Subsystem, INFN (by phone)
Hiro Tajima – TKR Subsystem (by phone)
Mutsumi Sugizaki – TKR Subsystem
Mike Huffer – Elex Subsystem
Diether Freytag – Elex subsystem
Oren Milgrome – Elex subsystem
Eduardo Do Couto e Silva – I&T
Rick Claus – I&T
Peter Michelson - LAT IPO
Lowell Klaisner - LAT IPO
Dick Horn – System Engineering (by phone)
Nick Virmani – LAT Parts Engineer (by phone)
Andy Lankford – UCI Off Project Support (by phone)
Agenda:
A. Specifications, Functions of GTRC R. Johnson 5 min
B. Design (including methodology) G. Haller et al. 15 min
C. Testing of GTRC6
a. Wafer Testing R. Johnson et al. 10min
b. MCM Testing at UCSC R. Johnson et al. 10 min
c. Testing at SLAC n.a. 10 min
d. Testing at INFN L. Latronico 15 min
e. Mini-tower testing at SLAC H. Tajima/E. do Couto10 min
D. GTRC6 Problem & Design changes in GTRC7 R. Johnson/ G. Haller 20min
E. Discussion on points 1-2 of the charge Committee 15min
F. Plan for further Tests GTRC6 R. Johnson at al. 20 min
G. Test Plan for GTRC7 R. Johnson/ G. Haller 20 min
H. Discussion on points 3-4 of the charge Committee 15min
I. Discussion on points 5 of the charge Committee 15min
J. Committee Discussions Committee only 20 min
K. Close-out 10 min