Status of the CSC Trigger Status of the CSC Trigger Darin Acosta University of Florida
Status of the CSC TriggerStatus of the CSC Trigger
Darin AcostaUniversity of Florida
CPT Week, April 2002 Darin Acosta2
OutlineOutline
Status of the CSC Track-Finder
Status of the CSC Local Trigger
PHOS4 Experience
TTC Jitter Measurements
CPT Week, April 2002 Darin Acosta3
The CSC TrackThe CSC Track--Finder CrateFinder Crate
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
SR/
SP
CC
B
BIT
3 C
ontr
olle
r
SP 2002 Card(3 Sector Receivers +
Sector Processor)(60° sector)
Clock and Control Board
Muon Sorter
To Global Trigger
From Trigger Timing
Control
From MPC(chamber 4)
From MPC(chamber 3)
From MPC(chamber 2)
From MPC(chamber 1B)
From MPC(chamber 1A)
Track-Finder crate (1.6 Gbits/s optical links)
To DAQ
MS
• Power consumption : ~ 1000W per crate• 16 optical connections per SP • Custom backplane for SP ↔ CCB and MS connections
CPT Week, April 2002 Darin Acosta4
CSC Track Finder BackplaneCSC Track Finder Backplane
Standard VME 64x J1/P1 backplaneA24/D16 (but D32 possible using address lines)
Standard VME J2/P2 backplane
Muon sorter
Clock and control
SRSP 6
SRSP 5
SRSP 4
SRSP 3
SRSP 2
SRSP 1
SRSP 12
SRSP 11
SRSP 10
SRSP 9
SRSP 8
SRSP 7
Custom GTLP 6U backplaneSignals specified, routing to commence
CPT Week, April 2002 Darin Acosta5
Sector Processor 2002 Board LayoutSector Processor 2002 Board LayoutDC-DC Converter
DDU FPGA
TLK2501 Transceiver
Optical Transceiver
Main FPGA
EEPROM
Front FPGA
Mezzanine Card
Phi Local LUTEta Global LUT
Phi Global LUTEEPROM
EEPROM
IndicatorsVME/CCB FPGA
From CCB
TRANSITION BOARD WITH LVDS TRANSCEIVERS
TO/ FROM BARREL
PT LUTTo MS
CPT Week, April 2002 Darin Acosta6
ME1 SR LUT TriadME1 SR LUT TriadCLCT PAT# - 4Q - 4 Phi_L -10
PhiB_L - 6CLCT_ID - 8L/R -1
CSC_ID - 4
Phi_L - 10WG_ID - 5
Phi_G -12
PhiB_L - 6Phi_L - 2CSC_ID - 4 WG_ID - 7
Eta_G - 7
PhiB_G - 5
CLK40P1
CLK40P2
CLK40
CSC_ID – 4WG_ID – 7
16 Bit
Transceiver
CLK40
C3
PHIL LUT 256K x 18
Flow ThroughSRAM
A18
A11
D16
C2
PHIG LUT512K x 36
FlowThrough
SRAM
ETAG LUT512K x 18
Flow ThroughSRAM
C3
16 Bit
Transceiver
D12
MAINFPGAMAIN
FPGAFRONTFPGA
FRONTFPGA
C2
To DTC4
Legend: A – Address LinesD - Data LinesC – Control LinesCLK – Clock
Phi_DT - 12
CSC ID - 4
45 synchronous memories for conversion of 15 track segments>64 MB per board ⇒ Need high VME bandwidth, broadcast
capability to identical chips, and crate broadcast capability to SPs
CPT Week, April 2002 Darin Acosta7
Main Sector Processor FPGAMain Sector Processor FPGA
MAINFPGAMAINFPGA
C4
D32
C2
2xD1
C2
CLK40
3xA22
3xC4PT
LUTPT
LUT3xD8
TRANTRAN
MUXMUX
CFGROMCFGROM
C3
D8
3xC1
2xD25
9xD24
3xD1
C3
3xD8
C1
3xD12 + 4xD1
3xD8
Legend:G – Number of Signal GroupsGxAn – G Groups of n Address Lines GxCn – G Groups of n Control Lines GxDn - G Groups of n Data Lines TRAN - Transceiver CCB&VME Int – Combined CCB and VME InterfaceCFG ROM – Configuration ROM CLK40 – Clock 40 MHz DDU- INT – Readout Interface
ME2/ME4
STUBS
ME2/ME4
STUBS
6xD24
ME1STUBSME1
STUBS
DTSTUBS
DTSTUBS
6xD9
DDUINT
DDUINT
A8CCB
&VMEINT
CCB&
VMEINT
9xD5
C9
D16
Placed on mezzanine cardFirmware written in “Verilog++” (see March TSWG talk)and implemented in ORCA as wellLatency only 4 BX
CPT Week, April 2002 Darin Acosta8
PT LUT & MUXPT LUT & MUXMAINFPGAMAINFPGA
D32
D32
A22
C4
PTLUTPTLUT
D8
PTLUTPTLUT
BackPlaneBackPlane
PTLUTPT
LUT
D32
D32
LVTT/GTLP
FRAME1
LVTT/GTLP
FRAME1
LVTT/GTLP
FRAME2
LVTT/GTLP
FRAME2
A22
C4
A22
C4
WIRED MUX
/OEAB=CLK40-90/OEAB=CLK40-270CLKAB=CLK80
Legend:
Ax – x Address LinesDx – x Data LinesCx - x Control Lines
Each Pt LUT is actually two 4M × 4 SRAMsData multiplexed at 80 MHz onto GTLP backplane to Sorter
CPT Week, April 2002 Darin Acosta9
DDU FPGADDU FPGAS1
C2 DDU
FPGA
DDU
FPGA
FRONTFPGA 5FRONTFPGA 5
FRONTFPGA 4FRONTFPGA 4
FRONTFPGA 3FRONTFPGA 3
FRONTFPGA 2FRONTFPGA 2
FRONTFPGA 1FRONTFPGA 1
MAINFPGA MAINFPGA
CCB&VMEFPGA
CCB&VMEFPGA
CONFIG.ROM
CONFIG.ROM
D32
D32
D32
D32
D32
D32
D32C3
TLK2501
TLK2501C5
D16
CLK80
S2
S1
C2
S1
C2
S1
C2
S1
C2
S4
C2
S1C2
CLK4
DDU FPGA collects data from input and output buffers for transmission to a single CSC DDU (aka FED)Optical link is bi-directional (if needed) for asynchronous xferCSC TF can exist in a separate partition from CSC chambers if
we are allowed to send one SLINK connection to DAQ
0
J. Gilmore CMS-EMU Meeting, Gainesville 4/5/02 4
Current DDU PrototypeCurrent DDU Prototype
CPT Week, April 2002 Darin Acosta10
SP2002 InterfacesSP2002 Interfaces● VME Interface – updated for chip-level broadcasts
(and crate broadcasts envisioned as well)● CCB Interface – using same interface as CSC Local Trigger● MPC Interface – updated (BC0 flag is sent through all TF
cards now: MPC->SP02->MS, To/From DT)MPC Data Validation - updated
● DDU Interface - updated (optional bi-directional) ● FM Interface – updated (RJ-45, 4 diff. signals)
Fast Monitoring Signals – updated (pinout)● MS Interface – updated (BC0)● DT Interface – updated (Synch/Calib -> BC0 ?)
Detailed accounting of all bits and protocols for these interfaces are specified in documents available fromhttp://www.phys.ufl.edu/~acosta/cms/trigger.html
CPT Week, April 2002 Darin Acosta11
DT InterfaceDT InterfaceData delivered from the Sector Receiver to the DT Track-Finder
@ 40 MHz using LVDS.Signal Bits / stub Bits / 3 stubs
(ME1: 30°) Bits / 6 stubs (ME1: 60°)
Description
φ 12 36 72 Azimuth coordinate η 1 3 6 DT/CSC region flag Quality 3 9 18 Derived from 4 bit Quality BXN – 2 4 2 LSB of BXN Clock – 1 2 Clock for data BC0 – 1 2 Bunch Crossing 0 Total: 16 52 104
Data delivered from the DT Track-Finder to the Sector Receiver @ 40 MHz using LVDS.
Signal Bits / stub Bits / 2 stubs (MB1: 60°)
Description
φ 12 24 Azimuth coordinate φb 5 10 φ bend angle Quality 3 6 Muon Flag 1 2 2nd muon of previous BX BXN 2 4 2 LSB of BXN Clock 1 2 Clock for data BC0 1 2 Bunch Crossing 0 Total: 25 50
CMS Note on DT/CSC interface ~ready to be released
CPT Week, April 2002 Darin Acosta12
SP2002 FPGA ChoicesSP2002 FPGA ChoicesFront FPGAs (3 Muons per FPGA, 5 FPGAs total)
Interfaces require at least 365 I/Os Choice: XC2V1000-?FF896C with 432 user I/Os
May be socketed (BGA soldered to high-density pin array)Main FPGA
Interfaces require at least 716 I/Os Choice: XC2V4000-?FF1152C with 824 user I/Os
Placed on mezzanine boardVME & CCB FPGA
Interfaces require at least 150 I/Os Choice: XC2V250-?FG456C with 200 user I/Os
DDU FPGAInterfaces require at least 110 I/Os
Choice: XC2V250-?FG256C with 172 user I/Os
Additionally, there are 51 SRAM chipsBoard will be dense! (Merger of 4 boards)
CPT Week, April 2002 Darin Acosta13
CSC TF Milestones for 2002CSC TF Milestones for 2002CSC Track-Finder:
Dec. 2001: Specify backplane connectionsApr. 2002: Conceptual design complete
Reviewed by US trigger group during 2 day workshop in March
May 2002: Schematics completeSep. 2002: Layout complete Nov. 2002: Finish fabrication of pre-production prototypes
2 month delay from original Sep. 2002 milestone
CPT Week, April 2002 Darin Acosta14
CSC Prototype Test ScheduleCSC Prototype Test ScheduleMPC logic tests: 10/1/02 – 12/31/02SP logic tests: 12/1/02 – 4/30/03MPC → TMB: 7/1/02 – 12/31/02MPC → SP: 3/1/03 – 4/30/03FAST site chain test: 5/1/03 – 6/30/03
Cosmic ray test with chambers and full chain of prototypesStructured beam test: 7/1/03 – 9/30/03 ?
Chain test with detectors in beam line, sometime in ’03DT TF → CSC TF: 7/1/03 – 9/30/03 ?
Crate test sometime before or after beam testTransition boards need to be designed
Note: we still have a lot of software to design and write to perform these system tests. Must interface previous trigger test code to FAST site test code, XDAQ, etc. Fortunately, we have several students and postdocs identified to help in this area. But we could use some guidance and examples on how to get started.
CPT Week, April 2002 Darin Acosta15
CSC Production and TestingCSC Production and TestingCurrently scheduled to complete CSC trigger production (CCB, MPC, SP) by Sep. 2004
Slice test scheduled for ~Oct. 2004If production not yet complete, will use current prototypes
Installation to begin Nov. 2004
Start integration with DAQ Mar. 2005
CPT Week, April 2002 Darin Acosta16
Special TriggersSpecial TriggersAfter commissioning, we’ll still want to monitor trigger efficiencies and alignment during data taking
Some ideas:Prescaled low pT thresholds
Useful to monitor efficiency, alignment, etc.Prescaled loose 2-station triggers or even single station
triggers Useful to monitor local trigger efficiency (BTI and LCT)
since this is dependant on analog chamber performance (i.e. can’t just run digital simulation)
Accelerator muon triggersCSC Track-Finder (and GMT) will have the ability to trigger
on muons traveling parallel to the beam axis during normal runningShould be useful for in-situ alignment studies of chambers
Hauser, CERN, April 2002 1
CSC Trigger Primitive Electronics Status
CSC Trigger Primitive Electronics Status
• Wire boards: ALCT (Anode Local Charged Track)
• Strip/coincidence/readout boards: TMB (Trigger MotherBoard)
Hauser, CERN, April 2002 2
80 MHz SCSI outputs(to Trigger Motherboard)
Main board for 384-ch type
Xilinx mezzanine cardPower, computer connectors
ALCT2001 BoardsALCT2001 Boards
Delay/ buffer ASICs,2:1 bus multiplexors (other side)
Input signal connectors
Analog section: test pulse generator, AFEB power, ADCs, DACs (other side)
Hauser, CERN, April 2002 3
ALCT FunctionsALCT Functions1. Inputs discriminated signals from AFEB front-
end boards, provides AFEB support:• Distributes power, shut-down, test pulse signals.• Sets and reads back discriminator thresholds.• Monitors board currents, voltages, and temperature.
2. Delay/translator ASIC on input does time alignment with bunch crossings.
3. Searches for muon patterns in anode signals. If found, sends information to trigger motherboard.
4. Records input and output signals at 40 MHz in case of level 1 trigger.
Hauser, CERN, April 2002 4
ALCT StatusALCT Status• Board has been thoroughly debugged
• Have extensive suite of testing software and an external test fixture.
• 3 versions:• ALCT2001-384 is in pre-production (30 boards, 40 mezzanine
cards)
• ALCT2001-672 have 6 prototype boards being assembled
• ALCT2001-288 have 6 bare prototype boards, assembly soon.
• Now integrating with software for chamber tests at U Florida and UCLA
Hauser, CERN, April 2002 5
TMB - Trigger MotherBoardTMB - Trigger MotherBoard• TMB2001 prototypes for use at
FAST sites and for system tests
• Produces cathode patterns from comparator outputs
• Correlates cathode and anode (from ALCT) patterns
• Sends chamber-level trigger decision to MPC
• Raw hits data “spooled” to DMB
• Interfaces to “everything”• CFEBs• DMB• CCB• ALCT• MPC• VME• RPC (later)• JTAG
CFEBs
ALCT(Future: will be via transition module) RPC via transition module
Hauser, CERN, April 2002 6
TMB Hardware StatusTMB Hardware Status• 17 TMB boards were produced (all FAST sites plus
development uses)• Bad job done by assembly company:
• Connectors soldered in crooked, boards could not be inserted• Connectors were removed and reinstalled properly by UCLA• Misc. errors (about seven per board)
• 2 boards have been fully corrected and debugged, including CFEB, ALCT, DMB, DDU, CCB interfaces
• 1 TMB was shipped to OSU for CFEB/DMB/DDU tests• The other TMB is for continued firmware development
at UCLA
Hauser, CERN, April 2002 7
PHOS4 ProblemPHOS4 Problem• PHOS4 delay chips are convenient, so were used in
TMB and CCB (Clock&Control Board)• But…
• They can only be reset once. This can be a big problem.• We have had a lot of difficulty getting them initialized properly
(months of work). Sometimes power cycling a crate is the only solution (ugh).
• They produce an asymmetric output clock, especially if one PHOS4 used in series with another.
• Delay = 0 setting gives about 3 ns variance between chips, not very tight.
• PHOS4 power-on sequence work-around allows board to operate• Wait 1 second, send a reset, wait 1 second, program via I2C bus• (Previously, PHOS4 chips did not power up in clocking state (dead
board!), 5ns delay sometimes became 10ns etc.)• Still have very asymmetric (58/42%) output clock
Hauser, CERN, April 2002 8
TMB-CFEB-LVDB TestTMB-CFEB-LVDB Test
Hauser, CERN, April 2002 9
ALCT-TMB TestingALCT-TMB Testing
Hauser, CERN, April 2002 10
Virtex-II For Future Use in TMBVirtex-II For Future Use in TMB• Allows faster
performance and some clocking and I/O advantages
• Initial layout done• Designed for “medium”
(896 ball) FPGAs for TMB (or ALCT)
• XC2V1000, 1500, 2000 chips
• Ball locations compatible with “large” (1152 ball) FPGAs for e.g. Sector Processor
• XC2V3000, 4000, 6000, 8000, 10000 chips
PHOS4 features
•CERN designed 4-channel delay ASIC with 1 ns delay precision•Radiation Hard•Programmable (write-only) over I2C bus
Issues with PHOS4
The PHOS4 is intended for use with non-interruptable clock source (since the chip utilized DLL circuitry)If clock is interrupted, then the behavior is unpredictable
wrong delays“dead-looking” channelsno direct way to identify what is wrong with the chip Only power cycling can help
Lack of dedicated “reset” pin Delay settings are not readable back via I2C bus
We have suggested that the chip be redesigned
Clock Interruptions?
In our opinion, the clock should never be interrupted (except on power cycling)TTC tree is designed so clock won’t be interruptedA question to CMS/LHC is “How often can we expect the clock to be interrupted?”
Interruptions
What to do if interruptions are a problem?
PHOS4 can be simply removed from the CCB boards (all 4 socketed) and clocks wired directly to LVDS transmitters
no individual slot-to-slot and module-to-module clock adjustmentsno extra effort to program I2Csame PCB layout, minimal on-board changes
Another Question
All clock lines in the custom peripheral backplane are point-to-point LVDS of the same length. Do we really need to adjust the 40Mhz clock from slot to slot and from module to module?
Another Alternative
If fine clock adjustments on the backplane are still needed
PHOS4 can be replaced with another device, for example, 3D7408 proposed by OSU
1-channel commercial CMOS programmable delay chipThis device has 45/55 output duty cycle instead of 50/50 at 40Mhz (tested at Rice). Is it acceptable for DMB/TMB/ALCT?layout changes on a CCB board requiredRadiation tolerance?
TTC IssuesTTC Issues
Trouble ReportedTrouble Reported
At the last CMS week (and At the last CMS week (and continuing) reports from many continuing) reports from many groups that they can not drive optical groups that they can not drive optical links with a TTC derived clocklinks with a TTC derived clock(Sorry, but nobody [but us] is (Sorry, but nobody [but us] is documenting what they are doing, so documenting what they are doing, so I can’t point you to anything written I can’t point you to anything written on the topic)on the topic)Our report is at Our report is at
http://http://bonnerbonner--ntserverntserver.rice..rice.eduedu//cmscms/jitter./jitter.pdfpdf
The IssueThe Issue
TTC group has specified that they TTC group has specified that they will deliver clock good to 50ps rms.will deliver clock good to 50ps rms.Groups using CERN GOL Groups using CERN GOL (not us)(not us)claim claim TTCrxTTCrx can’t drive their linkscan’t drive their linksReports of jitter measurements Reports of jitter measurements ~500ps~500psAt last CMS week microelectronics At last CMS week microelectronics group agreed to improve group agreed to improve TTCrx TTCrx
( time scale?)( time scale?)
Our MeasurementOur Measurement
Mezzanine Card ECP680-1102-630C ECP 680-1102-610B
TTCrx ASIC operating voltage +5.0V +3.3V +5.0V
Clock40Des1 jitter, ps (no BC, no L1A) 153 170 330
Clock40Des1 jitter, ps (BC commands + L1A) 183 215 360
New TTCrx Old TTCrx
Our TestTTCVI
TTCVX
OPTO
OPTO
1 m
100 m
40 Mhz
Clockmultiplier
•••
•• •
•
••••••
CCB
BIT3
ERROR
TTCrx
PC
100 m
VME 9U VME 6U
COPPER CABLE OPTICAL CABLE
• OLD AND NEW TTCrx BOARDS WERE TESTED WITH40.00 Mhz CLOCK SOURCE FROM TTCvx MODULE
• 40.00 Mhz CLOCK WAS MULTIPLIED BY 2 BY AV9170 CHIP• NO ERRORS OBSERVED IN PRBS TEST FROM ONE OPTOBOARD
TO ANOTHER AT 80.00 Mhz (BER < 10-13 c-1)
Our ResultOur Result
Clock jitter is lower for the newest Clock jitter is lower for the newest TTCrxTTCrxASIC (Version 3.1, 12/01)ASIC (Version 3.1, 12/01)Jitter increases if broadcast commands Jitter increases if broadcast commands and L1A are transmitted from the TTC and L1A are transmitted from the TTC systemsystemJitter distribution for ASIC Ver.3.1 looks Jitter distribution for ASIC Ver.3.1 looks “normal” (unlike previous version)“normal” (unlike previous version)Clock jitter is lower if the new ASIC is Clock jitter is lower if the new ASIC is powered from +5Vpowered from +5VJitter introduced by any of two Jitter introduced by any of two TTCrxTTCrxASICsASICs and other components in the clock and other components in the clock distribution circuitry at our testing setup is distribution circuitry at our testing setup is tolerable for TLK2501 transceivers tolerable for TLK2501 transceivers operating at 80.00 operating at 80.00 MhzMhz using prototype using prototype MPCMPC--SR linkSR link