5 October 20002nd ATLAS ROD Workshop1 The MROD The MDT Precision Chambers ROD Adriaan König University of Nijmegen.

5 October 2000 2nd ATLAS ROD Workshop 1

The MROD

The MDT Precision Chambers ROD

Adriaan König

University of Nijmegen

5 October 2000 2nd ATLAS ROD Workshop 2

Contents

• System Overview

• MROD-0 Prototype

• MROD-1 Prototype

• Performance Study

• FE Parameter Loading & Initialization

• Names

5 October 2000 2nd ATLAS ROD Workshop 3

System Overview

6 x MROD1.28 Gbit/s S-Link to ROB

TDC 1

TDC 18

CSM18 x

TDC 1

TDC 18

CSM18 x

Chamber Tower

5 October 2000 2nd ATLAS ROD Workshop 4

TDC Functionality

• 24 channels, 0.78 ns bin size

• entirely data driven: records time stamp for each hit (leading and/or trailing edges)

• stores hits in internal derandomizing buffer

• upon receipt of a L1A, it ouputs the relevant hit data words on a serial output link (40 Mbit/s) with header and trailer words

5 October 2000 2nd ATLAS ROD Workshop 5

CSM Functionality

Serial to Parallel&

Clock Domain Separator

40 Mbit/sData/Strobefrom TDC

18 x

Serial to Parallel&

Clock Domain Separator

40 Mbit/sData/Strobefrom TDC

Separator

(S-Link)

1 Gbit/s

1 Start bit32 Data bits 1 Parity bit 1 Stop bit35 bits @ 25 ns = 875 ns

1 Separator word (S)18 TDC data words19 words in 875 ns 87 MB/s

S1

18

CSM

5 October 2000 2nd ATLAS ROD Workshop 6

TDC0, word 1

TDC2, word 4

TDC3, word 2

TDC0, word 1

TDC1, word 3

TDC2, word 5

TDC3, word 3

TDC3, word 0TDC2, word 0TDC1, word 0

TDC1, word 1

TDC1, word 2

TDC2, word 1

TDC2, word 2

TDC2, word 3

TDC3, word 1

Build events in a partitioned memoryfrom TDC data fragments(tdc 1) 000…000

Separator word

Separator word

Separator word

Skip (do not store)

Check (do not store) MROD Function

time

(tdc 0) 000…000

TDC0, word 0

5 October 2000 2nd ATLAS ROD Workshop 7

MROD Throughput

MROD

1.28 Gbit/s

( 128 MB/s)

MROD input

MROD input

MROD input

MROD input

MROD input

MROD input

MROD output

Average 5 hits per TDC + header + trailer = 7 words/eventPer tower of 6 chambers max. 88 TDCs * 7 600 words/event (= 2.4 kB/event)Worst case est.: @ 100 kHz L1A rate 240 MB/s per MRODCalculation based on actual tower layout (J.Chapman): max. rate < 60 MB/s per MROD

S-Link

S-Link

S-Link

S-Link

S-Link

S-Link

S-Link

5 October 2000 2nd ATLAS ROD Workshop 8

MROD Form Factor• 9 U VME board (single slot), 6 inputs, 1 output• Optionally 2 extra inputs with “extension” board to accommodate special towers (> 6 chambers)• S-link interfaces on main board• SHARC II (ADSP21160), 2.5 x faster than 21060

• 1 MROD Crate contains: 12 MRODs (12 Segments) Max. 4 MROD Extension Boards 1 Standard (?) Crate Master with Ethernet Interface (DetDAQ) 1 TTC-Rx Interface Module 1 Busy Module ?? 1 DCS Interface Module ??

• @ 192 towers: 192/12 = 16 MROD Crates (1 per Sector)

5 October 2000 2nd ATLAS ROD Workshop 9

MROD-0 Prototype

MRODOUT

SHASLINK

PCISHARC

MRODIN

MCRUSH

sortedTDC-data

overSHARC Link

5 October 2000 2nd ATLAS ROD Workshop 10

1 MBZBT

Memory

SHARC

FPGA

Data FIFO

TetrisRegister

InputOutput

FIFO Control

Control/StatusError signaling

6 S

harc

link

s@

40

MB

/s e

ach

FIFO

Length FIFO

MCRUSH

MROD-0 Input Channel

5 October 2000 2nd ATLAS ROD Workshop 11

MROD-0 Output Channel

SHaSLINK

PCI 9054 SHARC

Altera10K10A

S-link@ 160 MB/s

6 SHARC links@ 40 MB/s each

PCI

bus

5 October 2000 2nd ATLAS ROD Workshop 12

(SHARC)

5 October 2000 2nd ATLAS ROD Workshop 13

MROD-1Prototype

Memory

SHARC

FPGA

SHARC(2x)

MemoryFPGA

3x (in total)

VME64x

TTCInterface

Memory

SHARC

FPGA

MemoryFPGA

Sharc Links

5 October 2000 2nd ATLAS ROD Workshop 14

SHARC-II

5 October 2000 2nd ATLAS ROD Workshop 15

The ADSP-21060 and the ADSP-21160 SHARCs

• 40 MHz / 80 100 MHz CPU (SIMD mode)

• 512 KB / 512 KB internal memory

• 6 x 40 / 80 100 MB/s links. Throughput of all links simultaneously is 160 / 480 600 (?) MB/s, without disturbing the CPU.

• No handshaking on links, but hardware XON-XOFF protocol,

• 10 / 14 DMA channels

• Support for bus arbitration: at max. 6 SHARCs can be connected to a common bus without glue logic. Each SHARC can access the internal memories of each other SHARC. The SHARCs also provide support for a so-called host interface, which can act as an additional master on the common bus.

• Fast interrupt servicing due to the presence of shadow registers

• Two 40 Mbit/s / 40 50 Mbit/s (at max.) synchronous serial ports

• Can be booted via link 4

5 October 2000 2nd ATLAS ROD Workshop 16

MROD-1 Form Factor

•9 U VME board, 6 inputs, 1 output• S-link interfaces on daughter boards

• SHARC II (ADSP21160), 2 x faster than 21060

(3 for input, 2 for output processing) • Altera APEX FPGAs, 200k gates• TTC interface (over back plane)• VME64x interface

Motherboard

Output

Input

Input

Input

S-linkdaughterboards

5 October 2000 2nd ATLAS ROD Workshop 17

MROD-1 Status & Planning

• VHDL design of FPGAs almost finished.

• Modules available by 1st April 2001.

• Extensive tests and performance measurements at NIKHEF.

• System integration tests with CSM.

• System integration tests with ROB and DAQ test set-up (possibly in test-beam).

• Read out of BOL test stand at NIKHEF.

5 October 2000 2nd ATLAS ROD Workshop 18

MROD Performance Study

MRODIN

MRODIN

MRODIN

MRODIN

MRODIN

MRODIN

MROD

MRODOUT

CSM

ROB

CSM

CSM

CSM

CSM

CSM

5 October 2000 2nd ATLAS ROD Workshop 19

MROD Emulation Hardware

MRODOUT ROBIN ROBSIM

SHASLINK CRUSH SHASLINK

PCISHARC

S-Link

(PCI-)interfaceto host PC

Module typexxxxx

SHARC-link

CSMSIM

SHASLINK

0 1

0MRODIN

(3x)

MCRUSH

0

2

3

4

4

RoIRR

RoIR/T2DR

RoID/T2OD

MROD-0

44

2 2

0 0

TDC-data

fragment lengths

sortedTDC-data

sorted +merged

TDC-data

sortedTDC-data

optionally double/tripleMRODIN output thus

simulating 2 or 3 MRODINs

event fragment lengthsvia SHARC-link simulates

future MROD-1 functionality

Region-of-Interest Requests,Decision Records, etc., everything

needed to run a ROBIN simulation

1 3

5 October 2000 2nd ATLAS ROD Workshop 20

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MROD

CSM-simulator performance

0.0

50.0

100.0

150.0

200.0

250.0

300.0

350.0

400.0

450.0

0 5 10 15 20 25 30 35

Words/TDC

Eve

nt

rate

[kH

z]

5 October 2000 2nd ATLAS ROD Workshop 21

MRODIN (1x) + MRODOUT + ROBIN, 18 TDCs

0,0

20,0

40,0

60,0

80,0

100,0

120,0

140,0

160,0

180,0

0 5 10 15 20 25 30 35

Words/TDC

Eventrate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MRODMRODPerformanceStudy Results

5 October 2000 2nd ATLAS ROD Workshop 22

MRODIN (1x) + MRODOUT + ROBIN, 6 TDCs

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

160.0

180.0

0 5 10 15 20 25

Words/TDC

Eventrate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MRODMRODPerformanceStudy Results

5 October 2000 2nd ATLAS ROD Workshop 23

MRODIN (2x) + MRODOUT + ROBIN, 18 TDCs

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

160.0

180.0

0 5 10 15 20 25 30 35

Words/TDC

Eventrate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MRODMRODPerformanceStudy Results

5 October 2000 2nd ATLAS ROD Workshop 24

MRODIN (2x) + MRODOUT + ROBIN, 6 TDCs

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

160.0

180.0

0 5 10 15 20 25

Words/TDC

Event rate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MRODMRODPerformanceStudy Results

5 October 2000 2nd ATLAS ROD Workshop 25

MRODIN (3x) + MRODOUT + ROBIN, 6 TDCs

0.0

20.0

40.0

60.0

80.0

100.0

120.0

140.0

160.0

180.0

0 5 10 15 20 25

Words/TDC

Event rate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT ROBIN ROBSIM CSMSIM MRODIN

MRODMRODPerformanceStudy Results

5 October 2000 2nd ATLAS ROD Workshop 26

MROD Performance Analysis

• Measured event rate for single output SHARC @ 40 MHz with 3 input channels: event rate min(50,1000/(10 + #words-per-CSM/6) kHz.

• MROD-1 uses SHARC-II @ 80 MHz: both processing speed and bandwidth increase proportionately event rate 100 kHz ?

• ‘Final’ MROD: SHARC-II @ 100 MHz.

5 October 2000 2nd ATLAS ROD Workshop 27

FE parameter loading/initialization

TTC

TDCsASDs CSM MROD ROB

DCS

MDT-DAQ

JTAG routing:Mezzanine boards

5 October 2000 2nd ATLAS ROD Workshop 28

JTAG Usage

• Initialize/Set/Reset ASD/TDC/CSM parameters• Reload CSM Flash Memory (if/when needed)• Calibration sequence:

1: JTAG enables calibration pulses in the ASD

2: TTC signals the CSM to send a test pulse

3: TTC provides calibration trigger

No calibration during regular data taking.

5 October 2000 2nd ATLAS ROD Workshop 29

MROD Names (NIKHEF and Univ.of Nijmegen)

• Henk Boterenbrood• Peter Jansweijer• Gerard Kieft• Adriaan König• Jos Vermeulen• Thei Wijnen• NN (Post-doc vacancy at Univ.of Nijmegen:

www.hef.kun.nl/vac-postdoc.html)