Outline

GBT, an integrated solution for data transmission and TTC distribution in the SLHC

Perugia, 17 May 2006

A. Marchioro, P. Moreira, G. PapottiCERN PH-MIC

2006/05/17 GBT 2

Outline

GBT an upgrade for: Data, TTC and Slow Control links

GBT operation: Operation and Data modes

Some link topologies Line codes and error correction System bandwidth New functionality Practical developments Summary

2006/05/17 GBT 3

Gigabit Bidirectional Trigger and Data Link

The target is to build a system based on a single ASIC which can provide a complete (link) solution for: Timing Trigger Slow Control Data Transmission

It will be the SLHC replacement of: TTCrx + GOL + CCU (or equivalent device)

For data transmission links: Increase the bandwidth Add redundancy to coupe with higher SEU rates

For triggers links: New CMOS technologies open a whole range of possibilities as is

discussed next

2006/05/17 GBT 4

TTC Link Upgrade Rationale Must be ready for SLHC higher demands Must be built on the past experience:

Merits Weak points

Profit from today's submicron technologies: Higher performance:

Larger bandwidth Complex commands can be sent and executed in a single bunch

crossing interval Higher integration:

Increased functionality Bi-direction link Robust error correction Robust handling of SEUs

Profit from today’s optoelectronics technologies: Bi-directional optical network

2006/05/17 GBT 5

Current TTC System Limitations

Transmission of a single trigger type Unidirectional system, return requires additional network

Not adapted to build a complete slow-control network Channel-B commands are:

Relatively slow Individually addressed commands “non-deterministic”.

No error correction on trigger bits (channel-A) Unspecified clock frequency if input link not present Clock quality not sufficient for GHz links

2006/05/17 GBT 6

The new GBT Transceiver GBT transceiver operation:

Operation modes: Trigger:

TTC functions Link

General purpose Simplex/Duplex

Data transmission modes: Continuous

Data is continuously transmitted

Packet Data is transmitted in

bursts of packets

RX TX

GBT

Link

User Application Side(Electrical world)

2006/05/17 GBT 7

GBT Operation Modes Continuous mode (think about a traditional optical link):

Each link is 100 % occupied by a single transmitter. The transmitter/receiver pair is fully synchronous at all times This is the case for:

A trigger source sending data to several trigger destinations Synchronous point-to-point data link

Packet mode (think about a backplane bus): Common Transmission medium is shared:

A transmitter can only send data upon the request of a master transmitter

Several devices can share the same medium and thus communicate with the same destination without collisions

This is the case for: Trigger return link (bus) Asynchronous point-to-point data link

2006/05/17 GBT 8

GBT System Configuration 1: Broadcast Network

Down-path: Passive Optical Tree Current TTC system architecture One source to N destinations For large N, an high optical power source is required Operation mode: continuous

2006/05/17 GBT 9

GBT System Configuration 2: Broadcast Network with O/E/O Repeaters

Down-path: Passive/Active Tree Passive power splitting with electrical regeneration One source to N destinations Optical or electrical: 1-to-8

Moderate optical power at each transmitter Operation mode: continuous Moderate increase in latency: o/e-r-e/o

2006/05/17 GBT 10

GBT System Configuration 3: Point-to-Point

Up/Down-paths: Optical Full bandwidth available for

data transmission Simplex/Duplex operation Operation mode:

continuous

2006/05/17 GBT 11

GBT System Configuration 4: Bidirectional One-to-N / N-to-One Down/up-links: Passive Optical Trees

Passive optical tree in both directions Down-link: 1 source to N destinations Fan-In-Out: 1-to-8 or perhaps 1-to-16

For moderate power optical source Possibility of WDM to reduce the optical source power

Operation mode: Down-link: continuous Up-link: packet Up-link under control of the master transmitter

2006/05/17 GBT 12

Line Codes and Error Correction To deal with higher SEU rates in SLHC the following scheme is

proposed: 64-bit data is first scrambled for DC balance The scrambled data is Reed-Solomon encoded: 16-Bit CRC field An 8-bit redundant header is added to form a frame This results in an 88-bit frame.

Line rate 40 MHz × 88-bit = 3.52 Mbit/s To minimize the dead-time due to a loss of synchronization the

scrambler is designed as self synchronizing: One LHC clock cycle is enough to synchronize the scrambler

The efficiency of the line encoding is: 64/88 = 72.7 %

scrambler64 64 646488 88RS enc. + header

RS dec. + header descramblerchannelscrambler64 64 646488 88RS enc. +

headerRS dec. +

header descramblerchannel

2006/05/17 GBT 13

Data Rate and User Bandwidth Transmission data rates must be multiples of the (S)LHC

bunch crossing frequency: Trigger system remains synchronous with the accelerator

cycles Fixed latency communication channels 130 nm CMOS technology:

Raw 3.2 Gbit/s ok, (~5 Gbit/s maybe feasible) Transmission of 88-bits encoded at 40 MHz (the LHC rate) Effective data bandwidth of 2.56 Gbit/s (for 3.2 Gbit/s raw)

90 nm CMOS technology: Raw 6.4 Gbit/s ok, (~10 Gbit/s maybe feasible) Transmission of 128-bits properly encoded at 40 MHz (the LHC

rate) … or transmission of 64-bits encoded at 80 MHz (the SLHC+

rate) Effective data bandwidth of 5.12 Gbit/s (for 6.4 Gbit/s raw)

2006/05/17 GBT 14

The GBT as a TTC/Controller transceiver

Receiver

Transmitter

1-WireI2C Master (3 – wires) (×4)JTAG Master (4 – wires) (×1)SPI Master (3 + 5 – wires) (×1)Parallel port (4 × 8-bit) (×1)Memory bus (A 16-bit, D 16-bit)Phase programmable clocks (×4)Trigger emulator (4 – wires) (× 1)Event CounterBunch CounterReceiver controlLink Flow controlTransmitter controlI2C SlaveJTAG Port

datatrigger

data

GBTGBT

2006/05/17 GBT 15

GBT Block Diagram

JTAGSlave

LHC Bunch Emulator

High ResolutionClock Phase Adj

Self TestLogic

ConfigurationRegisters

Parallel/SerialConverter

Encoder& FEC

IOInterface

CDR& PLL

Ser/ParConverter

Decoder &Error Correction

IOInterface

SEU monitor& Correct

Watchdog& Init Logic

Clock Generator

Lim

Am

p

RX

TX

Command Decoder& Trigger

Logic

Laser Diode

Cable

Needs 3*logic + Voting

I2C &P-Port

2006/05/17 GBT 16

Prototypes

Three GBT building blocks were prototyped in 130 nm CMOS: Laser driver (Gianni Mazza, INFN Torino) Encoder / decoder (Giulia Papotti, CERN) Limiting Amplifier (Paulo Moreira, CERN)

Tape out in December 2005 Dice received in March 2006 Two of the circuits already successfully tested:

Encoder / decoder Limiting Amplifier

2006/05/17 GBT 17

Encoder

64

64

6464

64

80

64

80

80

80

80

bypass scr

IDLE

demux scrambler

RS enc

serializer

scr data in encoded 80

to be encoded

scrd out

ser data in

to be ser

data register

control

8

data loaded[0]

resetn

resetn

ser start

ser data

data loaded[2]

enable

scr enable

enable

hold

load

resetndata enableclock

scr onRS on

64

bypassRS

scrd idledata out

data_e_strb

synch out

2006/05/17 GBT 18

Decoder

80

864

idle for scr 2

descr.

RS dec

mux

mux w

ord in

scr out

scrd in data dec out

framesynch

shift register

control

frame syncsynch in

rs load

des resetn

resetnclock

scr onRS on

RS

/scr bridge

bypassRS

1

data out

data_d_strb

64

idle for scr 1

bypassRS

2

bypass scr

64

64

data in

RS

on dec out

80

data frame

data to decode[0]

copy sr

64

6464

64 64

rs hold

rs enable

64

64 64

rs input

dec whole

80 64

error

data to decode[2]

mux enable

mux resetn

des enable

data/idle

fs on

2006/05/17 GBT 19

Limiting Amplifier

Specifications:• Data rate: 3.60 Gbit/s• Gain: > 55 dB• Bandwidth > 2.52 GHz• Equivalent input noise: < 1 mV• Minimum input signal (differential): 10 mV• Maximum input signal (differential): 600 mV

2006/05/17 GBT 20

Limiting AmplifierOffset

CancellationBias Gain

CellGainCell

GainCell

GainCell

Output Buffer

Size: 194 m × 194 m

2006/05/17 GBT 21

Limiting Amplifier

3.35 Gbit/s 1 Gbit/s

2006/05/17 GBT 22

Summary

We propose a Single ASIC solution for: Trigger Links; Data Acquisition Links; Slow Control Links.

For TTC applications we try to eliminate the drawbacks of the previous system: The link becomes bidirectional; Allows execution of complex commands in a single bunch crossing

interval; Trigger Decisions as well as data become protected against SEUs.

The system allows flexible link topologies Specifications are still evolving for which we need the

feedback of the potential users Some of the building blocks of the transceiver are “universal”

and for some of them we have already attempted practical implementations: Laser driver Encoder/decoder: Line code and CRC Limiting amplifier