The CMD-3 Data Acquisition and Control System .

10-th INTERNATIONAL CONFERENCEON INSTRUMENTATION FOR

COLLIDING BEAM PHYSICS Novosibirsk, Russia, February 28 - March 5, 2008

The CMD-3 Data Acquisition and Control System.

A.Ruban*, A.Aulchenko, K.Kakhuta, A.Kozyrev, A.Selivanov, V.Titov, Yu.Yudin

BINP, Novosibirsk.

March 5, 2008

The CMD-3 Data Acquisition and Control System.

March 5, 2008

VEPP-2000 Collider LayoutVEPP-2000 Collider Layout

Historical Siberian Snake Round Beams 2E max = 1.4 GeV 2 GeV L(1.4 GeV) = 31030 sm-2s-1 1031 sm-2s-1

L(2 GeV) =1032 sm-2s-1

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMDCMD – – 3 Subsystem Layout3 Subsystem Layout

1 – Vacuum Pipe

2 – Drift Chamber

3 – BGO Endcap Calorimeter

4 – Z – Chamber

5 – Superconducting Solenoid CMD-3

6 – LXe Calorimeter

7 – CsI Calorimeter

8 – Yoke

9 – Superconducting magnet lenses

Mu-System and TOF not showed

Number of Channels DC – 1260 Wires Time, ChargeX2 ZC – 48 Sectors Time, Charge - 512 Stripes Charge BGO – 680 Crystals Charge CsI – 1152 Crystals Charge LXe – 264 Towers Time, Charge - 2112 Stripes Charge Mu – 48 Counters TimeX2, ChargeX2 TOF - 16 Counters TimeX2, ChargeX2 FLT – 400 Chanels Words

Total up to 10k channels

General Requirements to CMD-3 DAQ

The CMD-3 Data Acquisition and Control System.

March 5, 2008

The CMD-3 Data Acquisition and Control System.

March 5, 2008

General Requirements to CMD-3 DAQ

Time Average Speed – 1k Evtps Dead Time – less then 5% Common Stop Jitter – less then 20ps

Reliability End-to-End testability On-Line Data Check DAQ Error Rate - less then 0.1%

Density Low EMI Low Power

Low-cost Legacy Interface if possible

The CMD-3 Data Acquisition and Control System.

March 5, 2008

The way we choose

Serial Bus rather then Parallel Point-to-point media connection Low signal strength differential media

System Clock locked to Bunch Crossing Data is transmitted along with A/D conversion

No Event buffers in Digitizing Modules No EMI when Event search in progress

Low EMI allow to have Preamps and A/D converter in single-board solution

Minimize Power consumption to increase channel density Commercially available components only CAMAC compatible if possible

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMD-3 Data Acquisition Electronics Layout

DAQ Synchro

Digitizer TQ

Digitizer SAD

Digitizer T2Q

VEPP2000

SND

ClbrPulser

ClbrPulser

ClbrPulser

CMD-3

Extended Decision

Claster Finder

TOF&Mu Frontend

Calorimeter Frontend

Particle Injection

Tracker Frontend

Track Finder

Bunch Crossing

RF Cavity Freq.Storage Ring Command,

Clock

to Event Builder

Data Delivery

Groupe 1

Groupe 2

Data Delivery

Switch Trigger Data

Pipeline Synchronization

Event Queue and Time Control

Data Collection Status Control

and Check Calibration

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Data Flow

DAQ Synchro

Digitizer

Command, Clock

Data Delivery

“DAQ Synchro” generates a message with StartBite, Command Code and Event Number

“Data Delivery” distribute it simultaneously to all “Digitizer” modules and Level-1 Trigger modules

All “Digitizers” return Data of current Event

“Data Delivery” collects the Data, and then transmit it to Event Builder

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMD-3 Link intro

Digitizer

DAQ Synchro

Data Delivery

CMD-3 DAQ Link

Down Link InterfaceUp Link Interface

All messages are transported by specially designed media “CMD-3 DAQ Link”

DAQ System Bus includes

~400 Links This Bus is a serial

backplane with Point-to-Point connection and root hierarchy

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMD-3 Link Stack

Event

Numbe

r

DAQ Synchro sends Message Dead

TimeAnalog signals setup time, up to 10us

Data

Wor

d1Event

Numbe

r

Comman

d

Data

Wor

d8Com

mand

DeadTime

A/D conversion in progress, up to 40us

Data

Wor

d62

Digitizer sends Data

Data

Wor

d1CRC

Bidirectional Data Line, No preamble due to dedicated Clock Line Coincidence of StartBit Level and Clock Line Transition is Common Stop Messages are transmitted to “Data Delivery” Modules through Links Answer from Digitizer contains Event Number, Board ID, Status Word Command Code is covered with parity, Data is covered with CRC

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMD-3 Link Phy

Type Synchronous

Modulation Level

Clock speed F bunch_Crossing*2 = 25 МHz

Data speed 25 Мbps

Electrical Levels LVDS, 4mA same as IEEE-1394

Media Double UTP Cat5130Ohm, double side

termination

Connector Type USB, type А, both ends

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Event’s Queue and Time Control

Mixed_StartSim_Normal_Start

Claster_StartTracker_Start

Start new EventEvent Manager

Event Number

nke16

Start Transaction at Link's Groupe1

Down-Link Interface1

Link16

Event Controller

Enable next Event Transaction Complete

Start Transaction at Link's Groupe2

Off-System Synchro 1

Off-System Synchro 2

Event Code Transaction Code

Time Set Code

Off-System Blocking

Sim_Random_Start

Read_StatusInjection_BeginInjection EndCalibration_StartCheck_Read

Off-System_Start Sta

rt M

e asu

r eS

tatu

s C

ont r

ol

Request Layer Event Layer Transaction Layer Signaling Layer

Link1

Link2

“Event Manager” accepts Requests, and Builds Event’s Queue “Event Controller” serves Queue one-by-one and convert it to Stream of DAQ

Messages Trigger’s Event are serviced immediately or skipped other are settled in queue Messages are transmitted to “Data Delivery” Modules through Links

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Data Collection

SRAM 8MB

Link from DAQ Synchro

Link to Digitizer 30

Link to Digitizer 1

Down_Link Interface 30

Down_Link Interface 1

Up_Link Interface

Eth100 Controller

Phy

Eth100 to Switch

SRAM Controller

Command and Clock FanOut

“Data Delivery” distributes the Message to Digitizers “Data Delivery” receives Data and accumulates them in RAM Ethernet 100 Controller transmits Data from RAM through

Ethernet Switch to Event Builder Ethernet Switch solves a collision problem

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Inside Digitizer

When Message reaches the Digitizer associated Command List is activated and executed

Any Digitizer resource can be accessed for read or write as preprogrammed in Message specific Command List.

“Redirector” feature includes Command List’s RAM and glue Logic.

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Trigger Data Pipeline Synchronization

Each Triggering Module has “Cavity Frequency Restorer”, based on FPGA’s PLL and binary counter. Relative phase of counter is settled by leading edge of Common Stop. This insures that all “Cavity Frequency Restorers” in DAQ has same phase. Introducing Link-to-Link delay we can control and align pipeline latency of different Triggering Module.

Calibration

When Calibration Event generated, Links to Calibrating Modules are started immediately while some delay introduced for all “Normal” Links. Pushing this Link-to-Link delay we can control Calibrating process.

The CMD-3 Data Acquisition and Control System.

March 5, 2008

CMD-3 “Standard Design” for board’s compatibility.

To obtain compatibility of different digitizers in DAQ single style is required.

Specific “Standard Design” was developed.

It includes all function required for digitizer to work in DAQ.

It supports full independent access from any interface.

Supports modular approach for Board specific Controllers

Code is written in Altera HDL

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Module for synChronization of System MChS

Logic functions which are implemented and tested with Prototype Module.

Hardware features to be added for specified jitter performance.

“Standard Design” Automatic Status

Control.

Module for synChronization of System MChS

DAQ Interface Up-Link

DA

Q C

MD

-3

Internal BusC

AM

AC

CAMAC Interface

DAQ Interface Down-Link 1

ce16

RF Cavity Freq. Conditioning

Clock Fanout

Phase Splitting and Aligning

Low

Jitt

e r

Re s

y nch

ron i

zatio

n B

u ffe

r

Bunch Crossing Freq. Conditioning

Triggering Signal Scalers

Off-System Signal Scalers and TDC

Utility Generators

Status Signal Conditioning

Que

ue C

ont r

ol

and

Tim

i ng

Log

ic

Veto Mu

Veto TOF

Status LEDs

DAQ Memory Residue Check

Link 1

Link 16

“DAQ Synchro”: Prototype module with 6 Down_Links are under tests since

April 07 Full size, full feature Module will come soon.

“Data Delivery”: Two serial modules are under tests since November 07

“Digitizers”: 5 types of Digitizers and TrackFinder are under tests now

Test modules Down_Link and Up_Link are in use since September 06

Error Rate in 1 Link at testbench – less than 10-8 Error Rate in full System – less than 10-4 All Speed requirements are achieved

CMD-3 DAQ Status

The CMD-3 Data Acquisition and Control System.

March 5, 2008

Thank You for attention

The CMD-3 Data Acquisition and Control System.

March 5, 2008

End