1 24-10- 2008 Bart Hommels for the CALICE-UK groups Data acquisition systems for future calorimetry at the International Linear Collider Introduction DAQ design at system level DAQ for ILC Calorimetry: CALICE
Mar 28, 2015
124-10-2008 Bart Hommels for the CALICE-UK groups
Data acquisition systems for future calorimetry
at the International Linear Collider
Introduction
DAQ design at system level
DAQ for ILC Calorimetry: CALICE
224-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
Introduction
324-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
IntroductionTraditionally, DAQ in HEP is at the back of the queue for R&D effort,
often leading to a conservative system implementation.
To make optimal use of the advancements in industry, a new DAQ should be considered as a ‘generic’ system, with a ‘use case’ for a particular (sub-)detector.
Detector Readout System
D
A
QIn most cases, Detectors and associated Readout Systems are designed, tested and approved first, before substantial DAQ effort is undertaken.
424-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
Introduction: generic DAQ design
This DAQ is designed as a generic, scalable, and self-contained system, build around commercial off-the-shelf components where possible
The generic DAQ is then configured towards multiple ‘use-cases’. ILC calorimetry might not be the only customer
Using the generic, core DAQ for other interesting cases is welcomed for stress-testing the system: think of linear collider experiments or SuperLHC
524-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
DAQ design
624-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
DAQ design: global architecture
Detector modules called ‘slabs’ deliver data over ~Gbit speed optical fibre
• Layer-1 switch routes event data to PC
• Target Control re-routes data should a certain PC not be available
• After processing, data is routed through a network switch towards event builder/data store
724-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
DAQ design: candidate Level-1 switch
Candidate Level-1 switch: state-of-the art full optical switch system
Currently investigating performance: switch timing characteristics etc.
M. WarrenV. Bartsch
824-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
DAQ design: PC-based receiver card
PC-based receiver card is a key component in the generic DAQ design
• Developed & built by PLD Applications
• Large & Fast FPGA: Xilinx Virtex4-FX100
• 8x PCIexpress bus• SFP cages for multi-Gbit/s
interfaces• Many high-speed I/O
connections
To be used for the following tasks:• Data reception• Clock source• Control data transmission• Configuration data distribution
924-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
A DAQ for ILC calorimetry
1024-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: Architecture
Link Data Aggregator(LDA)
Detector Interface(DIF)
Detector Unit
Off Detector Receiver(ODR)
DAQ software
1124-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: system overview
Detector Unit: long (>1.5m) detector ‘slab’ with integrated front-end electronics
DIF: Detector InterFace, servicing the Detector UnigLDA: Link-Data Aggregator. Concentrates data, fanin/fanout for clock
and control dataCCC: Clock&Control Card: Fanout of clock and fast controlsODR: Off-Detector Receiver: PC interface for the DAQ systemPC running DOOCS for run-control and DAQ monitoring
DAQ PC
LDA
LDA
ODR
DetectorUnit
DIF
CCC
DetectorUnit
DIF
DetectorUnit
DIF
DetectorUnit
DIF
StorageControl
PC (DOOCS)
DAQ PC
ODR
1224-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: links & protocols
1. DIF-LDA: 80-160 MHz serial link. 8b/10b encoded synchronous data transfers
2. LDA-ODR: Gbit ethernet (or possibly TLK2501) over optical fibre
3. CCC-LDA: trimmed version of DIF-LDA link
LDA
LDA
DetectorUnit
DIF
CCC
DetectorUnit
DIF
DetectorUnit
DIF
DetectorUnit
DIF
Control PC
(DOOCS)
12
3
1324-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: DIF
ECAL DIF prototype:65x72mm, 8 layers1. JTAG programming header2. LDA link HMDI connector3. DIF link connector4. mini-USB connector5. Xilinx PROM6. Cypress 2MB 10ns SRAM7. Xilinx Spartan3-1000 FPGA8. FDTI FT245R USB controller9. 20p user header connector10. reset pushbutton11. 90pin SAMTEC IB connector
• 2 DIFs produced, parts available for 10 more.• DIF hardware is (at least partly) functional
The DIF concept is generic in firmware, running on detector-specific hardware.
Different DIF flavours for ECAL, HCAL
ECAL DIF status:• Test hardware in place• Firmware development started
M. Goodrick, B. Hommels, R. Shaw
1424-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: DIF -LDA linkLink shows signs of life:
pseudo-LDA sends CLK & 8B/10B data @ 100MHz over AC-coupled LVDS on HDMI cables
DIF
pseudo-LDA
HDMI cable
AC-couplingadapter
stable data loop-back in firmware
HDMI connector + cabling standard adapted for excellent price/performance
1524-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: LDA
• Aggregate data from many DIF links and send to ODR over Gbit Ethernet link
• LDA should serve 10 DIFs
• Using commercial board from Enterpoint, with semi-custom add-on boards
• However, board came back with problems acknowledged by manufacturer - re-spin now
• Major firmware development underway
• ODR-LDA protocol progressing
• LDA-DIF link being documented
M Kelly
1624-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: CCC
Tasks:• Synchronise all subsystems
upon pre-spill warning• Act as reliable global clock
source for fast clock, and phase-aligned ILC machine clock
• Distribute asynchronous fast trigger and/or busy signals
• Capable to run stand-alone in tandem with DIF
CPLD-based for implementation of complex logic.Routing to/fro many sources and destinations possible
M Warren, M Postranecki
1724-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: ODR
ODR is the use-case for the PC-based DAQ interface card:
• Commercial board with custom firmware and software
• Task-specific performance optimisation
• System-specific extensions: decode LDA event data headers, etc.
170MB/s to disk
B. Green, A. Misejuk
1824-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: Software
Software to follow system architecture:
• Modularity• Generic nature with
specific interfaces• Scalability
DAQ software requirements in ILC calorimetry:• Support for several calorimeter types, each of significant complexity: O(104) channels, O(103) ASICs, O(102) electronics boards• Preferably use existing DAQ software (framework)• Chose DOOCS being developed at DESY for XFEL
tesla.desy.de/doocs/doocs.html
V. Bartsch, T. Wu
1924-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimeter DAQ: Software application• Starting from
DAQ-ODR interface: first hardware layer & most advanced
• Have set-up LDA emulator and passed data in this system:
PC->ODR->LDA
2024-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
ILC Calorimetry DAQ: CALICE
CALICE: physics prototype of ILC calorimeter detector segment: ECAL + HCAL
First demonstrator for newly developed DAQ:
CALICE testbeam in 2009
DAQ components: Software, ODR, CCC, LDA, DIF prototyped and on track to be ready in time -before everyone else!
2124-10-2008 Bart Hommels for the CALICE-UK groups: Cambridge U., Manchester U., RHUL, UCL
DAQ for future calorimetry at ILC: summary
Aim is to develop a DAQ system that is generic in nature, using commercial components where possible
The DAQ system should be modular and scalable
ILC Calorimeter sub-detectors demonstrators serve as the first well-described ‘use-case’ for the DAQ
All DAQ components prototyped in hardware, ready to merge into a full fledged DAQ system through firmware and software development.
First serious test opportunity: testbeam in 2009
Exciting times ahead!