Trigger and Data Acquisition at the Large Hadron Collider
Trigger and Data Acquisition at the Large Hadron Collider
17-June-2006 A. Cardini / INFN Cagliari 2
Acknowledgments• This overview talk would not exist without the help of many
colleagues and all the material available online
• I wish to thank the colleagues from ATLAS, CMS, LHCb and ALICE, in particular R. Ferrari, P. Sphicas, C. Schwick, E.Pasqualucci, A. Nisati, F. Pastore, S. Marcellini, S. Cadeddu , M. Zanetti, A. Di Mattia and many others for their excellent reports and presentations
17-June-2006 A. Cardini / INFN Cagliari 3
Day 1 - Summary• LHC
– Accelerator parameters– The experiments
• Triggering– General Concepts– LHC Requirements– Trigger architecture
• Implementation of First Level Trigger– ATLAS and CMS– LHCb– ALICE
• The First Level Trigger Technology
LHC
17-June-2006 A. Cardini / INFN Cagliari 5
LHC Accelerator Complex• Proton-proton
– CM energy = 14 TeV– L = 1034 cm-2s-1 ATLAS, CMS– L = 1032 cm-2s-1 LHCb
• Heavy Ions (ex.: Lead-lead)– CM Energy = 1312 TeV (!)– L = 1029 cm-2s-1 for ALICE
17-June-2006 A. Cardini / INFN Cagliari 6
Why LHC?• We need a high luminosity and a high center-of-mass
energy proton-proton collider to
– Search of Higgs boson(s)– Search for SUSY particles– Standard Model Physics– CP violation studies in the B sector– New Physics beyond SM– Ultra High Energy Heavy Ions Collisions
… hope that we will see many other things we do not expect…
17-June-2006 A. Cardini / INFN Cagliari 7
LHC vs. LEP vs. Tevatron• One way to increase the luminosity is to increase
the number of bunches circulating in the ring: LHC will have ~3600 bunches– 27 km (LEP tunnel) ring– 27000 m / 3600 = 7.5 m between bunches– 7.5 m / 3x108 m/s = 25 ns
17-June-2006 A. Cardini / INFN Cagliari 8
Proton-proton cross section• Interactions/second
– L = 1034 cm-2s-1 = 1010 Hz/b– σinel(pp) ~ 70 mb
7x108 interactions/s
• Events/crossing– @ 40 MHz (∆t = 25 ns)
17.5 interactions/crossing
• Not all bunches are full– Only about 4/5 (2835/3564)
22 interactions/”real” crossings
• What are we looking for?– 1 interesting physics event (Higgs for example) superimposed to
~20 minimum bias events !!!
σinel(pp) ~ 70 mb
17-June-2006 A. Cardini / INFN Cagliari 9
Here it is!• 20 minimum bias events overlapping + H ZZ 4 muons (the cleanest
golden signature)
• This “mess” (not the Higgs! - its production cross section in very small so we expect 0.01-0.1 Hz Higgs production) repeats every 25 ns!
17-June-2006 A. Cardini / INFN Cagliari 10
The Physics at LHC• Cross sections for various physics
processes vary of many orders of magnitude
• At the standard LHC luminosity we have:
– Inelastic (min. bias): 109 Hz– W lν: 102 Hz– ttbar: 10 Hz– Light Higgs (100 GeV): 0.1 Hz– Heavy Higgs (600 GeV): 0.01 Hz– bbbar: huge (106 Hz)
• An efficient selection mechanism capable of selecting 1 event over 1010-1011 is needed: this is the
TRIGGER
17-June-2006 A. Cardini / INFN Cagliari 11
How to build a LHC Experiment?• Depends obviously on the physics…
• New (very) heavy particles (Higgs, for example) are produced centrally with large transverse momentum symmetric detector– ATLAS, CMS
• Lighter particles (B, for example), are produced mainly at small angles. One can take advantage of the boost forward detector for B physics– LHCb
• When running in heavy ions mode ALICE will search for Quark-Gluon Plasma, needing both central and forward coverage
17-June-2006 A. Cardini / INFN Cagliari 12
How to build a LHC Experiment? (2)• Each experiment is made of
– “Inner” trackers– Calorimeters– Muon detectors
• This will allow to– Resolve the tracks – Measure the energy depositions– Identify the particles– Measure the decay vertices
• Experiment size and granularity is determined by– Required accuracy– Particle multiplicity @ ATLAS/CMS O(1000) particles/B.C.
• This determines– Number of detector “elements”– Number of electronic channels– Data size and throughput
17-June-2006 A. Cardini / INFN Cagliari 13
ATLAS
44 m length22 m diameter
17-June-2006 A. Cardini / INFN Cagliari 14
CMS
17-June-2006 A. Cardini / INFN Cagliari 15
LHCb
17-June-2006 A. Cardini / INFN Cagliari 16
ALICE
TPCTPC
PHOSPHOS
Muon armMuon arm
TOFTOF
TRDTRDHMPIDHMPID
PMDPMD
ITSITS
ACCORDEACCORDE
Triggering
17-June-2006 A. Cardini / INFN Cagliari 18
General Trigger Requirements• The role of the trigger is to make the online selection of particle
collisions potentially containing interesting physics
• Need high efficiency for selecting processes of interest for physics analysis, for which:– Efficiency should be precisely known– Selection should not have biases that affect physics results
• Need large reduction of rate from unwanted high-rate processes (capabilities of DAQ and also offline computers):– Instrumental background– High-rate physics processes that are not relevant for analysis (min.
bias)
• System must be affordable– Limits complexity of algorithms that can be used
• Not easy to achieve all the above simultaneously!
17-June-2006 A. Cardini / INFN Cagliari 19
LHC Trigger Challenges• Nchannels ~ O(107-108) and 20 interactions/25 ns
– Need huge number of connections– Need information super-highway
• Information coming from different detector parts should correspond to the same interactions– Need to synchronize detectors to (better than) 25 ns
• Note however that in some cases detector signals and/or time-of-flight exceeds 25 ns– Some detector will integrate information coming from more than 1 bunch
crossing
• Can store data at 100 MB/s 100 Hz for ATLAS/CMS (1 MB/ev.), 1 kHz for LHCb (100 kB/ev.)– Need to reject most interactions
• What is discarded is lost forever!– Need to careful monitor the selection
17-June-2006 A. Cardini / INFN Cagliari 20
Triggering Howto• Look at (almost) all bunch crossings, select most interesting
one, collect all detector information and store it for off-line analysis (for a reasonable amount of money)
• Since the detector data are not all promptly available and the selection function is rather complex, T() is evaluated by SUCCESSIVE APPROXIMATIONS called TRIGGER LEVELS(which should have possibly zero dead time)
17-June-2006 A. Cardini / INFN Cagliari 21
The multi-level triggerA multi-level trigger system provides:
– Rapid rejection of high-rate background without incurring in (much) dead-time: the fast first-level trigger (custom electronics)
• Needs high efficiency, but rejection power can be comparatively modest
• Short latency is essential since information from all (up to O(108)) detector channels needs to be buffered (often on detector) pending trigger decision
– High overall rejection power to reduce output to mass storage to affordable rate: one or more High Trigger Levels:
• Progressive reduction in rate after each stage of selection allows use of more and more complex algorithms at affordable cost
• Final stages of selection, running on computer farms, can use comparatively very complex (and hence slow) algorithms to achieve the required
Example: ATLASoverall rejection power
17-June-2006 A. Cardini / INFN Cagliari 22
First Level Selection
• First level (level 1) reduces event rate from 40 MHz to O(100) kHz
• This step exist in all experiments
• Not enough, still to go down by factor 100-1000 in one or more extra step…
17-June-2006 A. Cardini / INFN Cagliari 23
Successive Selection: 3 steps…
Additional processing in intermediate step reduces bandwidth requirements
17-June-2006 A. Cardini / INFN Cagliari 24
… or 2 steps
!
This solutions reduces the number on building blocks and could rely on commercial components for what concerns calculations and network
17-June-2006 A. Cardini / INFN Cagliari 25
Triggering @ LEP vs. LHC
100 ms
8 ms
30 µs
e+e– crossing rate 45 kHz (4 bunches)
22µs45 kHz
Level 2
10 Hz Readout
6 µs
Level 1
Level3
100 Hz
8 Hz
≈ µs
p p crossing rate 40 MHz (L=1033- 1034cm-2 s-1)
25 ns
Level 1
40 MHz
≈ ms
Level 2
100 kHz
Level n
1 kHz
• LEP– tL1 < inter bunch time– no event overlapping
• LHC– tL1 » inter bunch time– 22 overlapping events/BC
17-June-2006 A. Cardini / INFN Cagliari 26
Trigger/DAQ at LHC
!
!
!
!
100 (~103)
17-June-2006 A. Cardini / INFN Cagliari 27
Trigger/DAQ: past, present and future
Level-1 (L1) Trigger
17-June-2006 A. Cardini / INFN Cagliari 29
Algorithms for Level-1 Trigger• The Physics
– pp collision produce hadrons with pt ~ 1 GeV– Interesting Physics has particles with large pt:
• W lν: M(W) ~ 80 GeV/c2, pt(l) ~ 30÷40 GeV• H(120 GeV/c2) 2 photons, with pt(photon)~ 50÷60 GeV
• Trigger Requirements– Impose high thresholds on specific interaction products: “easy”
for muons, electrons and “jets”, then need complex algorithms– Typically:
• Single muon with pt > 20 GeV 10 kHz• dimuons with pt > 6 GeV 1 kHz• Single electron with pt > 30 GeV 10÷20 kHz• dielectrons with pt > 20 GeV 5 kHz• Single jet with pt > 300 GeV 200÷400 Hz
ATLAS/CMS requirements
17-June-2006 A. Cardini / INFN Cagliari 30
Pt cut in minimum bias events
All tracks pt > 2 GeV
Simulated H→ ZZ → 4 µ event + 17 minimum-bias events
17-June-2006 A. Cardini / INFN Cagliari 31
Which Detectors at Level-1?
17-June-2006 A. Cardini / INFN Cagliari 32
Which Detectors at Level-1? (2)
☺ In Muon detector / calorimeter low occupancy and patter recognition is easy– Simple reconstruction algorithms fast– Small amount of data– Can take “regional” decisions
In inner detectors– Complicated events!– Complex analysis algorithms slow– Huge amount of data– Need to link to other detector for
additional information
17-June-2006 A. Cardini / INFN Cagliari 33
Still is not easy…• It is not possible to generate a trigger in 25 ns
• Need a massive concurrent, pipelined processing to implement a dead-timeless L1 trigger
Primitive Gen
detector ≈ 50 ns
FE≈ 100 ns
≈ 100 ns
≈ 500 ns
≈ 500 ns
Local trigger
≈ 600 ns
Global trigger
≈ 300 ns≈ 600 ns
pipelinedelay
O(100)deep
µs
derandomizer
Example: CMS
~3
17-June-2006 A. Cardini / INFN Cagliari 34
Level-1 Processing
17-June-2006 A. Cardini / INFN Cagliari 35
Information Flow
17-June-2006 A. Cardini / INFN Cagliari 36
Information Flow (2)
L1 trigger architecturein the LHC Experiments
17-June-2006 A. Cardini / INFN Cagliari 38
Region of Interest (RoI)• The L1 selection can based on local
signatures called Region of Interest (RoI)– Based on coarse granularity, no inner
tracker info– Local analysis allows an important further
rejection
• The geographical location of interesting signatures are identified by L1– This allow access only to local data for
each relevant detector
17-June-2006 A. Cardini / INFN Cagliari 39
Region of Interest (2)• Region-of-Interest
– RoI data ~1% of L1 output– Complex mechanism for
data access– Many control messages– Smaller readout network
thanks to an intermediate trigger level which only processes local (in η,φspace) information
MORE COMPLEX SYSTEM
• Not RoI
– Very high throughput– Very large readout network– Simpler system– Flexible
MORE DEMANDING ON TECHNOLOGY
17-June-2006 A. Cardini / INFN Cagliari 40
The ATLAS ArchitectureRoI based L1 trigger
17-June-2006 A. Cardini / INFN Cagliari 41
ATLAS• Muon
– O(106) RPC (Barrel)/TGC (Endcap) trigger channels– Barrel and Endcap LOCAL Trigger Processor to estimate pt– Muon Central Trigger Processor
• Calorimeter – LAr (ECAL) and Tile/LAr (HCAL)– Analog preprocessor (analog pipeline!) to estimate Et– LOCAL Jet/Energy Sum and Cluster Processor stage
• The local triggers are sent to the Central Trigger Processor (CTP), which makes the FINAL decision
• The L1 trigger decision is sent to the Front End via the TTC (Timing-Trigger-Control) system
• For every accepted event the L1 trigger sends readout information to the Region-of-Interest (RoI) Builder which assembles the list of RoIs to be used by L2 Note: all digital design except input stage of calorimeter trigger pre-processor
17-June-2006 A. Cardini / INFN Cagliari 42
The ATLAS Muon Trigger• The L1 Muon Trigger
requires coincidence of RPC/TGC hits within a road, which is related to the pt cut applied
• A high and a low ptalgorithms are applied
• Multiple cuts can be used at the same time thanks to programmable coincidences
Fast and high redundancy system
- Wide pt threshold range- Safe bunch crossing identification (fast detectors)- Strong rejection of fake muons (noise, physics background)
40 kHz expected at L=1034 /cm2/s
17-June-2006 A. Cardini / INFN Cagliari 43
The CMS ArchitectureSimilar to ATLAS, but no RoI
The Global Calorimeter trigger selects the best 4 e,γ, τ and jetsand calculate Et and Et
miss
The Global Muon Trigger receives 4 muon candidates of maximum ptand select the best quality ones
The Global Trigger applies the thresholds and performs the trigger algorithms.
Up to 128 algorithms can run in parallel: arbitrary combinations of trigger objects passing thresholds and topological correlations…
17-June-2006 A. Cardini / INFN Cagliari 44
The CMS Calorimeter Trigger
• Divide Calorimeter in towers• Match towers between ECAL and HCAL• Isolation and deposit shape criteria to
identify electrons, photons, jets
17-June-2006 A. Cardini / INFN Cagliari 45
L1 Trigger Rates
1.63.3--Double µ
8.71023.2/3.86 (l) / 20 (h)Single µ
--1.0336Total ET
0.012751.051Etmiss
1.21406.0100Jet
3.28516.320τ
2.7121.412Double e/γ
5.7221117Single e/γ
Rate (kHz)Thr. (GeV)Rate (kHz)Thr. (GeV)Selection
CMSATLAS
L1 trigger rates at L = 1033 when applying 90% efficiency thresholds
17-June-2006 A. Cardini / INFN Cagliari 46
CMS vs. ATLAS
17-June-2006 A. Cardini / INFN Cagliari 47
The LHCb Architecture
Calorimeters + Muon system
10 MHz
1 MHz
L0: hight pT tracks+ not too busy eventsFully synchr. (40 MHz), 4 µs latencyOn custom boards
• 40 MHz crossing rate, but only 30 MHz real crossings
• Luminosity: 2·1032 cm-2 s-1
(50 times lower than ATLAS and CMS)
• Minimum bias rate: 10 MHz
• bb rate is ~ 100 kHz (15 kHz in detector acceptance)
• cc rate is ~ 600 kHz
• First level trigger (here called L0) selects high pt particles (muon, e,gamma, …)and events with only one interaction by means of the pile-up veto
17-June-2006 A. Cardini / INFN Cagliari 48
LHCb Muon Trigger• The LHCb muon system:
– 5 stations– Variable segmentation– Projective geometry
• Trigger strategy:– Straight line search in M2-M5 in
every quadrant– Look for compatible hits in M1
• Momentum measurement (∆p/p~20% for b-decays)
• Sent to L0 decision unit: 2 highest pTcandidates per quadrant
• Typical Performance:~88% efficiency on B->J/ψ(µµ)XAlgorithm latency ~1 µs
µ
>90% π/K decay
Nominal threshold
17-June-2006 A. Cardini / INFN Cagliari 49
LHCb Pile-up veto• LHCb needs to identify secondary (decay)
vertices
– This is performed @ a higher trigger level
– Works well if there is only and only one interactions per bunch crossing
– A veto against double interactions is implemented with 2 silicon detectors planes
– Hits are fitted by means of 4 large FPGAs and results are sent to L0 Decision Unit
– Typical Performance for identifying double interactions
• 60% efficiency• 95% purity
– Latency ~ 1 µs
17-June-2006 A. Cardini / INFN Cagliari 50
The ALICE Architecture• Heavy ions runs
– L=1027cm-2s-1
– Interaction rate < 10 kHz– Very high multiplicity & Huge event size (~ 50 MB)– Modest requirements on lower level triggers
• pp (or pA) runs – Interaction rate up to 200 kHz, limited by TPC pileup– Small event size (~ 2 MB)– Strong requirements on lower level triggers
• To accommodate all the different running conditions the first level trigger is split in 3 distinct levels (L1, L1 and L2)
17-June-2006 A. Cardini / INFN Cagliari 51
ALICE First Level Trigger(s)• Some of the Alice FEE is not pipelined
but await a trigger before processing
• Some detector need very early strobe(e.g. TOF), so a first early decision is taken in 1.2 µs (L0)
• For detectors which require longer timeL1 is used, which arrived 6.5 µs after interaction
• L2 comes after 88 µs, at the end of the drift time in TPC. Purpose of L2 is to wait for the end of the “past-future pile-up protection”, in order to make sure that there is only one event in TPC
17-June-2006 A. Cardini / INFN Cagliari 52
ALICE: Optimizing Trigger Efficiency• Requirements
– Some subdetectors need a long time to be read out after a L2 (silicon drift detector: 260 µs)
– However some interesting physics events need only a subset of detectors to be readout
• Concept of “Trigger Clusters”– Group of sub-detectors– Even if some sub-detectors are busy, triggers for not busy
clusters can be accepted, increasing acquired statistics
• “Rare Events”– When readout buffer “almost full” only accepts the so called
“rare” event
L1 Trigger Implementation
17-June-2006 A. Cardini / INFN Cagliari 54
L1 implementation issues• At L1 every operation must be extremely fast
• L1 logic is usually built using:• ASIC (Application Specific
Integrated Circuits)– Can be produced radiation
tolerant (to be installed on detectors)
– Can contain both analog and digital part
– Full-custom or Standard-Libraries (or mixed) design
– Long development cycle– Extensive simulation
necessary– Production cycle
expensive, but cost/ASIC can be extremely low
• FPGA (Field Programmable Gate Arrays), – Extremely versatile
nowadays– Might contain memory,
processors, high speed serial links…
– Complex design possible: processors, PCI interfaces, WEB-servers
– “Easy” to implement the requested design
– Reprogrammable (even in-situ!)
– (very) Expensive units
17-June-2006 A. Cardini / INFN Cagliari 55
LHCb Muon Off Detector Electronics
• SYNC ASIC
– CMOS IBM 0.25 um– 8x4 bit TDCs– 12 bit counter for Bxid
generation– L0 buffer and derandomizer– Interface to L0 trigger logic– Programmable via i2c– Radiation resistant– Triple-voted auto-correcting
registers for radiation immunity
– … many other things
17-June-2006 A. Cardini / INFN Cagliari 56
L1 implementation issues (2)
• Communication Technologies
– Very high-speed serial links (copper or fiber)• LVDS, G-link, Vitesse, …
– Backplanes• Very large number of connection, data multiplexing
17-June-2006 A. Cardini / INFN Cagliari 57
CMS Regional Calorimeter Trigger• Receiver card (there are
O(20) cards/crate and O(20) crates in the system): ~ 400
• Receives 64 trigger primitives, 32 from ECAL and 32 from HCAL
• Forms two 4x4 towers for Jet Trigger and 16 Et towers for electron isolation card
• Overall system input bandwidth: ~4000 Gb/s
17-June-2006 A. Cardini / INFN Cagliari 58
Today’s Conclusions• LHC: a very challenging environment
• Very demanding requests on trigger system, in particular on first level trigger
• Pipelines (analog or digital) everywhere for a (almost) dead timeless L1 trigger
• Different philosophies of the experiment: ATLAS vs. CMS want to study the same physics but adopt different approaches: RoI vs. not RoI
• Technology is progressing very rapidly and its performance appears adequate (but still systems are very complicated…)