05.10.2006 CNGS Interlock System Training / J. Wen ninger 1 Introduction to Machine protection for CNGS (and TI2/8) • Introduction to the new Beam Interlock Systems • CNGS extraction and protection requirements • Interlocks : what, why, how? • Operational aspects & issues J. Wenninger AB-OP
63
Embed
Introduction to Machine protection for CNGS (and TI2/8)
Introduction to Machine protection for CNGS (and TI2/8). J. Wenninger AB-OP. Introduction to the new Beam Interlock Systems CNGS extraction and protection requirements Interlocks : what, why, how? Operational aspects & issues. Introduction. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
05.10.2006 CNGS Interlock System Training / J. Wenninger 1
Introduction to Machine protection for CNGS (and TI2/8)
• Introduction to the new Beam Interlock Systems
• CNGS extraction and protection requirements
• Interlocks : what, why, how?
• Operational aspects & issues
J. Wenninger AB-OP
05.10.2006 CNGS Interlock System Training / J. Wenninger 2
Introduction
The aim of this presentation is to give an overview over :
• The new Beam Interlock Systems for SPS (and later LHC). In 2007 they will have replaced the present SPS Emergency System.
• Details on the interlocking for the CNGS fast extraction.
• Operational experience with the CNGS line and interlock system during the short CNGS run 18th -30th August 2006.
It is not possible to go into all details (time !). But the entire system is described in detail on the WEB page
05.10.2006 CNGS Interlock System Training / J. Wenninger 3
Beam Interlock Systems for SPS/LHC
Requirements:• Generic solution for CERN• Reliable• Available• Fast
05.10.2006 CNGS Interlock System Training / J. Wenninger 4
User Inputs
• The user inputs are partitioned into NOT MASKABLE and MASKABLE inputs.– Note : MASKED signal = DISABLED signal.
• NOT MASKABLE inputs are always active.
• The MASKABLE inputs can be masked provided the SAFE BEAM FLAG is TRUE.
• The SAFE BEAM FLAG is :– TRUE if BEAM_INTENSITY * BEAM_ENERGY is considered SAFE – i.e. no / little risk of damage. Needs input from BCT(s) and
potentially (not at the SPS) main converter currents.
– Presently distributed by the timing system (as a special telegram).
Beam
Interlock
System
Kicker SystemBEAM_PERMIT
MA
SK
AB
LE
NO
TM
AS
KA
BL
E
User System #1
User System #2
User System #n
User System #m
USER_PERMITS
SAFE_BEAM_FLAG
SBF= distributed information to provide some flexibility while maintaining safety.
05.10.2006 CNGS Interlock System Training / J. Wenninger 5
Interlock Signal ConnectionsBIS Rack
User System
Permit ‘A’
Permit ‘B’
‘A’
‘B’
User System Rack
User InterfaceBeam Interlock
Controller
■ On copper cable up to 1.2km
■ Standard interface for all interlock ‘clients’ at the SPS & LHC
■ Receives the USER_PERMIT signal (redundant) and transmits it to the nearest BIC
■ Allows direct connection with many different types of hardware platform (PLC, VME, etc…)
05.10.2006 CNGS Interlock System Training / J. Wenninger 6
BIS User Interface (CIBU)
• Standard interface for the user signals.
• Accepts various standards (VME/TTL/PLC) as input.
• Installed in the ‘client’ rack.
• Remote tests and diagnostics.
• Redundant power supply.
05.10.2006 CNGS Interlock System Training / J. Wenninger 7
BIS VME Crate
• Cables are connected with a special patch panel at the back of the BIS VME rack.
• NO cabling inside the rack: signal routing proceeds over printed circuit boards.
• Fixed and well defined topology : cable VME module
One of the TT40 racks
05.10.2006 CNGS Interlock System Training / J. Wenninger 8
BIC Module
The Beam Interlock Controller or BIC is the VME module that collects the User-permit (interlock) signals. It is the heart of the interlock system!
Each BIC has 14+1 inputs :
7 non-maskable HW inputs. Always active !!
7 maskable HW inputs, that can be masked provided the ‘Safe Beam Flag is TRUE. At the SPS the ‘Safe Beam Flag’ is generated 1 second after the last injection from an intensity measurement of the hadron BCT in LSS3. It is distributed by the timing system to the BICs. Presently the limit is set to 1012.
1 software input to be set through the standard controls network by SIS.
A standard BIC provides an output signal corresponding to the logical AND of all inputs.
IMPORTANT : unlike the SPS Emergency Dump System, the SPS BICs do not LATCH any signal !
BIC modules can be connected to each other:
Interlock loop (fiber optic link), used for the rings.
BIC output BIC User-permit input (‘daisy-chain’), used for transfer lines.
BIC internal history buffer:
All transitions are logged and time-stamped (to the s).
of the BIS
05.10.2006 CNGS Interlock System Training / J. Wenninger 9
Schematic BIS Hardware LayoutPatch panel
1 or 2 BICmodules
CPU + CTRPModules
VME crate with LynxOS
Control Network
Application
UserPermits
User Interfaces
User Interface
Opticalmodule to Kicker
10
Future (2007) BIS Architecture SPS
2 Beam_Permit_LoopsClockwise & Anticlockwise
BPC
BIC
BPCBPC
BIC
BIC
BIC
BIC
BIC
BA3
2
BA2
BA1
BA6
BA5
BA4
•One BIC / BA.•Connected by an interlock
loop.•Connected to beam dump in
BA1.•Details to be sorted out.
SPS BEAM DUMPSYSTEM
User System User System User System
User System
User SystemUser SystemUser SystemUser System
User SystemUser SystemUser SystemUser System
User System User System User System
User System
User System
User System
User System
Use
r Sys
tem
User System
User System
User System
User S
yste
m
2006 status :•3 BICs for MKE & extraction
BLMs in BA4 and BA6.•Used by new Software
Interlock System (SIS).•Enters as ‘Summary BIS’
into SPS emergency dump system.
05.10.2006 CNGS Interlock System Training / J. Wenninger 11
Extraction Interlocks for CNGS/TI8
TT41 line
TI8 line
TT40 line
CIBC-TT40A
CIBC-TT40B
CIBC-TT41A
CIBC-TT41B CIBC-TI8up CIBC-TI8dwTED TED
CIBC-EXT2
Injection region
& TI8 after TED
CNGS target
to MKE 4to MKE 4
Upstream part Downstream part
InstalledInstalled for 2007 Temporary link/loopCIBC = BIC module
05.10.2006 CNGS Interlock System Training / J. Wenninger 12
CNGS Interlocking Overview
05.10.2006 CNGS Interlock System Training / J. Wenninger 13
East Extraction for CNGS
Extract 2 x 10.5 s long SPS batches, nominally 2.4e+13 protons per batch at 400 GeV
• 4 extraction bumper magnets: – 31.5 mm extraction bump
• enlarged aperture quadrupole magnets
• instrumentation
CNGS
CNGS Extraction Timing• The CNGS cycle length is 6000 ms and the 400 GeV flat top is 90 ms long (from 4200 to 4290 ms).
• The two extractions are programmed at 4220 and 4270 ms.
• The RF extraction pre-pulse is programmed to arrive in the millisecond after (nominal) extraction time.
• For the extraction kicker 2 timings/moments are crucial : 13 ms before extraction trigger PFN charging (if EXTRACTION_PERMIT = TRUE)
arrival of extraction pre-pulse trigger magnets (if PFN charged and EXTRACTION_PERMIT = TRUE)
Legacy events New events RF extr. pre-pulse (RF2)
The timing should be the same on ALL CNGS users !Please do not change it - it has consequences on interlocks, logging…
05.10.2006 CNGS Interlock System Training / J. Wenninger 15
CNGS Protection
Protection of the extraction channel and the transfer lines (TT40, TT41) – similar to the requirements for TI8 / LHC.
Protection of the T40 target (and what is behind) : At high intensity it is important to hit the target within 0.5 mm of the axis
(target rod 4 & 5 mm) to avoid damage to the rods.
A single (isolated) hit can (must !) be accepted, but we must avoid sending
two consecutive extractions (50 ms) off-axis.
T40 intercepts the fast extracted 400 GeV proton beam
with two 10.5 µs long spills spaced by 50 ms.
The nominal beam intensity is 4.8x1013 protons per
cycle, the ultimate intensity 7x1013 protons.
The nominal normalised emittance is ~ 12 mm mrad,
the beam size @ T40 is ~ 0.5 mm.
16
CNGS BICs & User Signals in 2006
CIBC.TT40A
Vacuum TT40 (1)
WIC TT40 (10)
MKE status (31)
MSE magnet (12)
TT40 PC (18)
MSE PC (19)
Bumper PC (17)
FMCM MSE (23)
CIBC.TT40B
OP Inhibit (13)
TED TT40 (4)
BTV TT40 (8)
BLM TT40 (28)
BPM LSS4 (26)
Beam Intensity (27)
SIS TT40 + LSS4
CIBC.TT41A
Vacuum TT41 (2)
WIC TT41 (11)
TT41 PC (20)
Main Bend PC (21)
Main Bend DCCT (22)
FMCM MBSG (24)
FMCM Main Bend (25)
SIS TT41 + CNGS
CIBC.TT41B
TBSE TT41 (3)
CNGS Shutter (5)
T40 Target (6)
BTV TT41 (9)
BLM TT41 (29)
BPM TT40+TT41 (30)
T40 Target (7)
Horn/Refl. Status (14)
Hadron Stop Cooling (15)
Fire Alarm (16)Red = un-maskable, Green = maskable, Magenta = Soft. Interlock
In () : the interlock index that I will referred to later when I describe the inputs.
The logical AND of all signals is sent as EXTRACTION_PERMIT to the extraction kicker MKE4 which fires only if the EXTRACTION_PERMIT is TRUE.
05.10.2006 CNGS Interlock System Training / J. Wenninger 17
Fast Extraction Interlock Types
For the CNGS/TIx fast extractions there are 3 types of interlocks based on :
• Continuous surveillance of parameters. The associated permits change their state rather ‘rarely’ .
• Vacuum, WIC, TEDs, target…
• Pre-extraction surveillance where the user permits are evaluated a short time BEFORE extraction. The associated user permit is FALSE by default and switches to TRUE for a short time interval around extraction if all conditions are correct.
• Surveillance of the beam position around extraction point and of the PC currents.
• Post-extraction surveillance where the user permits are evaluated AFTER extraction. This type of surveillance concerns beam instrumentation. The associated user permit is switched to TRUE for a short time around extraction. The user permit is latched (FALSE) at the level of the client if a measured beam parameter is out of tolerance.
• Beam losses and beam positions in the transfer lines.
• Both Pre- and Post-extraction surveillance tasks are triggered by machine timing events coupled to the main extraction event.
05.10.2006 CNGS Interlock System Training / J. Wenninger 18
Continuous Surveillance
05.10.2006 CNGS Interlock System Training / J. Wenninger 19
Targets, Absorbers & Obstacles / 1
• Vacuum valves (1, 2) :• USER_PERMIT=TRUE only when all valves are OPEN. NOT MASKABLE !
• All valves are interlocked.
• TBSE block (3) :• USER_PERMIT=TRUE ONLY when TBSE is OUT of beam. NOT MASKABLE !
• TT40 TED (4) :• USER_PERMIT=TRUE when the TED is IN beam or OUT of beam (both positions are SAFE). NOT
MASKABLE !
• In 2007 are more subtle logic (with automatic masking/unmasking of downstream interlocks) will be implemented.
• CNGS (decay tunnel) shutter (5) :• USER_PERMIT=TRUE ONLY when the SHUTTER is OPEN (position ~ 0 mm). NOT MASKABLE !
• T40 Target (6,7) :• 2 USER_INPUTs, one NOT MASKABLE, one MASKABLE.
• For normal operation USER_PERMIT=TRUE ONLY when the target barrel is positioned with a rod assembly in beam.
• We (OP) have NO CONTROL over the target position !
05.10.2006 CNGS Interlock System Training / J. Wenninger 20
Targets, Absorbers & Obstacles / 2
• TL screens (8,9) :• The TL screens have 4 positions: OUT, Carbon OTR, Titanium OTR and Alumina screen.
• USER_PERMITs are MASKABLE.
• USER_PERMIT = TRUE : OUT position + Carbon OTR
• USER_PERMIT = FALSE : Al screen, Titanium OTR or device moving (should move during beam out).
05.10.2006 CNGS Interlock System Training / J. Wenninger 21
Magnet Interlocks
• A WIC (Warm Magnet Interlock Control) system surveys the magnet temperatures of TT40, TT41,
TI8, TT60 and TI2 (10, 11) :• USER_PERMIT=TRUE only when all temperatures are OK. NOT MASKABLE !
• When a sensor detects an over-temperature or a local IO module dies (partly installed in the tunnel), the system FIRST sets USER_PERMIT=FALSE, and 1 second later switches off the concerned PCs.
• Provides 2 separate inputs for TT40 and TT41.
• It is a PLC based system, at some time we should/will get the surveillance application.
• MSE.418 magnet and girder state (12) :• USER_PERMIT=TRUE ONLY when:
Girder is IN-BEAM and within tolerance (window of +- 2 mm, reference is HARDCODED).
Magnet temperatures & cooling are OK.
MSE PC is ON.
• NOT MASKABLE !
• Whenever an interlock on the magnet is generated, the USER_PERMIT is switched to FALSE ~ 100 ms BEFORE the PC is switched off.
05.10.2006 CNGS Interlock System Training / J. Wenninger 22
Miscellaneous Inputs
• CCC button / OP inhibit (13) :• USER_PERMIT=FALSE if the button ‘LSS4 Inhibit’ is pressed (second button from the right).
• NOT MASKABLE !
• Equivalent of the console switch for the fast extraction.
• Horn & reflector status (14) :• USER_PERMIT=TRUE when both HORN and REFLECTOR are ON. MASKABLE !
• Hadron Stop Cooling (15) :• USER_PERMIT=TRUE when the cooling circuit of the CHGC hadron stop is running. MASKABLE !
• Fire Alarm (16) :• USER_PERMIT=FALSE when a fire is detected in the CNGS target cavern. MASKABLE !
05.10.2006 CNGS Interlock System Training / J. Wenninger 23
Software Interlock System
• The new SIS (Software Interlock System) that will replace SSIS (2007 ?!) provides two software interlock inputs.
• It provides additional (and slower) cross-surveillance and diagnostics. Next year it will also directly cut the beam (based on destination) through the timing system.
• TT41 software surveillance:
• Summary of PC states (TT41), target position, BTV positions, TBSE position, Shutter, BLM gains (avoid saturation) and thresholds, etc is sent to BIC module CIBC.TT41A.
• The status is refreshed for every CNGS user (CNGS1/2/3/4) – at the end of the cycle.
• TT40 and LSS4 surveillance:
• Summary of PC states (TT40 & LSS4), BTV positions etc is sent to BIC module CIBC.TT40B.
• The status is refreshed for every CNGS or LHC user (CNGS1/2/3/4, LHC1/2, MD1/2/3/4) – at the end of the cycle.
• CNGS Protection does not depend critically on SIS. In case of SW problems you can therefore mask channels that give problems. This year we had quite frequently problems with the PC surveillance, and frequently had to mask all PC interlock (will have to be fixed next year).
05.10.2006 CNGS Interlock System Training / J. Wenninger 24
Pre-extraction Surveillance
05.10.2006 CNGS Interlock System Training / J. Wenninger 25
Powering failures
TT41 Main Bends
Tol.Tolerance
Powering ‘failures’ are among the most likely and most critical failures :
• Wrong converted setting surveillance of the current VALUE.
• Converter failure FAST surveillance of the current CHANGE.
Examples of simulated powering failures
Tolerance
05.10.2006 CNGS Interlock System Training / J. Wenninger 26
ROCS PC Current Surveillance
• The ROCS system provides a pre-extraction surveillance, the so-called FEI (Fast Extraction Interlock). The current of selected converters has to match a reference within a pre-defined tolerance. The surveillance is performed at the last possible moment ~ 2 milliseconds before extraction.
• This system provides in total 6 inputs to the BICs, all inputs are MASKABLE:
MSE.418 converter (19) Interlock DCCTs for shared main converter (22)
• Operational current tolerances :
MBHA, MBHC dipole strings 0.2%
Main dipole string 0.1%
Interlock DCCTs 1.0%
MBSG dipole string 0.1%
Septum 0.1%
Main quad strings (D/F) 0.2%
Matching quads 0.5%
Corrector magnets ~ 10 rad
Bumpers ~ 1 rad
05.10.2006 CNGS Interlock System Training / J. Wenninger 27
ROCS Surveillance Timing• For each extraction, the ROCS system provides four 3 ms long pulses with PERMIT = TRUE (if all OK) which
sets a strong constraint on the timing event sequence (minimizes possible errors).
• The LEGACY events that trigger the ROCS are:• OEX.FINT1-CTM at -13 ms
• OEX.FINT201-CTM at 0 ms (first extraction)
• OEX.FINT202-CTM at 0 ms (second extraction)
Extraction kicker triggers ~ Nominal Extraction Time + 800 s (arrival time of pre-pulse)
Do NOT change the delays !!!!!!
PFN charge only startsif the PERMIT is TRUE !
05.10.2006 CNGS Interlock System Training / J. Wenninger 28
Shared Main Bend Converter
MUGEF
DCCT
Ref.
Cyclestatus
MUGEF
ADC
ADC
Extractionpermit / abort
TI 8Main
Dipoles
CNGSMain
Dipoles
ControlSystem
3 600 V
5 400 Apeak
MB 1 MASTER
MB 2 SLAVE
Firing
Interlock DCCTs
MUGEF for ‘standard’surveillance
• The TT41 and TI8 main dipoles are powered by a single converter, with switches (presently mechanical, next year electronic) to send the current into the correct magnet string.
• To ensure that the switch position is REALLY correct, we have 2 ‘dummy’ ROCS channels that have only an interlock DCCT but no converter. The names of the ROCS are DCCT_TI8 and DCCT_CNGS (also accessible from equipstate).
• The 2 DCCTs are used to identify which branch is powered, and their current is surveyed like any other converter (22).
05.10.2006 CNGS Interlock System Training / J. Wenninger 29
FMCMs• The FMCM is a special high sensitivity device that monitors the current change of a circuit
(through the voltage) and triggers an interlock when the current derivative is too large.
• 3 FMCMs (Fast Magnet Current Change Monitors) are installed on the MSE.418 (23), the MBSG.4000 (24) and the MBG dipole string (25) . The FMCM trigger thresholds have been determined by programming step functions into the converter reference (with and without beam).
SC time (ms)
PCFMCM trigger
(current change)ROCS survey
tol.
MSE.418 < 0.1% 0.1%
MBSG.4000 < 2.5×10-4 0.1%
MBG/MBI.816 < 2.5×10-4 0.1%
In all cases the position change @ target is maintained < 0.5 mm
(as specified).
• To avoid un-necessary triggering of the circuit during the converter ramps, the FMCMs are activated (input to TRUE) only at the end of the ramp when the currents stabilize. In addition a minimum current must be detected. The USER_PERMIT of the FMCMs is usually FALSE and switches to TRUE in a window of some hundred(s) milliseconds around the extraction. It switches to FALSE when the converter ramps down (looks like a failure…).
05.10.2006 CNGS Interlock System Training / J. Wenninger 30
Ring BPMs
• A number of SPS ring BPMs in LSS4 are used to interlock the beam position at extraction as a pre-extraction surveillance (extraction bump amplitude) (26) .
• The interlock logic is implemented within the MOPOS system.
• Unfortunately MOPOS remains fragile (~ one reboot required / day, gain changes) and this interlock is our ‘talon d’Achille’ !
Interlock timing forMOPOS
05.10.2006 CNGS Interlock System Training / J. Wenninger 31
Ring BPM Settings
• A priori one could interlock any BPMs in sextant 4. But MOPOS cannot manage the data (too slow).
• 8 positions (5 H and 3 V) are selected for the interlock around the extraction point (418).
• The ‘minimum’ system corresponds to 2 H (416/418) and 2 V (417/419) monitors.
• The nominal amplitude of the bump is 31 mm at BPCE.418 (corrected for BPCE non-linearity !) – this corresponds to ~ 29 mm on the normal display. The corrected value is available from the ‘Extraction Interlock’ menu in YASP (SPS ring, ‘Machine Specials’
• The tolerances on the positions are:
± 1 mm good protection
± 2 mm ~ at the limit (fallback if problems)
05.10.2006 CNGS Interlock System Training / J. Wenninger 32
Ring BCT
• For LHC / TI8 test an interlock on the beam intensity is provided by the ion BCT4 (27) .
• The BCT can be configured to only provide USER_PERMIT=TRUE when the intensity is below a programmed threshold. The intensity measurement is performed 80 ms before extraction (warning event).
• This signal is not used/activated for CNGS. The USER_PERMIT must always be TRUE during CNGS operation.
05.10.2006 CNGS Interlock System Training / J. Wenninger 33
Post-extraction Surveillance
05.10.2006 CNGS Interlock System Training / J. Wenninger 34
Transfer line BLMs• Since BLMs in TLs can only measure AFTER the ‘action’, they provide a Post-extraction
surveillance. The USER_PERMIT is latched to prevent further extractions if an excessive loss is detected.
• Two user permits are provided for TT40 (28) and TT41 (29).
• The reset is done manually (for the moment via steering program).
• The USER_PERMIT transition from FALSE to TRUE is triggered by the ‘-20 ms extraction pre-warning’ event. If there is no (abnormal) loss, the USER_PERMIT resets to FALSE a few milliseconds after the extraction.
05.10.2006 CNGS Interlock System Training / J. Wenninger 35
Transfer line beam loss & thresholds• Beam losses in TT40 & TT41 normally are very low (< 0.3 mGray for 1013 protons).
• When the C OTR screens are inserted losses increase to ~ 2 mGray in some locations.
Thresholds to 5 mGray in TT41.
• The gains must be ≤ 16 ! Saturation ! Surveyed by SIS !
• For the moment, the interfaces shown below are available from… the steering program !!!
BLM after TED : high thresholdto avoid interlocks when beam issent to the TED.
05.10.2006 CNGS Interlock System Training / J. Wenninger 36
Transfer line BPMs• A Post-extraction surveillance has also been implemented for the CNGS trajectory (30) .
• Each of the 23 H+V BPMs in TT40 (4 BPMs) and TT41 can be interlocked. They can be activated individually. Tolerances:
• ±4 mm for 20 transfer line BPMs TL aperture
• ±0.5 mm for last 3 or 4 BPMs target tolerance
• The interlock is latched (reset through steering application).
• A position is used for interlocking only if the BPM has seen beam (auto-triggered): to trigger the BPMs one needs one batch of ~ 2-3×1012 protons (or an equivalent DENSITY).
• The interlock timing follows the same logic than for transfer line BLMs.
05.10.2006 CNGS Interlock System Training / J. Wenninger 37
Transfer line BPMs /2
Example : reference trajectory for the CNGS pilot run
Short-circuit on one electrode. De-activated for
interlocking.
Threshold ±4 mm Threshold ±0.5 mm
Misalignment transfer line-target-horn
MKE• The MKE extraction kicker provides a slow control interlock to the interlock system (31) .
• The MKE signal is only TRUE when the kicker is ON. NOT MASKABLE !
- PFN voltage
- BIC permit
- BETS window
- Beam
Timing of kicker, BIC permit and BETS:
Extr. 1 Extr. 2
• Internally the MKE has an energy tracking system (from a mains DCCT in BA3). This internal ‘BETS’ (Beam Energy Tracking System) system enforces the following interlock logic:
Kicker voltage 50 kV (nominal) ± 2 kV.
Energy must be in a window of 400 ± 5 GeV.
The BETS interlock is only visible in OASIS and on the scope display (example to the right) which is
available under: http://kestek4/
When the BETS interlocks, the magenta curve is flat !
A symptom for a BETS interlock is that the MKE does not kick although the
05.10.2006 CNGS Interlock System Training / J. Wenninger 39
Safe Beam Flag
• Masking of any MASKABLE input to the CNGS BIC system is conditioned by the SPS safe beam flag (SBF). SBF = TRUE if the intensity is below a predefined threshold.
• A MASKED INPUT is AUTOMATICALLY reactivated if SBF=FALSE.
• SBF Generation:
• The SPS high intensity BCT is used to measure the intensity 1 second after the start of the ramp (triggered by a timing event).
• The intensity is sent to the SPS MTG and compared to the safe beam threshold to generated the SBF.
• The SBF is distributed over the timing system to the BICs.
• Presently the limit for a safe beam is set to 1012 protons. This is a rather conservative limit and it is difficult to obtain CNGS beams of such a low intensity. I have therefore suggested to increase the limit to 3×1012 protons which should be sufficiently safe.
05.10.2006 CNGS Interlock System Training / J. Wenninger 40
Extraction-Ring Interlock Systems Connection
Two systems that are part of the extraction interlock system are (also) connected to the ring BIC system:
• The extraction kicker MKE dumps the beam:
• When it is pulsing in local.
• When the kick is enabled and the interlock system has not given green light for extraction. After three consecutive (same USER) cycles in such a condition the SPS Emergency System cuts all beam.
• The BLMs in LSS4:
• Typical loss on the first BLM ~ 10-20 mGray – threshold at 50 mGray.
• The loss is due to beam in the gap. If losses become excessive, check the gap population with the fast BCT. Gap population depends on the PS CT, and our RF settings (injection phase, bfield, RF voltage on FB).
• Do not increase the thresholds ! Radiation in ECA4 !!
• ‘As usual’ : interlock is latched by SPS Emergency System after 3 consecutive interlocks (same USER !).
• Fabio’s new application will be used from now on to read & control those BLMs.
05.10.2006 CNGS Interlock System Training / J. Wenninger 41
Radiation in ECA4
Radiation in ECA4:
H. Vincke
05.10.2006 CNGS Interlock System Training / J. Wenninger 42
BIC Supervision
05.10.2006 CNGS Interlock System Training / J. Wenninger 43
BIS Supervision Software
• The supervision applications for the SPS BIC systems (ring and CNGS) are available from the SPS console manager under:
Start Tasks SPS Control SPS Beam Interlocks
• Each ‘bubble’ on the top screens represents a BIC module, the color encodes the BIC output (RED=FALSE, GREEN=TRUE) and the state is sampled at ~ 1 Hz. Note that this is too slow to observe changes for the extraction where the signals are very short and fast.
05.10.2006 CNGS Interlock System Training / J. Wenninger 44
BIC Supervision – Main Screen
Clicking on a ‘bubble’ opens the main supervision screen for the corresponding BIC module. On the main screen :
• Connected input list
• Input status
• Masks (status & actions)
• Safe beam flag
• Permit status
Refreshed at ~ 1 Hz.
From the main screen:• (Un-)masking of inputs.
• History buffer view.
• Expert screens view.
• Bar graph view.
Inputs 8 to 14 are always maskable
Inputs 1 to 7 are always un-maskable
Mask status
Safe Beam Flag status
Software Interlock System Input
BIC Supervision – history buffer
Time within SPS cycle in ms/s
Signal name / transition
State of BIC output
Timing marker : start of cycle
Signal transition : (A) and (B) refer to the 2 redundant signals
for each input !
• The history buffer logs all transitions and signal changes (also masks and SBF) as well as the start of every cycle (as a marker).
• Very powerful for debugging, but becomes complex when there are many signal transitions !
BIC Supervision – Bar Graph
Time offset wrt start of cycleDepth in seconds wrt
Time offset
The bar graph view shows the status of the inputs/output as a function of the CYCLE TIME: it is a graphical display of the history buffer.
• Display signals for one SPS cycle, or a sub-range of a cycle (for example around extraction)
• Refreshed every cycle (i.e. every 6-12 seconds)
Extraction permit switching to TRUE for extraction (3 ms wide)
Check this box to hold selected time range values
Check this box to freeze display
05.10.2006 CNGS Interlock System Training / J. Wenninger 47
BIC supervision for CNGS operation
Graphical display of all signals for a (successful) CNGS extraction…
05.10.2006 CNGS Interlock System Training / J. Wenninger 48
Supervision Software – new ?
Permit Time Delta MS Description
15:30:01 12000 Timing Marker
15:30:05 4220
15:30:05 4270 PC TT40, Vaccum TT40
15:30:07 6000 Timing Marker
15:30:11 4220
15:30:11 4270
15:30:13 6000 Timing Marker
15:30:17 4270 FMCM MSE, BPM Extraction
15:30:17 4270 FMCM MSE
15:30:19 6000 Timing Marker
Same columns that are also found in the history buffer.
List of inputs in state FALSE at the time of the snapshot, or indication for timing marker
I have recently proposed to implement a new table display that would summarize the status at selected times (corresponding to the extraction):
• If the output of the BIC is FALSE, the list of inputs that are false is listed.
• This provides a simple snapshot of the system status at the time of extraction.
The depth of this table should be ~ 50 to 100 cycles…
05.10.2006 CNGS Interlock System Training / J. Wenninger 49
Diagnostics software
For continuous surveillance interlocks, the diagnostics is rather ‘easy’ since the errors are more or less static – one just has to find the appropriate application !
It is more tricky for highly dynamic interlocks ….
05.10.2006 CNGS Interlock System Training / J. Wenninger 50
General Post-mortem• A post-mortem application to browse logged data has been written by Verena. It can be
found in the console manager in the same place as other interlock stuff :
Start Tasks SPS Control SPS Beam Interlocks Extraction post-mortem
• This application is very powerful, but it can be tricky to start using it just when a problem appears !
• It is very useful to analyze trends, changes over a few hours…
• Try it out when you have a quiet moment during a shift !
Diagnostics for position interlocks : steering application
• Post-mortem freeze of the display (background becomes ORANGE) when there is no valid data freezes the last acquisition for (interlock) diagnostics.
• The possibility to define a GOLDEN reference for the steering such that everyone uses the same reference (please use it !).
• The possibility to monitor the trajectory on all cycles (CNGS1,2,3) in the same application. But beware when steering since the context selection corresponds to a defined user !
‘Acq&Hw’ menu Acq Configuration ‘Timing’ tab
Change USER to : SPS.USER.CNGS*
Click on button ‘Set User’
• Interlock diagnostics …
Latch status
05.10.2006 CNGS Interlock System Training / J. Wenninger 52
BLM diagnostics
• There is no application for the transfer line BLMs. The only tools for diagnostics are :
• For the LSS4 extraction BLMs, there is a new application from Fabio (replaces Labview !).
• Fixed displays.
• Verena’s PM (through the logging).
• A diagnostics panel in the steering application (also for thresholds !). The same panel also provides the RESET command for a latched BLM threshold.
05.10.2006 CNGS Interlock System Training / J. Wenninger 53
ROCS diagnostics
• Diagnostics for the ROCS interlocks is in the form of an expert program, with references in bits… Not recommended for the non-initiated (so far Michel, Jorg, Rossano, Verena) since one can easily screw up the system.
• Fortunately this system proved very stable and gave no problem over the entire run (I use it since May).
• For the moment, refer to one of the ‘initiated’ – I will see if one can provide something for diagnostics only.
05.10.2006 CNGS Interlock System Training / J. Wenninger 54
CNGS Operation
•Experience from Pilot run in August
•Tentative guidelines for the coming CNGS run
05.10.2006 CNGS Interlock System Training / J. Wenninger 55
Extraction & TL Stability
• The good news from commissioning and pilot run is that the CNGS extraction & transfer line are VERY stable.
• Observed integrated changes during the 12 days of the pilot run:
Total drift in horizontal plane 0.2 mm rms
Total drift in vertical plane 0.4 mm rms (dominated by drift in SPS ring)
Vertical position change in the SPS ~ 0.5 mm
• Trajectory corrections:
• During the pilot run I made one correction in each plane every 1-2 days using 1-2 correctors (MICADO), which maintains the rms change around 0.1 mm or less.
• The corrections over a week are sufficiently small to fit into the interlock tolerances for the orbit correctors of TT40/TT41.
• In principle it is sufficient to steer 1-2/week. The interlock system will protect us against excessive trims (corrector limitations !).
• I will try to prepare an operational Autopilot for CNGS ...
05.10.2006 CNGS Interlock System Training / J. Wenninger 56
Interlocks : the bad & the ugly
From the entire protection system, only 2 interlocks gave problems and/or required ‘interventions’ during the pilot run:
• The transfer line BPM interlock (30) due to unphysical readings in the region near the target where the tolerances are very tight.
• The LSS4 MOPOS BPM interlock (26) due to spurious interlocks and problems with gains during the second ½ of the pilot run (increased intensity):
The signal amplitudes were near the lower part of the range for the optimum gain setting (10 dB).
Gain increases (necessary to measure the first turn) led to saturated signals interlock.
Intensity drops (PS) led to insufficient signal amplitudes interlock.
We also observed that some MOPOS settings seemed to be occasionally ‘corrupted’. On 2 occasions during this period, the gain of the system changed without any actions from the shift crews. On such occasions the gain always returned to 20 dB! Has been observed again during recent MD periods.
Beam not extracted dumped out-gassing injection kicker inhibit (vacuum)
I fear the this problem will come back, in particular because the beam dump is not much more conditioned than in August !
05.10.2006 CNGS Interlock System Training / J. Wenninger 57
Fake Interlock from TL BPMsExample for a trajectory leading to a FAKE (and latched !) position interlock in the transfer line.
• The interlock is generated by the 2 BPMs near the target that are interlocked to ±0.5 mm.
• Observed on 2 occasions (at the beginning and near the end of the run).
• False readings always occurred after a period without beam (at least a few cycles).
• After reset the readings are OK again.
Not really understood…
Example for a bad trajectory (difference wrt reference) leading to a fake interlock
Bad readings
05.10.2006 CNGS Interlock System Training / J. Wenninger 58
Guidelines…
The extraction interlock system has significant redundancy for the most critical and tricky operation issues concerning PCs, beam positions and BLMs – I think we are well protected !
• Guideline 0: The coming run is for us, not for OPERA. We do not have to be stressed by them, and can therefore take time to study & learn.
• Guideline 1 : Do not touch interlock settings if you are not 100% sure of what you do ! It is better to be a bit inefficient (for this year) than to drill holes. I take the blame !!!
• Guideline 2 : in case of doubts,
• change the beam to MD3 (CNGS low intensity).
• test (you can mask !).
• Guideline 3 : after a long(er) stop, for example an MD,
• Put in the TED
• Make one extraction (nominal beam), check trajectory to TED (change < 1 mm).
• Make two extractions, check.
• If all OK, open TED, extract to target.
• If NOT OK Guideline 2 !
05.10.2006 CNGS Interlock System Training / J. Wenninger 59
More guidelines…
• Guideline 4 : When an interlock is latched, and yoy have to decide if you should reset or not..
the TT40/41 position interlock is latched – must take a decision : reset or not !
• If you have the frozen steering display and if it looks like isolated large readings:
•Reset and try again.
• If you have no info on the trajectory:
•Start the steering display.
•Reset and try again.
• If the interlock latches again: think hard and be careful – possibly move to Guideline 2.
beam losses in the transfer line (latched) or REPEATED losses in LSS4.
• ABNORMAL !!!!
• Stop and check everything you can (abort gap etc…).
• Reduce intensity, try a single extraction. Be sure you have all diagnostics running for such test shots.
• Guideline 5 : people with (some) experience on this system
• Jorg (I will be away for ~ first days of the CNGS run), Verena, Rossano
05.10.2006 CNGS Interlock System Training / J. Wenninger 60
WEB documentation
https://cern.ch/sps-mp-operation/A WEB page is installed for MP