Review of actions from 2 last reviews · •Double shunt on quadrupole bus •Insulation system •Spiders and lyras electrical insulation ... The Splice Task Force needs to work

Third LHC Splices Review

Review of actions from 2 last reviews

•Double shunt on quadrupole bus

•Insulation system

•Spiders and lyras electrical insulation

•DFBA consolidation

•Actions covered by TE-MPE

•Technical aspects

•Management aspects

•Other

NB : 33 actions from first review, 15 non-closed ones

18 actions from second review

13th of November 2012 Third LHC Splice Review


Review I: II.3 1) p7/14 : Double shunt on the quadrupole bus The Committee recommends that a double shunt be applied to the quad bus.


Double shunt on quadrupole bus

See presentations by F Savary, C Scheuerlein, Knud Dahlerup Petersen,

and other ones

Review II: 1 II.1 p4/11/14 : Criteria for single shunt on quad bus The following criteria must be met, if a single shunt is to be used on the quadrupole bus:

- The rebuild criteria presented in Christian Scheuerlein’s talk must be applied rigorously.

- The insulating cover must be effective in limiting the deflection and stress on the splice,

to prevent the future degradation of the copper stabilizer connection.

- The energy extraction time constant must be reduced to ≤ 20 sec.

- The shunt must be soldered properly and QC of the shunt must be done correctly. The

soldering procedures and QC plans should be presented at the next review.

2nd Review: The Task Force has not found a satisfactory way to implement this

recommendation. We believe that the single shunt can be acceptable if certain

other conditions discussed above are met. This cannot be closed until the

insulating cover design is complete and shown to provide adequate mechanical

constraint, and the other criteria cited in recommendation (n) above.


Review I: II.3 4) p7/14 : Mechanical calculations Make mechanical calculations for the shunt design and the insulating part (3D stress analysis including fatigue

and thermal stresses) and document them to support the final design. All the new model calculations need to

incorporate shunts in the most severe conditions assumptions (for example no mechanical connections internal to

the original splice due to the lack of soldering).


Consolidated insulation system

2nd Review: Much of this has been accomplished, but the current model for the

insulation box is not fully adequate

Review I: II.3 6) p8/14 : Clamping system Continue the development of the design for a clamp that would hold together the shunt and splice in the absence

of solder, to help limit the damage during the time of an energy extraction, and implement the chosen design.

2nd Review:Good progress, but design is not yet final

Review II: 2 II.1 p4/11 : Insulation cover modelling Review the design and the modelling of the insulation cover to show that it is effective in reducing the mechanical

stresses on the splice and shunt, for both dipole and quadrupole connections

See presentations by F Savary

Review I: II.4 1) p9/14 : Industrialize manufacturing procedures The Splice Task Force needs to work on improving the manufacturing procedures and the tooling in such a way

these are reproducible and not depending on the individuals doing the work. This includes developing adequate

QC plans. The review committee recognized the fact that, at this stage of the consolidation programme, this is

more important than working out a detailed schedule.

2nd Review:Excellent progress overall, but cannot be completed until the design is

complete, particularly of the insulation box


Review II: 12 II.3.3 p6/11 : Spiders insulation Continue to explore ways

to insulate the spider from ground potential

and to increase the creep path of the buss to metal support of the spider.


Actions from second review

Review II: 13 II.3.3 p6/11 : Lyras insulation Continue to explore external methods to insulate the lyras from ground potential. Even if a solution

cannot be found for all case where these two electrical insulation problems are in issue it would still

be best to repair what is readily accessible.

Triggered by: Screening for Defects in the LHC Magnets Electrical Circuits (Mar2010-Jun2011), Final Report,

Conclusions and Recommendations [EDMS 1176186]

Presented at the Review by H Ten Kate

Working method 1. The screening person prepares a presentation with supporting documents and raises a list

of risk issues he or she thinks are relevant.

2. The experts group defines (a list of) practical recommendations for the system under

study…

3. A group (TE-MSC-LMF) documents a detailed analysis, design and engineering solution.

4. CERN management decides to execute or not the engineered solution following the

recommendation




The consolidated electrical insulation (H Prin) https://indico.cern.ch/getFile.py/access?contribId=40&sessionId=4&resId=1&materialId=slides&confId=157231

Presentation of a first concept, only partial and not finalised

Implementation

Heating during welding (~300°C)

Friction during cooling

Fixation to the flange

Behavior during quench

Reliability in time

Remnant in the cryogenics filters

A lot of work was

accomplished

(solution, tests,…)

https://indico.cern.ch/getFile.py/access?contribId=40&sessionId=4&resId=1&materialId=slides&confId=157231

https://indico.cern.ch/getFile.py/access?contribId=40&sessionId=4&resId=1&materialId=slides&confId=157231




Available partial solution H Prin :SMACC Meeting 11.07.2012 [EDMS 1232922] / LHC splices TF 12.09.2012 [EDMS 1241232]

Rolled sheet of fiber glass impregnated with EPOXY (0.25mm)

Fixation to the insulation box (Stainless Steel)

Flange aperture not covered to avoid degradation during welding




Available partial

Solution implemented on the

Q8-Q9 test in Sm18 on both

M1 & M3 lines




Experience: Spiders: Only one NC [EDMS 831608]

Bellows was cut, insulation reinforced and spider was

corrected

A corrective procedure is available and mastered

Was detected during standard ELQA test during

assembly, prior to operation

Likely generated by assembly work (surface or

interconnection in the tunnel)

This kind of defect (if any) will be detected during

the ELQA test during splices consolidation

Present ELQA diagnostics guarantee that there is

no such defect in the LHC

QPS diagnostics will likely be improved

(amplitude to be defined)

The PAQ HV test of the half-cell 23R6 not passed. Short between the main BB M3 ext.

and the ground.Resistance main BB M3 ext. versus ground is 0.3 Ohm (measured at

QBQI.22R6) and 5.3 Ohm (measured at QBQI.23R6)




Experience: Lyras: [LMC 22.09.2009]

Was detected by ELQA monitoring during cool-down, prior to operation

Generated by motion during thermal contraction

This kind of defect (if any) will be detected by the ELQA diagnostics

Present ELQA diagnostics guarantee that there is no such defect in the LHC

QPS diagnostics will likely be improved (amplitude to be defined)

After 3 day of cool-down, a short circuit to ground was detected on the MBA.A67 line of

the main dipole circuit. After diagnostic, the defect is localized at the level of dipole 3287

at the position A12R6. The warm-up is launched for repair.

Sleeves and bellows were cut and insulation reinforced

Corrective procedure available and mastered

No preventive action recommended (LMC)




Experience: Lyras: Repair procedure



INVENTORY OF THE SINGULAR POINTS AND FIRST RISK ASSESSMENT FOR THE LHC 13 KA

CIRCUITS [EDMS 1144154: 267 pages]

Risk index = (Technical) ( Criticality + Sensibility ) Frequency

Technical : mechanical aspects, cu-cu joint, SC splice,

insulation concern [4 to 12]

Criticality : quality of the construction process : Sum of

construction difficulty and access difficulty [0 to 5]

Sensibility : consequences of a failure : Sum of failure

detectability and failure consequences [0 to 5]

Frequency : Recurrence of each singular point / Total

number of Singular Points [X / ≈ 22 500]

Singular points are : every splices in the cold mass, lyras,

fixed points, diodes connections, ….

For the spiders : RI =

6 * (1 + 3) * 0.064 = 1.54 (dipole circuit)

6 * (1 + 3) * 0.032 = 0.768 (quadrupole circuits)

For the lyras/sextupole corrector supports : RI =

7 * (3 + 4) * 0.064 = 3.144





Aren’t we contaminated by the lamppost syndrome ?

Ex 1. Splices inside the cold masses have a higher risk index

Corrective actions for the more critical cases

8 dipoles and 1 SSS replaced




Ex 2. BUS BAR – HEAT EXCHANGER CONTACT

No systematic preventive action

Reinforced on cold masses installed during LS1

A corrective procedure is available and mastered

Other cases for which no systematic

action is planned because not

pragmatically feasible.

The best solution would be to redesign the spiders and other weak components

• Not pragmatically feasible

schedule [many months], budget [several MCHF]: in the best cases

• Not riskless operations




Our approach is to carry out corrective

interventions complemented by

systematic inspection of new magnets

and random partial inspections in the

LHC tunnel because:

Corrective procedures are available and mastered

No failure occurred during operation

Diagnostics efficiency for this kind of issue is proven

LHC machine can tolerate one fault to ground per sector

Systematic preventive interventions are not pragmatically feasible and are

not riskless

In the unlikely case of failure, the extent of the damages will be limited thanks

to finalisation of consolidation measures (Safety relief devices, SSS

anchoring to ground)



DFBA consolidation

See presentation by A Perin

Review II: 10 II.3.3 p6/11 : Initiate design for DFBA splices consolidation Initiate the design for the DFBA splice repair immediately

Review II: 11 II.3.3 p6/11 : Study methods to lower risks for specific DFBAs Investigate methods and/or procedures that would lower the risk of repair of the two DFBAs that

require re-location to perform the work.

Review II: 9 II.3.3 p6/11 : Consolidate DFBA splices Apply shunts to the 13 kA splices in all DFBAs, including the most difficult circuits Repair these

joint up to the same standards and based on the same designs as in the magnet to magnet

interconnect. Splice repairs should be done by the same crew that repairs the magnet to magnet

interconnects.


Review I: II.7 3) p14/14 : Actual operating voltages Determine the actual operating voltages to judge required hipot levels


Actions covered by TE-MPE

2nd Review: In progress, but new hipot voltages have not been formally established.

Review I: II.7 4) p14/14 : Proper dump voltages Check documented dump circuit requirements or redo quench calculations to assure dump voltage

levels are proper

2nd Review: In progress

Review I: II.7 5) p14/14 : Simultaneous hipots of different buses Consider simultaneous hipots of different buses to test bus-to bus withstand capability without

going to needlessly high hipot stresses

2nd Review: This technique seems not to have been considered as part of the re-

evaluation of hipot voltages. It should still be considered.

See presentations by K Dahlerup Petersen & R Schmidt

Review I: II.2 4) p5/14 : Ground current/voltage monitors. Make necessary changes so that ground current monitors are installed on all lower current circuits.

These ground current signals should be included in the control system to be logged and to be

alarmed when above a pre-set level. The alarm level should be ubstantially below the power supply

hardware interlock for high ground current

2nd Review: Closed, Implementing it


Review I: II.2 3) p5/14 : Hot spot calculations. Include in the integrated system surveys the temperature limits of the various components of each

circuit. Perform the calculations to determine the maximum temperatures for the components in the

different circuits. Check to see that there is an appropriate margin for the maximum temperature of

the circuit components and that the overall risk are balanced between voltage and temperature.


TE-MPE

2nd Review: Work in progress. This analysis should be completed, at least for the more

critical elements, before LS1 to be able to profit from anything that is learned from the

analysis.

Review II: 8 II.3.2 p6/11 : IT and MS splices measurement Provide sufficient resources to make splice measurements on the inner triplet and matching section

circuits well before the beginning of the long shutdown.

Review II: 15 II.3.3 p7/11 : Documentation and modeling Supply the necessary resources to finish the documentation and modeling tasks. The Task Force

needs this information and tools to analyze the various electrical systems to better understand

potential failure mechanisms and to improve testing of various system

See presentations by R Schmidt


Review I: II.3 Com2 p6/14 : Document shunt testing The Committee finds results from high current shunt testing between FRESCA and SM18

confusing. Modelling of all test cases should be documented and qualified based on experimental

data, and then applied to the shunt design with worst case assumptions.


Technical aspects

• Report was released on 20.01.2012

“Electro-thermal and mechanical validation experiment of the consolidation of the LHC

13 kA arc splices”

• See presentation by F Lackner & F Savary


Review I: II.3 3) p7/14 : Multiple samples to get experience Make multiple samples/trials of critical steps, as part of a qualification plan to be developed, to get a

statistically significant basis for assessing the risks of there being any important imperfections in the

machine after installation


Technical aspects

2nd Review : Have done multiple tests on many variants of the splice repair

components. Once the design is completed, a statistically significant number of samples

of the final design still need to be tested

Review I: II.5 3) p11/14 : Lead containing solder The Splices Task Force should consider the potential hazards for workers associated to the

utilization of lead-containing solder, and work with the Safety Commission to plan the work to

minimize or eliminate this risk

2nd Review: Barely begun. This is important, since in the original design of the LHC the

use of lead-containing solder was rejected on safety grounds.

Review I: II.5 4) p11/14 : Tooling calibration plan A verification and calibration plan for the special tooling should be drawn up and implemented to

assure a constant quality of the work throughout the production

2nd Review: Nothing presented at this review.

T Otto’s presentation

Started


Review I: II.6 2) p12/14 : Optimize tooling and validated on mock-ups The Splices Task Force should draw out a work plan for redesigning and optimizing the various

tooling needed for the consolidation activities. These tooling should be validated on the surface

simulating as real working conditions as possible (e.g. space constraints)


Technical aspects

2nd Review: Good progress. Potential still exists for improvement in some testing

fixtures and tooling related to installation of the insulating cover has not been finalized.

Review II: 4 II.1 p4/11 : Procedures for training Write detailed manufacturing procedures and QA procedures within the next few months in order to

be used for the training of the crews.

Review II: 5 II.1 p4/11 : Procure tools Start procurement of the M-line cutting machines, busbar machining tooling, shunt soldering tooling

and main splice tooling now in order to stay on schedule.

Review II: 6 II.2.2 p4/11 : Consolidated QC plan, clarify criteria Consolidate the QC plan, in particular on how to implement the plans and procedures and some of

the qualification criteria which have not yet been clarified.

Launched


Review I: II.4 2) p9/14 : Workplan, schedule, WBS It is necessary to have a preliminary work plan with its associated schedule and WBS


Management aspects

2nd Review:In progress. No WBS exists yet.

Review I: II.4 3) p9/14 : Professional scheduler It would be advantageous for the Splice Task Force to work with an experienced/ professional

planner/scheduler. The planning needs to be integrated with the planning for other work that will be

done in the tunnel during the same shutdown.

2nd Review:This has begun, but most of the work still remains to be done

Review I: II.4 4) p9/14 : Contingency and extra resources The planning needs to take account of the possibility of unforeseen developments that will slow

down or disrupt the orderly work flow. For example, a larger number of splices than the currently

estimated 15% may be required to be remade, or problems with the machining of the copper

stabilizer could arise. Include schedule contingency into the baseline and ensure that additional

resources are available to able to maintain the schedule.

2nd Review: Little progress since last year

See presentations on SMACC Organisation and by K Foraz, M Pojer

The Organization, Scheduling and Safety section of EN-MEF was reinforced by M.

Bernardini (Work in close collaboration). See also K Foraz’s presentation

R Principe (MSC-LMF) is in charge of the detailed scheduling of the splices consolidation

See presentations on SMACC Organisation

Enough?


Review I: II.6 1) p12/14 : Alternative scenario with unforeseen issues Study alternate scenarios in view of optimizing the utilization of the critical resources that will

certainly be limited (in terms of qualification and experience). Study unforeseen/unexpected

scenarios


Management aspects

2nd Review: No evidence that this has started

Review II: 3 II.1 p4/11 : Ratio exp/non-exp staff Do not reduce the ratio of experienced staff to contract or other temporary labor below the currently

planned level, in order to maintain the required quality of the repairs

Review II: 16 II.4 p7/11 : Update plans, for work QC resources and schedules Update the plans for the work organization, the quality control, the resources and the schedules.

Make further updates once the designs are final.

Review II: 17 II.4 p7/11 : Resources loaded schedule for the preparation phase Prepare a detailed resource-loaded schedule of the work that remains to prepare for the shutdown.

This needs be developed within the next month to be an effective planning tool.

Not formally but through SMACC project and TF structures


Enough?


Enough?


Enough?


Review I: II.6 2) p12/14 : Risk analysis CERN should insure that enough resources are allocated to ensure that the risk assessment and

subsequent risk analysis can be executed on the planned time scale


Other

2nd Review: Excellent work was done on the screening exercise, but it is important that

the recommendations that result be implemented. It should be the goal to complete the

critical ones before LS1

Review II: 7 II.2.3 p5/11 : CSCM to be considered for periodic requalification Consider the utility and practicality of utilizing the CSCM for periodic “requalification” tests of the

main quadrupole bus circuit.

Review II: 14 II.3.3 p6/11 : Diode joint resistance Continue working on determining the root cause of the measured increase in diode mechanical joint

resistance

See next presentation by F Savary

To be revisited by the splices TF

See dedicated session (H Thiesen)


Review II: II.4 p7/11 : Address the recommendations that remain open from the

previous review before the next review.


Conclusion

This presentation in particular and more generally this review

?

Review of actions from 2 last reviews · •Double shunt on quadrupole bus •Insulation system •Spiders and lyras electrical insulation ... The Splice Task Force needs to work

Documents