Priest Rapids P08 GCB Incident WSU Relay School · Priest Rapids P08 GCB Incident WSU Relay School Ty Ehrman Senior Manager, Power Production Engineering Bob Lemon Electrical Engineer

Priest Rapids P08 GCB IncidentWSU Relay School

Ty EhrmanSenior Manager, Power Production Engineering

Bob LemonElectrical Engineer

March 17, 2017

Topics

• Generator circuit breaker functionality and layout• Incident significant switches/disconnects• Events leading up to the incident• Incident timeline summary• Incident causes• Summary• Questions?

GCB Functionality and Layout• What size of GCBs are at Priest Rapids?

– Medium-voltage circuit breakers: 1-72 kV– All Priest Rapids units = 13.8 kV

• What type of breaker is at Priest Rapids?– Sulfur hexafluoride (SF6) breaker– These breakers interrupt the current by creating and extinguishing the

arc in a container or bottle, with its contacts surrounded by SF6 to extinguish the arc

P08 GCB Functionality and Layout

Incident Significant Switches/DisconnectsMidway A1000 circuit breaker• The breaker controls Line 2 from Midway substation to Priest Rapids

– This line services transformers C and D – four units (P05 thru P08)• Continued on clearance at the beginning of the day (10/8) - Line 2 de-

energized• Clearance through Power Management’s DispatchTransformer D disconnect 1285• Prohibits energy coming into or leaving the “stepped up side (230 kV)” of

transformer D for unit 8• Operated – and left open – by Electricians and I&C Technicians as a part

of scheduled maintenanceP08 Generator Disconnects 831/833• Prohibits energy coming into or leaving P08 (13.2 kV)• Continued on clearance at the beginning of the day (10/8) – prohibits

energy coming into or leaving P08 generator and Generator Circuit Breaker (GCB)

• Clearance through Power Management’s Dispatch

Lower cabinet• Actuator

Middle cabinet• Interrupters

Upper cabinet• Controls

Hydraulic Drive Spring – Center Panel (plexiglas removed)

MOUNTING HOLES FOR PLEXIGLAS

Events Leading Up To The Incident

September 3• P08 taken out of service to repair a turbine oil leak

October 5 - 8• Transformer D and relay maintenance outage

– Serves P07 and P08• Line 2 out of service

– Serves transformers C and D: four units P05 – P08

October 8 (Thursday – end of Hydro work week)• Line 2 scheduled to return to service• Units P05, P06, P08 scheduled to return to standby

P08 GCB Functionality and Layout

Incident Timeline SummaryThe following timeline is a summary of the incident events on October 8, 2015

13:16 > Midway A1000 Clearance released• Notification and switching starts to energize Line 2

15:27 > Midway A 1000 breaker closed• Line 2 energized to disconnect 1285

16:01 > P08 Generator Disconnects 831/833 closed• Breaker not energized since Transformer D disconnect

1285 still open> P08 GCB open

Incident Timeline Summary16:02 > Operator observes problem with P08 pump and reports to

Control Room• The pump provides pressure for breaker to open/close

> Senior Operator calls Electrical Foreman• Reported P08 breaker will not close

> Electrical Foreman dispatches 2 Electricians (E1 & E2)> E1 proceeds to GCB> E2 gets FR clothing on and tells other Electricians (E3 & E4)

about breaker problem

16:16 > Chief starts pump and stopwatch (typical troubleshooting check), goes to Control Room

16:17 > E1 arrives at GCB, followed by E3> Electricians turn off pump in GCB> Electricians begin troubleshooting> Chief Operator returns to GCB

Incident Timeline Summary16:17 (cont.) > Chief Operator opens the GCB lower cabinet doors

> E4, Foreman & E2 arrive at GCB> Chief Operator starts pump> Electricians verify pump gear condition/operation> Plexiglas protective cover removal started> E4 attempts to assist with cover removal, but Foreman keeps

E4 out due to lack of FR clothing

16:19 > Operator closes Transformer D Disconnect 1285 - continuation of tasks to bring units back to standby

> GCB now energized from the transformer side

16:20 > E1 & E3 start to leave (time to catch carpool)> E4 asks if they tried the (manual actuation) solenoids, then

reaches in and presses left (open) solenoid and then presses right (close) solenoid

> Breaker slowly closes and incident occurs

Generator Breaker Cubicle

Cubicle Module

Forensic Analysis Underway

Incident CausesDirect Cause• The breaker was manually slow closed while energized with unit P08 in a

stopped position.

When the breaker was manually closed – 30 to 40 times longer than designed -the unit was at a dead stop.

Manually closing the breaker bypasses all interlocks designed to prevent it from closing under unsafe conditions.

The breaker is designed to close into a moving, synced unit; NOT slow closed into a stopped unit.

This caused severe arcing and overpressure within the GCB which resulted in the explosion.

Incident CausesRoot Causes• Management is not adequately setting and enforcing safety expectations

which has resulted in a poor safety culture.• On-site leaders (Chief and Foreman) did not control the work activities.

Additionally they did not ensure the energy sources were isolated prior to allowing the Electricians to troubleshoot the GCB.

Contributing Causes• The policy requiring job briefs has not been adequately enforced and does

not explicitly say that it applies to emergent as well as planned work.• Knowledge gap related to the GCB equipment among the Electricians, the

Chief Operator and Senior Operator.• None of the employees involved had a questioning attitude about safe work

practices or situational awareness in the events leading up to the incident.• The GCB external failure protection scheme did not account for the failure

mode experienced during this event.

Relay Discussion Points

• Describe Generator Protection, Line Protection and Transformer Protection.

• Describe Sequence of Events.• Discuss Breaker protection schemes.• Describe GCPUD new generator breaker

failure scheme.

Zones of Protection

Generator Protection

• 64G Stator Ground Fault• 87 Current Differential• 46 Negative Sequence• 51V B/U Line Protection• 40 Loss of Field• 59G Overvoltage

Sequence of EventsP8 ‐ GEN BREAKER FAILURE

TIME EVENT Time from previous even Time from breaker closure

16:07:42.801 SEL‐311C‐1 POWERS UP (DC BREAKER CLOSED) 00:00:00.00016:19:10.923 Transformer D is powered up 00:11:28.122 00:02:37.85516:21:48.778 P8 BREAKER CLOSES (52b clears‐ Breaker has closed at least 10 Degrees). 00:02:37.855 00:00:00.00016:21:49.612 5C TRIPS 00:00:00.834 00:00:00.83416:22:05.146 P8 TRIP COIL RECIVES TRIP SIGNAL 00:00:15.534 00:00:16.36816:22:05.146 SEL‐311C‐1 SENDS TRIP TO 2ND TRIP COIL 00:00:00.000 00:00:16.36816:22:05.171 P8 TRIP COIL DE‐ENERGIZES (52a CONTACT OPEN) 00:00:00.025 00:00:16.39316:22:05.171 SEL‐311C‐1 REMOVES TRIP TO 2ND TRIP COIL 00:00:00.000 00:00:16.39316:22:05.193 P8 BREAKER OPENS (52b makes). 00:00:00.022 00:00:16.41516:22:06.000 SEL‐311C‐1 SETS A ZONE 3 PHASE INSTANTANEOUS 00:00:00.807 00:00:17.22216:22:06.020 SEL‐387‐D TRANSFORMER DIFF IS SET 00:00:00.020 00:00:17.24216:22:06.020 SEL‐387‐D TRIPS THE 86‐1D 00:00:00.000 00:00:17.24216:22:06.020 SEL‐387‐D SENDS DIRECT TRANSFER TRIP TO SEL‐311C'S 00:00:00.000 00:00:17.24216:22:06.029 SEL‐311C‐1 RECIEVES IN106 (DIRECT TRIP FROM 387‐D) 00:00:00.009 00:00:17.25116:22:06.029 SEL‐311C‐1 TRANSMITTS DIRECT TRANSFER TRIP 00:00:00.000 00:00:17.25116:22:06.029 SEL‐311C‐1 TRANSMITTS BLOCK RECLOSE 00:00:00.000 00:00:17.25116:22:06.154 SEL‐311C‐1 RECEIVES TRANSFER TRIP ECHO FROM OTHER ENDS 00:00:00.125 00:00:17.37616:22:06.175 SEL‐311C‐1 RECIEVES DIRECT TRIP FROM OPPOSITE ENDS 00:00:00.021 00:00:17.39716:22:06.175 SEL‐311C‐1 ASSERTS OUT101 TO TRIP THE 86L‐2 00:00:00.000 00:00:17.39716:22:06.175 SEL‐311C‐1 SENDS TRIPS TO BREAKERS P5,P6,P7,P8. 00:00:00.000 00:00:17.39716:29:02.053 BREAKER P‐8 52b contacts show open ‐ Probably DC to breaker is off 00:06:55.878

Event ReportEvent: TRIP1 Targets: TRIP INST 87_1 87_2 87_3 A B CWinding 1 Currents (A Sec), ABC: 12.3 12.2 12.1Winding 2 Currents (A Sec), ABC: 0.0 0.0 0.0Winding 3 Currents (A Sec), ABC: 0.1 0.2 0.3Winding 4 Currents (A Sec), ABC: 0.0 0.2 0.0

SEL-387D WINDING EVENT REPORT

Winding 1 Winding 2 Winding 3 Winding 4 OUT IN OUT 1 = 86TAmps Sec Amps Sec Amps Sec Amps Sec 1357 135 OUT 3 = DTT

IAW1 IBW1 ICW1 IAW2 IBW2 ICW2 IAW3 IBW3 ICW3 IAW4 IN ICW4 246A 246 OUT 4 = ALARM[6] IN 1 = GEN 7 52b-4.68 2.26 2.42 0.00 0.00 0.00 0.01 -1.09 1.10 0.00 -0.03 0.00>.... b4. IN 2 = GEN 8 52b3.45 -10.1 6.82 0.00 0.00 0.00 0.03 0.92 -0.94 0.00 0.16 0.00 .... b4. IN 4 = GEN 8 LO10.8 -4.36 -6.43 0.00 0.00 0.00 -0.04 0.02 0.00 0.00 0.06 0.00 .... b4.

-1.95 11.4 -9.56 0.00 -0.00 0.00 -0.05 -0.19 0.25 0.00 -0.18 0.00 .... b4.[7]-12.2 4.72 7.39 0.00 0.00 0.00 0.05 -0.05 0.02 0.00 -0.06 0.00* .... b4.1.80 -11.2 9.58 0.00 0.00 0.00 0.03 0.34 -0.36 0.00 0.17 0.00 .... b4.12.0 -4.63 -7.35 0.00 0.00 0.00 -0.06 0.26 -0.22 0.00 0.06 0.00 1b.. b4.

-1.86 11.3 -9.56 0.00 -0.00 0.00 0.00 -0.01 0.00 0.00 -0.17 0.00 1b.. b4.[14]-11.9 4.68 7.25 0.00 0.00 0.00 0.03 -0.00 -0.03 0.00 0.05 0.00 1b.. b4.1.74 -9.74 9.28 0.00 0.00 0.00 0.15 0.01 -0.16 0.00 1.32 0.00 1b.. b4.8.51 -2.40 -6.59 0.00 0.00 0.00 -0.05 0.00 0.05 0.00 -0.52 0.00 1b.. b4.

-0.82 4.68 -5.75 0.00 0.00 0.00 -0.10 -0.00 0.10 -0.00 -1.89 0.00 1b.. b4.

Circuit Breaker Auxiliary Contacts

Circuit Breaker Spring Stroke Switch

SEL-387 Event Report

BREAKER FAILURE DESIGN PRIOR TO INCIDENT

BREAKER FAILURE DESIGN AFTER INCIDENT

Breaker Failure

Questions?

Thank you.