Top Banner
U.S.-Canada Power System Outage Task Force Interim Report: Causes of the August 14th Blackout in the United States and Canada November 2003
134

Causes of the August 14 Blackout in the US and Canada

Sep 08, 2015

Download

Documents

lisused

Intern Report - Causes of the August 14 Blackout in the US and Canada
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • U.S.-Canada Power System Outage Task Force

    Interim Report:

    Causes of theAugust 14th Blackout

    in theUnited States and Canada

    November 2003

  • U.S.-Canada Power System Outage Task Force

    Interim Report:

    Causes of theAugust 14th Blackout

    in theUnited States and Canada

    November 2003

  • Acknowledgments

    The U.S.-Canada Power System Outage TaskForce would like to acknowledge all the research-ers, analysts, modelers, investigators, planners,designers, and others for their time and effortspent on completing this Interim Report. Theresult is an international coordinated report pro-viding factual reasons as to why the power outageoccurred. This Interim Report was prepared forthe U.S. Secretary of Energy and the Minister ofNatural Resources Canada (NRCAN) under thedirection of Jimmy Glotfelty (USDOE) and Dr.Nawal Kamel and the three working groups: elec-tric system, nuclear, and security.

    Members of the three working groups and investi-gative teams spent an incalculable number ofhours researching in various locations to better

    understand the intricacies of the August 14, 2003,power outage. It was a huge endeavor to achieve,and they did an excellent job providing the factsthough a variety of data requests; analysis of oper-ations, generator and transmission modeling;sequence of events, and root cause analysis. Alongwith countless interviews and a variety of sideinvestigations, the planning and preparation,coordinated internationally, all proved to be aconfidently coordinated effort.

    Thank you for spending countless hours on in-depth research and participating in a report thatwill help the North American public and theworld better understand why and what caused theAugust 14, 2003, blackout. Your efforts are greatlyappreciated! Thank you.

    ii U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

  • ContentsPage

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout iii

    1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

    2. Overview of the North American Electric Power System and Its Reliability Organizations . . . . . . . . 3The North American Power Grid Is One Large, Interconnected Machine . . . . . . . . . . . . . . . . . . . . . . 3Planning and Reliable Operation of the Power Grid Are Technically Demanding . . . . . . . . . . . . . . . . 4Reliability Organizations Oversee Grid Reliability in North America . . . . . . . . . . . . . . . . . . . . . . . . . 7Key Parties in the Pre-Cascade Phase of the August 14 Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    3. Status of the Northeastern Power Grid Before the Blackout Sequence Began . . . . . . . . . . . . . . . . . . . 15Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Electric Demands on August 14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Power Flow Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16System Frequency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Generation Facilities Unavailable on August 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Voltages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Unanticipated Outages of Transmission and Generation on August 14 . . . . . . . . . . . . . . . . . . . . . . . . 19Model-Based Analysis of the State of the Regional Power System at 15:05 EDT,Before the Loss of FEs Harding-Chamberlin 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    4. How and Why the Blackout Began . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Chapter Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Phase 1: A Normal Afternoon Degrades: 12:15 EDT to 14:14 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Phase 2: FEs Computer Failures: 14:14 EDT to 15:59 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Phase 3: Three FE 345-kV Transmission Line Failures and Many Phone Calls:15:05 EDT to 15:57 EDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Phase 4: 138-kV Transmission System Collapse in Northern Ohio: 15:39 to 16:08 EDT . . . . . . . . . . . 43

    5. The Cascade Stage of the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Phase 5: 345-kV Transmission System Cascade in Northern Ohio and South-Central Michigan . . . . 51Phase 6: The Full Cascade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Phase 7: Several Electrical Islands Formed in Northeast U.S. and Canada:16:10:46 EDT to 16:12 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Why the Blackout Stopped Where It Did. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Voltage Collapse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Why the Generators Tripped Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

    6. The August 14 Blackout Compared With Previous Major North American Outages . . . . . . . . . . . . . . 67Incidence and Characteristics of Power System Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Outage Descriptions and Major Causal Factors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Common or Similar Factors Among Major Outages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Comparisons With the August 14 Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

    7. Performance of Nuclear Power Plants Affected by the Blackout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Findings of the U.S. Nuclear Working Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Findings of the Canadian Nuclear Working Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

    8. Physical and Cyber Security Aspects of the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Security Working Group: Mandate and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Cyber Security in the Electricity Sector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Information Collection and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Cyber Timeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Findings to Date. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

  • Appendixes Page

    Tables

    Figures

    iv U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    3.1. Generators Not Available on August 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176.1. Changing Conditions That Affect System Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687.1. U.S. Nuclear Plant Trip Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 807.2. Summary of Shutdown Events for Canadian Nuclear Power Plants . . . . . . . . . . . . . . . . . . . . . . . . 92

    2.1. Basic Structure of the Electric System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2. NERC Interconnections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3. PJM Load Curve, August 18-24, 2003. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.4. Normal and Abnormal Frequency Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.5. NERC Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.6. NERC Regions and Control Areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.7. NERC Reliability Coordinators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.8. Reliability Coordinators and Control Areas in Ohio and Surrounding States. . . . . . . . . . . . . . . . . . 10

    3.1. August 2003 Temperatures in the U.S. Northeast and Eastern Canada. . . . . . . . . . . . . . . . . . . . . . . 163.2. Generation, Demand, and Interregional Power Flows on August 14 at 15:05 EDT . . . . . . . . . . . . . . 163.3. Northeast Central Area Scheduled Imports and Exports: Summer 2003

    Compared to August 14, 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.4. Frequency on August 14, 2003, up to 15:31 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.5. MW and MVAr Output from Eastlake Unit 5 on August 14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    4.1. Timeline: Start of the Blackout in Ohio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224.2. Timeline Phase 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244.3. Eastlake Unit 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.4. Timeline Phase 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.5. FirstEnergy 345-kV Line Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.6. Voltages on FirstEnergys 345-kV Lines: Impacts of Line Trips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.7. Timeline Phase 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.8. Harding-Chamberlin 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364.9. Hanna-Juniper 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

    4.10. Cause of the Hanna-Juniper Line Loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374.11. Star-South Canton 345-kV Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394.12. Cumulative Effects of Sequential Outages on Remaining 345-kV Lines . . . . . . . . . . . . . . . . . . . . . . 404.13. Timeline Phase 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.14. Voltages on FirstEnergys 138-kV Lines: Impacts of Line Trips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434.15. Simulated Effect of Prior Outages on 138-kV Line Loadings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    5.1. Area Affected by the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.2. Sammis-Star 345-kV Line Trip, 16:05:57 EDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515.3. Sammis-Star 345-kV Line Trips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.4. Ohio 345-kV Lines Trip, 16:08:59 to 16:09:07 EDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.5. New York-Ontario Line Flows at Niagara . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.6. Michigan and Ohio Power Plants Trip. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545.7. Transmission and Generation Trips in Michigan, 16:10:36 to 16:10:37 EDT . . . . . . . . . . . . . . . . . . 545.8. Michigan Lines Trip and Ohio Separates from Pennsylvania, 16:10:36 to 16:10:38.6 EDT . . . . . . . 555.9. Active and Reactive Power and Voltage from Ontario into Detroit . . . . . . . . . . . . . . . . . . . . . . . . . . 55

    A. Description of Outage Investigation and Plan for Development of Recommendations. . . . . . . . . . . 101B. List of Electricity Acronyms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107C. Electricity Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109D. Transmittal Letters from the Three Working Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

  • Figures (Continued) Page

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout v

    5.10. Western Pennsylvania Separates from New York, 16:10:39 EDT to 16:10:44 EDT . . . . . . . . . . . . . . 565.11. More Transmission Line and Power Plant Losses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565.12. Northeast Disconnects from Eastern Interconnection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575.13. New York and New England Separate, Multiple Islands Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585.14. Electric Islands Reflected in Frequency Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.15. Area Affected by the Blackout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605.16. Cascade Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625.17. Events at One Large Generator During the Cascade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645.18. Power Plants Tripped During the Cascade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

    6.1. North American Power System Outages, 1984-1997. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

  • 1. Introduction

    On August 14, 2003, large portions of the Midwestand Northeast United States and Ontario, Canada,experienced an electric power blackout. The out-age affected an area with an estimated 50 millionpeople and 61,800 megawatts (MW) of electricload in the states of Ohio, Michigan, Pennsylva-nia, New York, Vermont, Massachusetts, Connect-icut, and New Jersey and the Canadian province ofOntario. The blackout began a few minutes after4:00 pm Eastern Daylight Time (16:00 EDT), andpower was not restored for 2 days in some parts ofthe United States. Parts of Ontario suffered rollingblackouts for more than a week before full powerwas restored.

    On August 15, President George W. Bush andPrime Minister Jean Chrtien directed that a jointU.S.-Canada Power System Outage Task Force beestablished to investigate the causes of the black-out and how to reduce the possibility of futureoutages. They named U.S. Secretary of EnergySpencer Abraham and Herb Dhaliwal, Minister ofNatural Resources, Canada, to chair the joint TaskForce. Three other U.S. representatives and threeother Canadian representatives were named to theTask Force. The U.S. members are Tom Ridge,Secretary of Homeland Security; Pat Wood, Chair-man of the Federal Energy Regulatory Commis-sion; and Nils Diaz, Chairman of the NuclearRegulatory Commission. The Canadian membersare Deputy Prime Minister John Manley, DeputyPrime Minister; Kenneth Vollman, Chairman ofthe National Energy Board; and Linda J. Keen,President and CEO of the Canadian Nuclear SafetyCommission.

    The Task Force divided its work into two phases:

    Phase I: Investigate the outage to determine itscauses and why it was not contained.

    Phase II: Develop recommendations to reducethe possibility of future outages and minimizethe scope of any that occur.

    The Task Force created three Working Groups toassist in the Phase I investigation of the blackoutan Electric System Working Group (ESWG), aNuclear Working Group (NWG), and a SecurityWorking Group (SWG). They were tasked withoverseeing and reviewing investigations of theconditions and events in their respective areas anddetermining whether they may have caused oraffected the blackout. The Working Groups aremade up of State and provincial representatives,Federal employees, and contractors working forthe U.S. and Canadian government agencies repre-sented on the Task Force.

    This document provides an Interim Report, for-warded by the Working Groups, on the findings ofthe Phase I investigation. It presents the facts thatthe bi-national investigation has found regardingthe causes of the blackout on August 14, 2003. TheWorking Groups and their analytic teams are con-fident of the accuracy of these facts and the analy-sis built upon them. This report does not offerspeculations or assumptions not supported byevidence and analysis. Further, it does not attemptto draw broad conclusions or suggest policy rec-ommendations; that task is to be undertaken inPhase II and is beyond the scope of the Phase Iinvestigation.

    This report will now be subject to public reviewand comment. The Working Groups will considerpublic commentary on the Interim Report and willoversee and review any additional analyses andinvestigation that may be required. This reportwill be finalized and made a part of the Task ForceFinal Report, which will also contain recommen-dations on how to minimize the likelihood andscope of future blackouts.

    The Task Force will hold three public forums, orconsultations, in which the public will have theopportunity to comment on this Interim Reportand to present recommendations for consider-ation by the Working Groups and the Task Force.

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 1

  • The public may also submit comments and recom-mendations to the Task Force electronically or bymail. Electronic submissions may be sent to:

    [email protected]

    [email protected].

    Paper submissions may be sent by mail to:

    Dr. Nawal KamelSpecial Adviser to the Deputy MinisterNatural Resources Canada21st Floor580 Booth StreetOttawa, ON K1A 0E4

    and

    Mr. James W. GlotfeltyDirector, Office of Electric Transmissionand Distribution

    U.S. Department of Energy1000 Independence Avenue, S.W.Washington, DC 20585

    This Interim Report is divided into eight chapters,including this introductory chapter:

    Chapter 2 provides an overview of the institu-tional framework for maintaining and ensuringthe reliability of the bulk power system in NorthAmerica, with particular attention to the rolesand responsibilities of several types of reliabil-ity-related organizations.

    Chapter 3 discusses conditions on the regionalpower system before August 14 and on August14 before the events directly related to theblackout began.

    Chapter 4 addresses the causes of the blackout,with particular attention to the evolution ofconditions on the afternoon of August 14, start-ing from normal operating conditions, thengoing into a period of abnormal but still poten-tially manageable conditions, and finally intoan uncontrollable cascading blackout.

    Chapter 5 provides details on the cascade phaseof the blackout.

    Chapter 6 compares the August 14, 2003, black-out with previous major North American poweroutages.

    Chapter 7 examines the performance of thenuclear power plants affected by the August 14outage.

    Chapter 8 addresses issues related to physicaland cyber security associated with the outage.

    This report also includes four appendixes: a de-scription of the investigative process that pro-vided the basis for this report, a list of electricityacronyms, a glossary of electricity terms, and threetransmittal letters pertinent to this report from thethree Working Groups.

    2 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

  • 2. Overview of the North American Electric PowerSystem and Its Reliability Organizations

    The North American Power GridIs One Large, InterconnectedMachine

    The North American electricity system is one ofthe great engineering achievements of the past 100years. This electricity infrastructure representsmore than $1 trillion in asset value, more than200,000 miles (320,000 kilometers) of transmis-sion lines operating at 230,000 volts and greater,950,000 megawatts of generating capability, andnearly 3,500 utility organizations serving wellover 100 million customers and 283 millionpeople.

    Modern society has come to depend on reliableelectricity as an essential resource for nationalsecurity; health and welfare; communications;finance; transportation; food and water supply;heating, cooling, and lighting; computers andelectronics; commercial enterprise; and evenentertainment and leisurein short, nearly allaspects of modern life. Customers have grown toexpect that electricity will almost always be avail-able when needed at the flick of a switch. Mostcustomers have also experienced local outagescaused by a car hitting a power pole, a construc-tion crew accidentally damaging a cable, or a

    lightning storm. What is not expected is the occur-rence of a massive outage on a calm, warm day.Widespread electrical outages, such as the onethat occurred on August 14, 2003, are rare, butthey can happen if multiple reliability safeguardsbreak down.

    Providing reliable electricity is an enormouslycomplex technical challenge, even on the mostroutine of days. It involves real-time assessment,control and coordination of electricity productionat thousands of generators, moving electricityacross an interconnected network of transmissionlines, and ultimately delivering the electricity tomillions of customers by means of a distributionnetwork.

    As shown in Figure 2.1, electricity is produced atlower voltages (10,000 to 25,000 volts) at genera-tors from various fuel sources, such as nuclear,coal, oil, natural gas, hydro power, geothermal,photovoltaic, etc. Some generators are owned bythe same electric utilities that serve the end-usecustomer; some are owned by independent powerproducers (IPPs); and others are owned by cus-tomers themselvesparticularly large industrialcustomers.

    Electricity from generators is stepped up tohigher voltages for transportation in bulk over

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 3

    Figure 2.1. Basic Structure of the Electric System

  • transmission lines. Operating the transmissionlines at high voltage (i.e., 230,000 to 765,000 volts)reduces the losses of electricity from conductorheating and allows power to be shipped economi-cally over long distances. Transmission lines areinterconnected at switching stations and substa-tions to form a network of lines and stations calledthe power grid. Electricity flows through theinterconnected network of transmission linesfrom the generators to the loads in accordancewith the laws of physicsalong paths of leastresistance, in much the same way that waterflows through a network of canals. When thepower arrives near a load center, it is steppeddown to lower voltages for distribution to cus-tomers. The bulk power system is predominantlyan alternating current (AC) system, as opposed toa direct current (DC) system, because of the easeand low cost with which voltages in AC systemscan be converted from one level to another. Somelarger industrial and commercial customers takeservice at intermediate voltage levels (12,000 to115,000 volts), but most residential customerstake their electrical service at 120 and 240 volts.

    While the power system in North America is com-monly referred to as the grid, there are actuallythree distinct power grids or interconnections(Figure 2.2). The Eastern Interconnection includesthe eastern two-thirds of the continental UnitedStates and Canada from Saskatchewan east to theMaritime Provinces. The Western Interconnectionincludes the western third of the continentalUnited States (excluding Alaska), the CanadianProvinces of Alberta and British Columbia, and aportion of Baja California Norte, Mexico. The thirdinterconnection comprises most of the state of

    Texas. The three interconnections are electricallyindependent from each other except for a fewsmall direct current (DC) ties that link them.Within each interconnection, electricity is pro-duced the instant it is used, and flows over virtu-ally all transmission lines from generators toloads.

    The northeastern portion of the Eastern Intercon-nection (about 10 percent of the interconnectionstotal load) was affected by the August 14 blackout.The other two interconnections were notaffected.1

    Planning and Reliable Operationof the Power Grid Are TechnicallyDemanding

    Reliable operation of the power grid is complexand demanding for two fundamental reasons:

    First, electricity flows at the speed of light(186,000 miles per second or 297,600 kilome-ters per second) and is not economicallystorable in large quantities. Therefore electric-ity must be produced the instant it is used.

    Second, the flow of alternating current (AC)electricity cannot be controlled like a liquid orgas by opening or closing a valve in a pipe, orswitched like calls over a long-distance tele-phone network. Electricity flows freely along allavailable paths from the generators to the loadsin accordance with the laws of physicsdivid-ing among all connected flow paths in the net-work, in inverse proportion to the impedance(resistance plus reactance) on each path.

    Maintaining reliability is a complex enterprisethat requires trained and skilled operators, sophis-ticated computers and communications, and care-ful planning and design. The North AmericanElectric Reliability Council (NERC) and its tenRegional Reliability Councils have developed sys-tem operating and planning standards for ensur-ing the reliability of a transmission grid that arebased on seven key concepts:

    Balance power generation and demandcontinuously.

    Balance reactive power supply and demand tomaintain scheduled voltages.

    Monitor flows over transmission lines and otherfacilities to ensure that thermal (heating) limitsare not exceeded.

    4 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 2.2. NERC Interconnections

  • Keep the system in a stable condition.

    Operate the system so that it remains in a reli-able condition even if a contingency occurs,such as the loss of a key generator or transmis-sion facility (the N-1 criterion).

    Plan, design, and maintain the system to oper-ate reliably.

    Prepare for emergencies.

    These seven concepts are explained in more detailbelow.

    1. Balance power generation and demand contin-uously. To enable customers to use as muchelectricity as they wish at any moment, produc-tion by the generators must be scheduled ordispatched to meet constantly changingdemands, typically on an hourly basis, and thenfine-tuned throughout the hour, sometimesthrough the use of automatic generation con-trols to continuously match generation to actualdemand. Demand is somewhat predictable,appearing as a daily demand curvein thesummer, highest during the afternoon and eve-ning and lowest in the middle of the night, andhigher on weekdays when most businesses areopen (Figure 2.3).

    Failure to match generation to demand causesthe frequency of an AC power system (nomi-nally 60 cycles per second or 60 Hertz) toincrease (when generation exceeds demand) ordecrease (when generation is less than demand)(Figure 2.4). Random, small variations in fre-quency are normal, as loads come on and offand generators modify their output to follow thedemand changes. However, large deviations infrequency can cause the rotational speed of gen-erators to fluctuate, leading to vibrations thatcan damage generator turbine blades and otherequipment. Extreme low frequencies can trigger

    automatic under-frequency load shedding,which takes blocks of customers off-line inorder to prevent a total collapse of the electricsystem. As will be seen later in this report, suchan imbalance of generation and demand canalso occur when the system responds to majordisturbances by breaking into separateislands; any such island may have an excessor a shortage of generation, compared todemand within the island.

    2. Balance reactive power supply and demand tomaintain scheduled voltages. Reactive powersources, such as capacitor banks and genera-tors, must be adjusted during the day to main-tain voltages within a secure range pertaining toall system electrical equipment (stations, trans-mission lines, and customer equipment). Mostgenerators have automatic voltage regulatorsthat cause the reactive power output of genera-tors to increase or decrease to control voltages toscheduled levels. Low voltage can cause electricsystem instability or collapse and, at distribu-tion voltages, can cause damage to motors andthe failure of electronic equipment. High volt-ages can exceed the insulation capabilities ofequipment and cause dangerous electric arcs(flashovers).

    3. Monitor flows over transmission lines andother facilities to ensure that thermal (heating)limits are not exceeded. The dynamic interac-tions between generators and loads, combinedwith the fact that electricity flows freely acrossall interconnected circuits, mean that powerflow is ever-changing on transmission and dis-tribution lines. All lines, transformers, andother equipment carrying electricity are heatedby the flow of electricity through them. The

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 5

    Figure 2.3. PJM Load Curve, August 18-24, 2003

    Figure 2.4. Normal and Abnormal FrequencyRanges

  • flow must be limited to avoid overheating anddamaging the equipment. In the case of over-head power lines, heating also causes the metalconductor to stretch or expand and sag closer toground level. Conductor heating is also affectedby ambient temperature, wind, and other fac-tors. Flow on overhead lines must be limited toensure that the line does not sag into obstruc-tions below such as trees or telephone lines, orviolate the minimum safety clearances betweenthe energized lines and other objects. (A shortcircuit or flashoverwhich can start fires ordamage equipmentcan occur if an energizedline gets too close to another object). All electriclines, transformers and other current-carryingdevices are monitored continuously to ensurethat they do not become overloaded or violateother operating constraints. Multiple ratings aretypically used, one for normal conditions and ahigher rating for emergencies. The primarymeans of limiting the flow of power on trans-mission lines is to adjust selectively the outputof generators.

    4. Keep the system in a stable condition. Becausethe electric system is interconnected anddynamic, electrical stability limits must beobserved. Stability problems can develop veryquicklyin just a few cycles (a cycle is 1/60th ofa second)or more slowly, over seconds orminutes. The main concern is to ensure thatgeneration dispatch and the resulting powerflows and voltages are such that the system isstable at all times. (As will be described later inthis report, part of the Eastern Interconnectionbecame unstable on August 14, resulting in acascading outage over a wide area.) Stability

    limits, like thermal limits, are expressed as amaximum amount of electricity that can besafely transferred over transmission lines.

    There are two types of stability limits: (1) Volt-age stability limits are set to ensure that theunplanned loss of a line or generator (whichmay have been providing locally critical reac-tive power support, as described previously)will not cause voltages to fall to dangerouslylow levels. If voltage falls too low, it begins tocollapse uncontrollably, at which point auto-matic relays either shed load or trip generatorsto avoid damage. (2) Power (angle) stability lim-its are set to ensure that a short circuit or anunplanned loss of a line, transformer, or genera-tor will not cause the remaining generators andloads being served to lose synchronism withone another. (Recall that all generators andloads within an interconnection must operate ator very near a common 60 Hz frequency.) Lossof synchronism with the common frequencymeans generators are operating out-of-step withone another. Even modest losses of synchro-nism can result in damage to generation equip-ment. Under extreme losses of synchronism,the grid may break apart into separate electricalislands; each island would begin to maintain itsown frequency, determined by the load/genera-tion balance within the island.

    5. Operate the system so that it remains in a reli-able condition even if a contingency occurs,such as the loss of a key generator or transmis-sion facility (the N minus 1 criterion). Thecentral organizing principle of electricity reli-ability management is to plan for the unex-pected. The unique features of electricity mean

    6 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Local Supplies of Reactive Power Are Essential to Maintaining Voltage Stability

    A generator typically produces some mixture ofactive and reactive power, and the balancebetween them can be adjusted at short notice tomeet changing conditions. Active power, mea-sured in watts, is the form of electricity that pow-ers equipment. Reactive power, a characteristicof AC systems, is measured in volt-amperes reac-tive (VAr), and is the energy supplied to create orbe stored in electric or magnetic fields in andaround electrical equipment. Reactive power isparticularly important for equipment that relieson magnetic fields for the production of inducedelectric currents (e.g., motors, transformers,pumps, and air conditioning.) Transmission

    lines both consume and produce reactive power.At light loads they are net producers, and atheavy loads, they are heavy consumers. Reactivepower consumption by these facilities or devicestends to depress transmission voltage, while itsproduction (by generators) or injection (fromstorage devices such as capacitors) tends to sup-port voltage. Reactive power can be transmittedonly over relatively short distances, and thusmust be supplied as needed from nearby genera-tors or capacitor banks. If reactive power cannotbe supplied promptly and in sufficient quantity,voltages decay, and in extreme cases a voltagecollapse may result.

  • that problems, when they arise, can spread andescalate very quickly if proper safeguards arenot in place. Accordingly, through years ofexperience, the industry has developed asequence of defensive strategies for maintainingreliability based on the assumption that equip-ment can and will fail unexpectedly uponoccasion.

    This principle is expressed by the requirementthat the system must be operated at all times toensure that it will remain in a secure condition(generally within emergency ratings for currentand voltage and within established stabilitylimits) following the loss of the most importantgenerator or transmission facility (a worst sin-gle contingency). This is called the N-1 crite-rion. In other words, because a generator orline trip can occur at any time from random fail-ure, the power system must be operated in apreventive mode so that the loss of the mostimportant generator or transmission facilitydoes not jeopardize the remaining facilities inthe system by causing them to exceed theiremergency ratings or stability limits, whichcould lead to a cascading outage.

    Further, when a contingency does occur, theoperators are required to identify and assessimmediately the new worst contingencies,given the changed conditions, and promptlymake any adjustments needed to ensure that ifone of them were to occur, the system wouldstill remain operational and safe. NERC operat-ing policy requires that the system be restoredas soon as practical but within no more than 30minutes to compliance with normal limits, andto a condition where it can once again with-stand the next-worst single contingency with-out violating thermal, voltage, or stabilitylimits. A few areas of the grid are operated towithstand the concurrent loss of two or morefacilities (i.e., N-2). This may be done, forexample, as an added safety measure to protecta densely populated metropolitan area or whenlines share a common structure and could beaffected by a common failure mode, e.g., a sin-gle lightning strike.

    6. Plan, design, and maintain the system to oper-ate reliably. Reliable power system operationrequires far more than monitoring and control-ling the system in real-time. Thorough plan-ning, design, maintenance, and analysis arerequired to ensure that the system can be oper-ated reliably and within safe limits. Short-term

    planning addresses day-ahead and week-aheadoperations planning; long-term planningfocuses on providing adequate generationresources and transmission capacity to ensurethat in the future the system will be able towithstand severe contingencies without experi-encing widespread, uncontrolled cascadingoutages.

    A utility that serves retail customers must esti-mate future loads and, in some cases, arrangefor adequate sources of supplies and plan ade-quate transmission and distribution infrastruc-ture. NERC planning standards identify a rangeof possible contingencies and set correspondingexpectations for system performance under sev-eral categories of possible events. Three catego-ries represent the more probable types of eventsthat the system must be planned to withstand.A fourth category represents extreme eventsthat may involve substantial loss of customerload and generation in a widespread area. NERCplanning standards also address requirementsfor voltage support and reactive power, distur-bance monitoring, facility ratings, system mod-eling and data requirements, system protectionand control, and system restoration.

    7. Prepare for emergencies. System operators arerequired to take the steps described above toplan and operate a reliable power system, butemergencies can still occur because of externalfactors such as severe weather, operator error,or equipment failures that exceed planning,design, or operating criteria. For these rareevents, the operating entity is required to haveemergency procedures covering a crediblerange of emergency scenarios. Operators mustbe trained to recognize and take effective actionin response to these emergencies. To deal with asystem emergency that results in a blackout,such as the one that occurred on August 14,2003, there must be procedures and capabilitiesto use black start generators (capable ofrestarting with no external power source) and tocoordinate operations in order to restore thesystem as quickly as possible to a normal andreliable condition.

    Reliability Organizations OverseeGrid Reliability in North America

    NERC is a non-governmental entity whose mis-sion is to ensure that the bulk electric system inNorth America is reliable, adequate and secure.

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 7

  • The organization was established in 1968, as aresult of the Northeast blackout in 1965. Since itsinception, NERC has operated as a voluntary orga-nization, relying on reciprocity, peer pressure andthe mutual self-interest of all those involved toensure compliance with reliability requirements.An independent board governs NERC.

    To fulfill its mission, NERC:

    Sets standards for the reliable operation andplanning of the bulk electric system.

    Monitors and assesses compliance with stan-dards for bulk electric system reliability.

    Provides education and training resources topromote bulk electric system reliability.

    Assesses, analyzes and reports on bulk electricsystem adequacy and performance.

    Coordinates with Regional Reliability Councilsand other organizations.

    Coordinates the provision of applications(tools), data and services necessary to supportthe reliable operation and planning of the bulkelectric system.

    Certifies reliability service organizations andpersonnel.

    Coordinates critical infrastructure protection ofthe bulk electric system.

    Enables the reliable operation of the intercon-nected bulk electric system by facilitating infor-mation exchange and coordination amongreliability service organizations.

    Recent changes in the electricity industry havealtered many of the traditional mechanisms,incentives and responsibilities of the entitiesinvolved in ensuring reliability, to the point thatthe voluntary system of compliance with reliabil-ity standards is generally recognized as not ade-quate to current needs.2 NERC and many otherelectricity organizations support the developmentof a new mandatory system of reliability standardsand compliance, backstopped in the United Statesby the Federal Energy Regulatory Commission.This will require federal legislation in the UnitedStates to provide for the creation of a new electricreliability organization with the statutory author-ity to enforce compliance with reliability stan-dards among all market participants. Appropriategovernment entities in Canada and Mexico areprepared to take similar action, and some havealready done so. In the meantime, NERC encour-ages compliance with its reliability standardsthrough an agreement with its members.

    NERCs members are ten Regional ReliabilityCouncils. (See Figure 2.5 for a map showing thelocations and boundaries of the regional councils.)The regional councils and NERC have openedtheir membership to include all segments of theelectric industry: investor-owned utilities; federalpower agencies; rural electric cooperatives; state,municipal and provincial utilities; independentpower producers; power marketers; and end-usecustomers. Collectively, the members of the NERCregions account for virtually all the electricity sup-plied in the United States, Canada, and a portionof Baja California Norte, Mexico. The ten regionalcouncils jointly fund NERC and adapt NERC stan-dards to meet the needs of their regions. TheAugust 14 blackout affected three NERC regionalreliability councilsEast Central Area ReliabilityCoordination Agreement (ECAR), Mid-AtlanticArea Council (MAAC), and Northeast Power Coor-dinating Council (NPCC).

    Control areas are the primary operational enti-ties that are subject to NERC and regional councilstandards for reliability. A control area is a geo-graphic area within which a single entity, Inde-pendent System Operator (ISO), or RegionalTransmission Organization (RTO) balances gener-ation and loads in real time to maintain reliableoperation. Control areas are linked with eachother through transmission interconnection tielines. Control area operators control generationdirectly to maintain their electricity interchangeschedules with other control areas. They alsooperate collectively to support the reliability of

    8 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 2.5. NERC Regions

  • their interconnection. As shown in Figure 2.6,there are approximately 140 control areas in NorthAmerica. The control area dispatch centers havesophisticated monitoring and control systems andare staffed 24 hours per day, 365 days per year.

    Traditionally, control areas were defined by utilityservice area boundaries and operations werelargely managed by vertically integrated utilitiesthat owned and operated generation, transmis-sion, and distribution. While that is still true insome areas, there has been significant restructur-ing of operating functions and some consolidationof control areas into regional operating entities.Utility industry restructuring has led to anunbundling of generation, transmission and dis-tribution activities such that the ownership andoperation of these assets have been separatedeither functionally or through the formation ofindependent entities called Independent SystemOperators (ISOs) and Regional TransmissionOrganizations (RTOs).

    ISOs and RTOs in the United States have beenauthorized by FERC to implement aspects of theEnergy Policy Act of 1992 and subsequent FERCpolicy directives.

    The primary functions of ISOs and RTOs are tomanage in real time and on a day-ahead basisthe reliability of the bulk power system and theoperation of wholesale electricity marketswithin their footprint.

    ISOs and RTOs do not own transmission assets;they operate or direct the operation of assetsowned by their members.

    ISOs and RTOs may be control areas them-selves, or they may encompass more than onecontrol area.

    ISOs and RTOs may also be NERC ReliabilityCoordinators, as described below.

    Five RTOs/ISOs are within the area directlyaffected by the August 14 blackout. They are:

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 9

    Figure 2.6. NERC Regions and Control Areas

  • Midwest Independent System Operator (MISO)

    PJM Interconnection (PJM)

    New York Independent System Operator(NYISO)

    New England Independent System Operator(ISO-NE)

    Ontario Independent Market Operator (IMO)

    Reliability coordinators provide reliability over-sight over a wide region. They prepare reliabilityassessments, provide a wide-area view of reliabil-ity, and coordinate emergency operations in realtime for one or more control areas. They do notparticipate in the wholesale or retail market func-tions. There are currently 18 reliability coordina-tors in North America. Figure 2.7 shows thelocations and boundaries of their respective areas.

    Key Parties in the Pre-CascadePhase of the August 14 Blackout

    The initiating events of the blackout involved twocontrol areasFirstEnergy (FE) and AmericanElectric Power (AEP)and their respective reli-ability coordinators, MISO and PJM (see Figures2.7 and 2.8). These organizations and their reli-ability responsibilities are described briefly in thisfinal subsection.

    1. FirstEnergy operates a control area in north-ern Ohio. FirstEnergy (FE) consists of sevenelectric utility operating companies. Four ofthese companies, Ohio Edison, Toledo Edison,The Illuminating Company, and Penn Power,operate in the NERC ECAR region, with MISO

    serving as their reliability coordinator. Thesefour companies now operate as one integratedcontrol area managed by FE.3

    2. American Electric Power (AEP) operates a con-trol area in Ohio just south of FE. AEP is both atransmission operator and a control areaoperator.

    3. Midwest Independent System Operator(MISO) is the reliability coordinator forFirstEnergy. The Midwest Independent SystemOperator (MISO) is the reliability coordinatorfor a region of more than one million squaremiles, stretching from Manitoba, Canada in thenorth to Kentucky in the south, from Montanain the west to western Pennsylvania in the east.Reliability coordination is provided by twooffices, one in Minnesota, and the other at theMISO headquarters in Indiana. Overall, MISOprovides reliability coordination for 37 controlareas, most of which are members of MISO.

    4. PJM is AEPs reliability coordinator. PJM is oneof the original ISOs formed after FERC orders888 and 889, but was established as a regionalpower pool in 1935. PJM recently expanded itsfootprint to include control areas and transmis-sion operators within MAIN and ECAR (PJM-West). It performs its duties as a reliability coor-dinator in different ways, depending on thecontrol areas involved. For PJM-East, it isboth the control area and reliability coordinatorfor ten utilities, whose transmission systemsspan the Mid-Atlantic region of New Jersey,most of Pennsylvania, Delaware, Maryland,West Virginia, Ohio, Virginia, and the District ofColumbia. The PJM-West facility has the reli-ability coordinator desk for five control areas(AEP, Commonwealth Edison, Duquesne Light,

    10 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 2.7. NERC Reliability Coordinators

    Figure 2.8. Reliability Coordinators and ControlAreas in Ohio and Surrounding States

  • Dayton Power and Light, and Ohio Valley Elec-tric Cooperative) and three generation-onlycontrol areas (Duke Energys WashingtonCounty (Ohio) facility, Dukes LawrenceCounty/Hanging Rock (Ohio) facility, and Alle-gheny Energys Buchanan (West Virginia)facility.

    Reliability Responsibilities of ControlArea Operators and ReliabilityCoordinators

    1. Control area operators have primary responsi-bility for reliability. Their most importantresponsibilities, in the context of this report,are:

    N-1 criterion. NERC Operating Policy 2.ATransmission Operations:

    All CONTROL AREAS shall operate so thatinstability, uncontrolled separation, or cas-cading outages will not occur as a result ofthe most severe single contingency.

    Emergency preparedness and emergencyresponse. NERC Operating Policy 5Emer-gency Operations, General Criteria:

    Each system and CONTROL AREA shallpromptly take appropriate action to relieveany abnormal conditions, which jeopardizereliable Interconnection operation.

    Each system, CONTROL AREA, and Regionshall establish a program of manual and auto-matic load shedding which is designed toarrest frequency or voltage decays that couldresult in an uncontrolled failure of compo-nents of the interconnection.

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 11

    Institutional Complexities and Reliability in the Midwest

    The institutional arrangements for reliability inthe Midwest are much more complex than theyare in the Northeast-the areas covered by theNortheast Power Coordinating Council (NPCC)and the Mid-Atlantic Area Council (MAAC).There are two principal reasons for this complex-ity. One is that in NPCC and MAAC, the inde-pendent system operator (ISO) also serves as thesingle control area operator for the individualmember systems. In comparison, MISO providesreliability coordination for 35 control areas in theECAR, MAIN, and MAPP regions and 2 others inthe SPP region, and PJM provides reliability coor-dination for 8 control areas in the ECAR andMAIN regions (plus one in MAAC). (See tablebelow.) This results in 18 control-area-to-control-area interfaces across the PJM/MISO reli-ability coordinator boundary.

    The other is that MISO has less reliability-relatedauthority over its control area members than PJMhas over its members. Arguably, this lack ofauthority makes day-to-day reliability operationsmore challenging. Note, however, that (1) FERCsauthority to require that MISO have greaterauthority over its members is limited; and (2)before approving MISO, FERC asked NERC for aformal assessment of whether reliability could bemaintained under the arrangements proposed byMISO and PJM. After reviewing proposed plansfor reliability coordination within and betweenPJM and MISO, NERC replied affirmatively butprovisionally. NERC conducted audits inNovember and December 2002 of the MISO andPJM reliability plans, and some of the recommen-dations of the audit teams are still beingaddressed. The adequacy of the plans andwhether the plans were being implemented aswritten are factors in the NERCs ongoinginvestigation.

    Reliability Coordinator (RC)

    ControlAreas inRC Area

    Regional ReliabilityCouncils Affected and

    Number of Control Areas Control Areas of Interest in RC Area

    MISO 37 ECAR (12), MAIN (9),MAPP (14), SPP (2)

    FE, Cinergy,Michigan Electric Coordinated System

    PJM 9 MAAC (1), ECAR (7),MAIN (1)

    PJM, AEP,Dayton Power & Light

    ISO New England 2 NPCC (2) ISONE, Maritimes

    New York ISO 1 NPCC (1) NYISO

    Ontario Independent Market Operator 1 NPCC (1) IMO

    Trans-Energie 1 NPCC (1) Hydro Qubec

  • NERC Operating Policy 5.ACoordinationwith Other Systems:

    A system, CONTROL AREA, or pool that isexperiencing or anticipating an operatingemergency shall communicate its currentand future status to neighboring systems,CONTROL AREAS, or pools and throughout theinterconnection. A system shall informother systems whenever the systemscondition is burdening other systems orreducing the reliability of the Interconnec-tion . [or whenever] the systems line load-ings and voltage/reactive levels are such thata single contingency could threaten the reli-ability of the Interconnection.

    NERC Operating Policy 5.CTransmissionSystem Relief:

    Action to correct an OPERATING SECURITYLIMIT violation shall not impose unaccept-able stress on internal generation or transmis-sion equipment, reduce system reliabilitybeyond acceptable limits, or unduly imposevoltage or reactive burdens on neighboringsystems. If all other means fail, correctiveaction may require load reduction.

    Operating personnel and training: NERC Oper-ating Policy 8.BTraining:

    Each OPERATING AUTHORITY should period-ically practice simulated emergencies. The

    scenarios included in practice situationsshould represent a variety of operating condi-tions and emergencies.

    2. Reliability Coordinators such as MISO andPJM are expected to comply with all aspects ofNERC Operating Policies, especially Policy 9,Reliability Coordinator Procedures, and itsappendices. Key requirements include:

    NERC Operating Policy 9, Criteria for Reliabil-ity Coordinators, 5.2:

    Have detailed monitoring capability of theRELIABILITY AREA and sufficient monitoringcapability of the surrounding RELIABILITYAREAS to ensure potential security violationsare identified.

    NERC Operating Policy 9, Functions of Reliabil-ity Coordinators, 1.7:

    Monitor the parameters that may have sig-nificant impacts within the RELIABILITY AREAand with neighboring RELIABILITY AREASwith respect to sharing with otherRELIABILITY COORDINATORS any informationregarding potential, expected, or actual criti-cal operating conditions that could nega-tively impact other RELIABILITY AREAS. TheRELIABILITY COORDINATOR will coordinatewith other RELIABILITY COORDINATORS andCONTROL AREAS as needed to develop appro-priate plans to mitigate negative impacts ofpotential, expected, or actual critical operat-ing conditions.

    NERC Operating Policy 9, Functions of Reliabil-ity Coordinators, 6:

    Conduct security assessment and monitor-ing programs to assess contingency situa-tions. Assessments shall be made in real timeand for the operations planning horizon atthe CONTROL AREA level with any identifiedproblems reported to the RELIABILITY CO-ORDINATOR. The RELIABILITY COORDINATORis to ensure that CONTROL AREA, RELIABILITYAREA, and regional boundaries are suffi-ciently modeled to capture any problemscrossing such boundaries.

    Endnotes

    12 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    What Constitutes an Operating Emergency?

    An operating emergency is an unsustainablecondition that cannot be resolved using theresources normally available. The NERC Oper-ating Manual defines a capacity emergency aswhen a systems or pools operating generationcapacity, plus firm purchases from other sys-tems, to the extent available or limited by trans-fer capability, is inadequate to meet its demandplus its regulating requirements. It defines anenergy emergency as when a load-servingentity has exhausted all other options and canno longer provide its customers expectedenergy requirements. A transmission emer-gency exists when the systems line loadingsand voltage/ reactive levels are such that a singlecontingency could threaten the reliability of theInterconnection. Control room operators anddispatchers are given substantial latitude todetermine when to declare an emergency. (Seepage 42 in Chapter 4 for more detail.)

    1The province of Quebec, although considered a part of theEastern Interconnection, is connected to the rest of the East-ern Interconnection primarily by DC ties. In this instance, theDC ties acted as buffers between portions of the Eastern Inter-connection; transient disturbances propagate through themless readily. Therefore, the electricity system in Quebec wasnot affected by the outage, except for a small portion of the

  • U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 13

    provinces load that is directly connected to Ontario by ACtransmission lines. (Although DC ties can act as a bufferbetween systems, the tradeoff is that they do not allow instan-taneous generation support following the unanticipated lossof a generating unit.)2See, for example, Maintaining Reliability in a CompetitiveElectric Industry (1998), a report to the U.S. Secretary ofEnergy by the Task Force on Electric Systems Reliability;National Energy Policy (2001), a report to the President of the

    United States by the National Energy Policy DevelopmentGroup, p. 7-6; and National Transmission Grid Study (2002),U.S. Dept. of Energy, pp. 46-48.3The remaining three FE companies, Penelec, Met-Ed, andJersey Central Power & Light, are in the NERC MAAC regionand have PJM as their reliability coordinator. The focus of thisreport is on the portion of FE in ECAR reliability region andwithin the MISO reliability coordinator footprint.

  • 3. Status of the Northeastern Power GridBefore the Blackout Sequence Began

    Summary

    This chapter reviews the state of the northeast por-tion of the Eastern Interconnection during thedays prior to August 14, 2003 and up to 15:05 EDTon August 14 to determine whether conditions atthat time were in some way unusual and mighthave contributed to the initiation of the blackout.The Task Forces investigators found that at 15:05EDT, immediately before the tripping (automaticshutdown) of FirstEnergys (FE) Harding-Cham-berlin 345-kV transmission line, the system wasable to be operated reliably following the occur-rence of any of more than 800 contingencies,including the loss of the Harding-Chamberlin line.At that point the system was being operated near(but still within) prescribed limits and in compli-ance with NERCs operating policies.

    Determining that the system was in a reliableoperational state at that time is extremely signifi-cant for understanding the causes of the blackout.It means that none of the electrical conditions onthe system before 15:05 EDT was a direct cause ofthe blackout. This eliminates a number of possiblecauses of the blackout, whether individually or incombination with one another, such as:

    High power flows to Canada

    System frequency variations

    Low voltages earlier in the day or on prior days

    Low reactive power output from IPPs

    Unavailability of individual generators or trans-mission lines.

    It is important to emphasize that establishingwhether conditions were normal or unusual priorto and on August 14 has no direct bearing on theresponsibilities and actions expected of the orga-nizations and operators who are charged withensuring power system reliability. As described inChapter 2, the electricity industry has developedand codified a set of mutually reinforcing reliabil-ity standards and practices to ensure that system

    operators are prepared for the unexpected. Thebasic assumption underlying these standards andpractices is that power system elements will failor become unavailable in unpredictable ways.Sound reliability management is designed toensure that safe operation of the system will con-tinue following the unexpected loss of any keyelement (such as a major generator or key trans-mission facility). These practices have beendesigned to maintain a functional and reliablegrid, regardless of whether actual operatingconditions are normal. It is a basic principle ofreliability management that operators must oper-ate the system they have in front of themunconditionally.

    In terms of day-ahead planning, this means evalu-ating and if necessary adjusting the plannedgeneration pattern (scheduled electricity transac-tions) to change the transmission flows, so that if akey facility were lost, the operators would still beable to readjust the remaining system and operatewithin safe limits. In terms of real-time operations,this means that the system should be operated atall times so as to be able to withstand the loss ofany single facility and still remain within the sys-tems thermal, voltage, and stability limits. If afacility is lost unexpectedly, the system operatorsmust determine whether to make operationalchanges to ensure that the remaining system isable to withstand the loss of yet another key ele-ment and still remain able to operate within safelimits. This includes adjusting generator outputs,curtailing electricity transactions, and if neces-sary, shedding interruptible and firm customerloadi.e., cutting some customers off tempo-rarily, and in the right locations, to reduce elec-tricity demand to a level that matches what thesystem is then able to deliver safely.

    Electric Demands on August 14

    Temperatures on August 14 were above normalthroughout the northeast region of the United

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 15

  • States and in eastern Canada. As a result, electric-ity demands were high due to high air condition-ing loads typical of warm days in August, thoughnot unusually so. System operators had success-fully managed higher demands both earlier in thesummer and in previous years. Recorded peakelectric demands throughout the region on August14 were below peak demands recorded earlier inthe summer of 2003 (Figure 3.1).

    Power Flow Patterns

    On August 14, the flow of power through theECAR region was heavy as a result of large trans-fers of power from the south (Tennessee, Ken-tucky, Missouri, etc.) and west (Wisconsin,Minnesota, Illinois, etc.) to the north (Ohio, Mich-igan, and Ontario) and east (New York). The desti-nations for much of the power were northernOhio, Michigan, PJM, and Ontario (Figure 3.2).

    While heavy, these transfers were not beyond pre-vious levels or in directions not seen before(Figure 3.3). The level of imports into Ontario onAugust 14 was high but not unusual, and wellwithin IMOs import capability. Ontarios IMO is afrequent importer of power, depending on theavailability and price of generation withinOntario. IMO had imported similar and higheramounts of power several times during the sum-mers of 2002 and 2003.

    System Frequency

    Although system frequency on the Eastern Inter-connection was somewhat more variable on

    16 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 3.1. August 2003 Temperatures in the U.S.Northeast and Eastern Canada

    PJM

    New YorkISO

    MichiganElectric

    CoordinatedSystems

    AmericanElectric PowerCompany

    Independent Electricity

    Market Operator

    DuquesneLight

    AlleghenyPower

    Systems

    Dayton Power & Light

    NorthernIndiana PublicService

    First Energy

    Load 12,080 MWGeneration 9,383 MW

    Load 61,747 MWGeneration 60,858 MW

    Load 27,672 MWGeneration 27,376 MW

    Load 23,215 MWGeneration 21,986 MW

    Load 19,155 MWGeneration 18,221 MW

    Load 27,450 MWGeneration 28,225 MW

    1434 MW

    363 MVAR

    380 MVAR

    381 MW

    962 MW

    142 MVAR

    188 MVAR120 MW

    2766 MW

    1733 MW

    168 MVAR

    341 MW

    114 MVAR

    295 MVAR

    312 MW

    149 MVAR

    Figure 3.2. Generation, Demand, and Interregional Power Flows on August 14 at 15:05 EDT

  • August 14 prior to 15:05 EDT compared withrecent history, it was well within the bounds ofsafe operating practices as outlined in NERC oper-ating policies. As a result, system frequency varia-tion was not a cause of the initiation of theblackout. But once the cascade was initiated, thelarge frequency swings that were induced became

    a principal means by which the blackout spreadacross a wide area (Figure 3.4).

    Assuming stable conditions, the system frequencyis the same across an interconnected grid at anyparticular moment. System frequency will varyfrom moment to moment, however, depending onthe second-to-second balance between aggregategeneration and aggregate demand across the inter-connection. System frequency is monitored on acontinuous basis.

    Generation Facilities Unavailableon August 14

    Several key generators in the region were out ofservice going into the day of August 14. On anygiven day, some generation and transmissioncapacity is unavailable; some facilities are out forroutine maintenance, and others have been forcedout by an unanticipated breakdown and requirerepairs. August 14, 2003, was no exception (Table3.1).

    The generating units that were not available onAugust 14 provide real and reactive power directlyto the Cleveland, Toledo, and Detroit areas. Understandard practice, system operators take intoaccount the unavailability of such units and any

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 17

    Frequency Management

    Each control area is responsible for maintaininga balance between its generation and demand. Ifpersistent under-frequency occurs, at least onecontrol area somewhere is leaning on the grid,meaning that it is taking unscheduled electric-ity from the grid, which both depresses systemfrequency and creates unscheduled powerflows. In practice, minor deviations at the con-trol area level are routine; it is very difficult tomaintain an exact balance between generationand demand. Accordingly, NERC has estab-lished operating rules that specify maximumpermissible deviations, and focus on prohibit-ing persistent deviations, but not instantaneousones. NERC monitors the performance of con-trol areas through specific measures of controlperformance that gauge how accurately eachcontrol area matches its load and generation.

    Figure 3.3. Northeast Central Area ScheduledImports and Exports: Summer 2003 Compared toAugust 14, 2003

    Note: Area covered includes ECAR, PJM, Ontario, and NewYork, without imports from the Maritime Provinces, ISO-NewEngland, or Hydro-Quebec.

    Table 3.1. Generators Not Available on August 14Generator Rating Reason

    Davis-Besse Nuclear Unit 750 MW Prolonged NRC-ordered outage beginning on 3/22/02

    Eastlake Unit 4 238 MW Forced outage on 8/13/03

    Monroe Unit 1 817 MW Planned outage, taken out of service on 8/8/03

    Cook Nuclear Unit 2 1,060 MW Outage began on 8/13/03

    Figure 3.4. Frequency on August 14, 2003,up to 15:31 EDT

  • transmission facilities known to be out of servicein the day-ahead planning studies they perform todetermine the condition of the system for the nextday. Knowing the status of key facilities also helpsoperators determine in advance the safe electricitytransfer levels for the coming day.

    MISOs day-ahead planning studies for August 14took these generator outages and known transmis-sion outages into account and determined that theregional system could still be operated safely. Theunavailability of these generation units and trans-mission facilities did not cause the blackout.

    Voltages

    During the days before August 14 and throughoutthe morning and mid-day on August 14, voltageswere depressed in a variety of locations in north-ern Ohio because of high air conditioning demandand other loads, and power transfers into andacross the region. (Unlike frequency, which isconstant across the interconnection, voltage variesby location, and operators monitor voltages con-tinuously at key locations across their systems.)However, actual measured voltage levels at keypoints on FEs transmission system on the morn-ing of August 14 and up to 15:05 EDT were withinthe range previously specified by FE as acceptable.Note, however, that many control areas in theEastern Interconnection have set their acceptablevoltage bands at levels higher than that used

    by FE. For example, AEPs minimum acceptablevoltage level is 95% of a lines nominal rating, ascompared to FEs 92%.1

    Voltage management is especially challenging onhot summer days because of high air conditioningrequirements, other electricity demand, and hightransfers of power for economic reasons, all ofwhich increase the need for reactive power. Oper-ators address these challenges through long-termplanning, day-ahead planning, and real-timeadjustments to operating equipment. On August14, for example, PJM implemented routine voltagemanagement procedures developed for heavy loadconditions. FE also began preparations early in theafternoon of August 14, requesting capacitors tobe restored to service2 and additional voltage sup-port from generators.3 Such actions were typicalof many system operators that day as well as onother days with high electric demand. As the dayprogressed, operators across the region took addi-tional actions, such as increasing plants reactivepower output, plant redispatch, transformer tapchanges, and increased use of capacitors torespond to changing voltage conditions.

    The power flow data for northern Ohio on August14 just before the Harding-Chamberlin line trip-ped at 15:05 EDT (Figure 3.2) show that FEs loadwas approximately 12,080 MW. FE was importingabout 2,575 MW, 21% of its total system needs,and generating the remainder. With this high levelof imports and high air conditioning loads in the

    18 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Independent Power Producers and Reactive Power

    Independent power producers (IPPs) are powerplants that are not owned by utilities. They oper-ate according to market opportunities and theircontractual agreements with utilities, and may ormay not be under the direct control of grid opera-tors. An IPPs reactive power obligations aredetermined by the terms of its contractual inter-connection agreement with the local transmis-sion owner. Under routine conditions, some IPPsprovide limited reactive power because they arenot required or paid to produce it; they are onlypaid to produce active power. (Generation ofreactive power by a generator can require scalingback generation of active power.) Some con-tracts, however, compensate IPPs for following avoltage schedule set by the system operator,which requires the IPP to vary its output of reac-tive power as system conditions change. Further,contracts typically require increased reactivepower production from IPPs when it is requested

    by the control area operator during times of a sys-tem emergency. In some contracts, provisionscall for the payment of opportunity costs to IPPswhen they are called on for reactive power (i.e.,they are paid the value of foregone active powerproduction).

    Thus, the suggestion that IPPs may have contrib-uted to the difficulties of reliability managementon August 14 because they dont provide reactivepower is misplaced. What the IPP is required toproduce is governed by contractual arrange-ments, which usually include provisions for con-tributions to reliability, particularly duringsystem emergencies. More importantly, it is theresponsibility of system planners and operators,not IPPs, to plan for reactive power requirementsand make any short-term arrangements neededto ensure that adequate reactive power resourceswill be available.

  • metropolitan areas around the southern end ofLake Erie, FEs system reactive power needs rosefurther. Investigation team modeling indicatesthat at 15:00 EDT, with Eastlake 5 out of service,FE was a net importer of about 132 MVAr. Asignificant amount of power also was flowingthrough northern Ohio on its way to Michigan andOntario (Figure 3.2). The net effect of this flow pat-tern and load composition was to depress voltagesin northern Ohio.

    Unanticipated Outages ofTransmission and Generation

    on August 14

    Three significant unplanned outages occurred inthe Ohio area on August 14 prior to 15:05 EDT.Around noon, several Cinergy transmission linesin south-central Indiana tripped; at 13:31 EDT,FEs Eastlake 5 generating unit along the south-western shore of Lake Erie tripped; at 14:02 EDT, aDayton Power and Light (DPL) line, the Stuart-Atlanta 345-kV line in southern Ohio, tripped.

    Transmission lines on the Cinergy 345-, 230-,and 138-kV systems experienced a series of out-ages starting at 12:08 EDT and remained out ofservice during the entire blackout. The loss ofthese lines caused significant voltage andloading problems in the Cinergy area. Cinergymade generation changes, and MISO operatorsresponded by implementing transmission load

    relief (TLR) procedures to control flows on thetransmission system in south-central Indiana.System modeling by the investigation team (seedetails below, page 20) showed that the loss ofthese lines was not electrically related to subse-quent events in northern Ohio that led to theblackout.

    The DPL Stuart-Atlanta 345-kV line, linkingDPL to AEP and monitored by the PJM reliabil-ity coordinator, tripped at 14:02 EDT. This wasthe result of a tree contact, and the lineremained out of service during the entire black-out. As explained below, system modeling bythe investigation team has shown that this out-age was not a cause of the subsequent events innorthern Ohio that led to the blackout. How-ever, since the line was not in MISOs footprint,MISO operators did not monitor the status ofthis line, and did not know that it had gone outof service. This led to a data mismatch that pre-vented MISOs state estimator (a key monitoringtool) from producing usable results later in theday at a time when system conditions in FEscontrol area were deteriorating (see detailsbelow, page 27).

    Eastlake Unit 5 is a 597-MW generating unitlocated just west of Cleveland near Lake Erie. Itis a major source of reactive power support forthe Cleveland area. It tripped at 13:31. Thecause of the trip was that as the Eastlake 5 oper-ator sought to increase the units reactive power

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 19

    Power Flow Simulation of Pre-Cascade Conditions

    The bulk power system has no memory. It doesnot matter if frequencies or voltage were unusualan hour, a day, or a month earlier. What mattersfor reliability are loadings on facilities, voltages,and system frequency at a given moment and thecollective capability of these system componentsat that same moment to withstand a contingencywithout exceeding thermal, voltage, or stabilitylimits.

    Power system engineers use a technique calledpower flow simulation to reproduce known oper-ating conditions at a specific time by calibratingan initial simulation to observed voltages andline flows. The calibrated simulation can then beused to answer a series of what if questions todetermine whether the system was in a safe oper-ating state at that time. The what if questionsconsist of systematically simulating outages byremoving key elements (e.g., generators or trans-

    mission lines) one by one and reassessing thesystem each time to determine whether line orvoltage limits would be exceeded. If a limit isexceeded, the system is not in a secure state. Asdescribed in Chapter 2, NERC operating policiesrequire operators, upon finding that their systemis not in a reliable state, to take immediateactions to restore the system to a reliable state assoon as possible and within a maximum of 30minutes.

    To analyze the evolution of the system on theafternoon of August 14, this process was fol-lowed to model several points in time, corre-sponding to key transmission line trips. For eachpoint, three solutions were obtained: (1) condi-tions immediately before a facility tripped off; (2)conditions immediately after the trip; and (3)conditions created by any automatic actionstaken following the trip.

  • output (Figure 3.5), the units protection systemdetected a failure and tripped the unit off-line.The loss of the Eastlake 5 unit did not put thegrid into an unreliable statei.e., it was stillable to withstand safely another contingency.However, the loss of the unit required FE toimport additional power to make up for the lossof the units output (540 MW), made voltagemanagement in northern Ohio more challeng-ing, and gave FE operators less flexibility inoperating their system (see details below, page27).

    Model-Based Analysis of the Stateof the Regional Power System at

    15:05 EDT, Before the Loss of FEsHarding-Chamberlin 345-kV Line

    As the first step in modeling the evolution of theAugust 14 blackout, the investigative team estab-lished a base case by creating a power flow simula-tion for the entire Eastern Interconnection andbenchmarking it to recorded system conditions at15:05 EDT on August 14. The team started with aprojected summer 2003 power flow case devel-oped in the spring of 2003 by the Regional Reli-ability Councils to establish guidelines for safeoperations for the coming summer. The level ofdetail involved in this region-wide study farexceeds that normally considered by individualcontrol areas and reliability coordinators. It con-sists of a detailed representation of more than43,000 buses (points at which lines, transformers,and/or generators converge), 57,600 transmissionlines, and all major generating stations across thenorthern U.S. and eastern Canada. The team thenrevised the summer power flow case to matchrecorded generation, demand, and power inter-change levels among control areas at 15:05 EDT onAugust 14. The benchmarking consisted of match-ing the calculated voltages and line flows torecorded observations at more than 1,500 loca-tions within the grid. Thousands of hours of effortwere required to benchmark the model satisfacto-rily to observed conditions at 15:05 EDT.

    Once the base case was benchmarked, the teamran a contingency analysis that considered morethan 800 possible events as points of departurefrom the 15:05 EDT case. None of these contingen-cies resulted in a violation of a transmission lineloading or bus voltage limit prior to the trip of FEs

    Harding-Chamberlin 345-kV line. That is, accord-ing to these simulations, the system at 15:05 EDTwas able to be operated safely following the occur-rence of any of the tested contingencies. From anelectrical standpoint, therefore, the Eastern Inter-connection was then being operated within allestablished limits and in full compliance withNERCs operating policies. However, after loss ofthe Harding-Chamberlin 345-kV line, the systemwould have exceeded emergency ratings on sev-eral lines for two of the contingencies studied. Inother words, it would no longer be operating incompliance with NERC operating policies.

    Conclusion

    Determining that the system was in a reliableoperational state at 15:05 EDT is extremely signifi-cant for understanding the causes of the blackout.It means that none of the electrical conditions onthe system before 15:05 EDT was a cause of theblackout. This eliminates high power flows toCanada, unusual system frequencies, low voltagesearlier in the day or on prior days, and the unavail-ability of individual generators or transmissionlines, either individually or in combination withone another, as direct, principal or sole causes ofthe blackout.

    Endnotes

    20 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 3.5. MW and MVAr Output from EastlakeUnit 5 on August 14

    1DOE/NERC fact-finding meeting, September 2003, state-ment by Mr. Steve Morgan (FE), PR0890803, lines 5-23.2Transmission operator at FE requested the restoration of theAvon Substation capacitor bank #2. Example at Channel 3,13:33:40.3From 13:13 through 13:28, reliability operator at FE callednine plant operators to request additional voltage support.Examples at Channel 16, 13:13:18, 13:15:49, 13:16:44,13:20:44, 13:22:07, 13:23:24, 13:24:38, 13:26:04, 13:28:40.

  • 4. How and Why the Blackout Began

    Summary

    This chapter explains the major eventselectri-cal, computer, and humanthat occurred as theblackout evolved on August 14, 2003, and identi-fies the causes of the initiation of the blackout. Italso lists initial findings concerning violations ofNERC reliability standards. It presents facts col-lected by the investigation team and does not offerspeculative or unconfirmed information orhypotheses. Some of the information presentedhere, such as the timing of specific electricalevents, updates the Sequence of Events1 releasedearlier by the Task Force.

    The period covered in this chapter begins at 12:15Eastern Daylight Time (EDT) on August 14, 2003when inaccurate input data rendered MISOs stateestimator (a system monitoring tool) ineffective.At 13:31 EDT, FEs Eastlake 5 generation unit trip-ped and shut down automatically. Shortly after14:14 EDT, the alarm and logging system in FEscontrol room failed and was not restored untilafter the blackout. After 15:05 EDT, some of FEs345-kV transmission lines began tripping outbecause the lines were contacting overgrown treeswithin the lines right-of-way areas.

    By around 15:46 EDT when FE, MISO and neigh-boring utilities had begun to realize that the FEsystem was in jeopardy, the only way that theblackout might have been averted would havebeen to drop at least 1,500 to 2,500 MW of loadaround Cleveland and Akron, and at this time theamount of load reduction required was increasingrapidly. No such effort was made, however, and by15:46 EDT it may already have been too lateregardless of any such effort. After 15:46 EDT, theloss of some of FEs key 345-kV lines in northernOhio caused its underlying network of 138-kVlines to begin to fail, leading in turn to the loss of

    FEs Sammis-Star 345-kV line at 16:06 EDT. Thechapter concludes with the loss of FEs Sammis-Star line, the event that triggered the uncontrolla-ble cascade portion of the blackout sequence.

    The loss of the Sammis-Star line triggered the cas-cade because it shut down the 345-kV path intonorthern Ohio from eastern Ohio. Although thearea around Akron, Ohio was already blacked outdue to earlier events, most of northern Ohioremained interconnected and electricity demandwas high. This meant that the loss of the heavilyoverloaded Sammis-Star line instantly createdmajor and unsustainable burdens on lines in adja-cent areas, and the cascade spread rapidly as linesand generating units automatically took them-selves out of service to avoid physical damage.

    Chapter Organization

    This chapter is divided into several phases thatcorrelate to major changes within the FirstEnergysystem and the surrounding area in the hoursleading up to the cascade:

    Phase 1: A normal afternoon degrades

    Phase 2: FEs computer failures

    Phase 3: Three FE 345-kV transmission line fail-ures and many phone calls

    Phase 4: The collapse of the FE 138-kV systemand the loss of the Sammis-Star line

    Key events within each phase are summarized inFigure 4.1, a timeline of major events in the originof the blackout in Ohio. The discussion that fol-lows highlights and explains these significantevents within each phase and explains how theevents were related to one another and to thecascade.

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 21

  • Phase 1:A Normal Afternoon Degrades:

    12:15 EDT to 14:14 EDT

    Overview of This Phase

    Northern Ohio was experiencing an ordinaryAugust afternoon, with loads moderately high toserve air conditioning demand. FirstEnergy (FE)was importing approximately 2,000 MW into itsservice territory, causing its system to consumehigh levels of reactive power. With two of Cleve-lands active and reactive power productionanchors already shut down (Davis-Besse andEastlake 4), the loss of the Eastlake 5 unit at 13:31further depleted critical voltage support for theCleveland-Akron area. Detailed simulation model-ing reveals that the loss of Eastlake 5 was a signifi-cant factor in the outage later that afternoonwithEastlake 5 gone, transmission line loadingswere notably higher and after the loss of FEsHarding-Chamberlin line at 15:05, the system

    eventually became unable to sustain additionalcontingencies without line overloads above emer-gency ratings. Had Eastlake 5 remained in service,subsequent line loadings would have been lowerand tripping due to tree contacts may not haveoccurred. Loss of Eastlake 5, however, did not ini-tiate the blackout. Subsequent computer failuresleading to the loss of situational awareness in FEscontrol room and the loss of key FE transmissionlines due to contacts with trees were the mostimportant causes.

    At 14:02 EDT, Dayton Power & Lights (DPL) Stu-art-Atlanta 345-kV line tripped off-line due to atree contact. This line had no direct electricaleffect on FEs systembut it did affect MISOs per-formance as reliability coordinator, even thoughPJM is the reliability coordinator for the DPL line.One of MISOs primary system condition evalua-tion tools, its state estimator, was unable to assesssystem conditions for most of the period between12:37 EDT and 15:34 EDT, due to a combination ofhuman error and the effect of the loss of DPLs

    22 U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout

    Figure 4.1. Timeline: Start of the Blackout in Ohio

  • Stuart-Atlanta line on other MISO lines asreflected in the state estimators calculations.Without an effective state estimator, MISO wasunable to perform contingency analyses of genera-tion and line losses within its reliability zone.Therefore, through 15:34 EDT MISO could notdetermine that with Eastlake 5 down, other trans-mission lines would overload if FE lost a majortransmission line, and could not issue appropriatewarnings and operational instructions.

    In the investigation interviews, all utilities, con-trol area operators, and reliability coordinators

    indicated that the morning of August 14 was a rea-sonably typical day. FE managers referred to it aspeak load conditions on a less than peak loadday.2 Dispatchers consistently said that whilevoltages were low, they were consistent with his-torical voltages.3 Throughout the morning andearly afternoon of August 14, FE reported a grow-ing need for voltage support in the upper Midwest.

    The FE reliability operator was concerned aboutlow voltage conditions on the FE system as earlyas 13:13 EDT. He asked for voltage support (i.e.,increased reactive power output) from FEs

    U.S.-Canada Power System Outage Task Force Causes of the August 14th Blackout 23

    The Causes of the Blackout

    The initiation of the August 14, 2003, blackoutwas caused by deficiencies in specific practices,equipment, and human decisions that coincidedthat afternoon. There were three groups ofcauses:

    Group 1: Inadequate situational awareness atFirstEnergy Corporation (FE). In particular:

    A) FE failed to ensure the security of its transmis-sion system after significant unforeseen con-tingenci