I IU .. 1 22 ZOl - IN.gov

STATE OF INDIANA

I IU .. 1 22 ZOl INDIA' UIIU I !

R GULATORY CO v1fvll SIOI J

BEFORE THE INDIANA UTILITY REGULATORY COMMISSION

PETITION OF INDIANAPOLIS POWER & LIGHT COMPANY ("IPL") FOR AUTHORITY TO INCREASE RATES AND CHARGES FOR ELECTRIC UTILITY SERVICE AND FOR APPROVAL OF: (1) ACCOUNTING RELIEF, INCLUDING IMPLEMENTATION OF MAJOR STORM DAMAGE RESTORATION RESERVE ACCOUNT; (2) REVISED DEPRECIATION RATES; (3) THE INCLUSION IN BASIC RATES AND CHARGES OF THE COSTS OF CERTAIN PREVIOUSLY APPROVED QUALIFIED POLLUTION CONTROL PROPERTY; (4) IMPLEMENTATION OF NEW OR MODIFIED RATE ADJUSTMENT MECHANISMS TO TIMELY RECOGNIZE FOR RATEMAKING PURPOSES LOST REVENUES FROM DEMAND-SIDE MANAGEMENT PROGRAMS AND CHANGES IN (A) CAPACITY PURCHASE COSTS; (B) REGIONAL TRANSMISSION ORGANIZATION COSTS; AND (C) OFF SYSTEM SALES MARGINS; AND (5) NEW SCHEDULES OF RATES, RULES ANDREGULATIONS FOR SERVICE.

IN THE MATTER OF THE INDIANA UTILITY REGULATORY COMMISSION'S INVESTIGATION INTO INDIANAPOLIS POWER & LIGHT COMPANY'S ONGOING INVESTMENT IN,AND OPERATION AND MAINTENANCE OF, ITS NETWORK FACILITIES

) ) ) ) ) ) ) ) ) ) CAUSE NO. 44576 ) ) ) ) ) ) ) ) )

) ) ) CAUSE NO. 44602 ) )

INDIANA UTILITY REGULATORY COMMISSION TESTIMONIAL STAFF'S SUBMISSION OF INVESTIGATION REPORT

The Testimonial Staff of the Indiana Utility Regulatory Commission, by counsel, hereby

submits its attached Investigation Report. Pursuant to the Prehearing Conference Order entered

in this consolidated cause on April 15, 2015, the Testimonial Staff will prefile its prepared

testimony on or before July 27, 2015, including testimony of those individuals sponsoring the

Investigation Report submitted today.

Information deemed confidential by Indianapolis Power & Light Company, ("IPL") has

been redacted or removed from this public version of the Investigation RepOli. It will be

provided to parties with appropriate confidentiality agreements in place and to the Indiana Utility

Regulatory Commission upon an entry detennining whether it finds the infonnation confidential.

Testimonial Staff understands IPL will shortly file a motion to request a finding of

confidentiality for the redacted and withheld infonnation.

Respectfully,

eremyR. Comeau . 26310-53) Indiana Utility egulatory Commission 101 West Washington Street, Suite 1500E Indianapolis, Indiana 46204 Telephone: (317) 232-2102 Email: [email protected]

CERTIFICATE OF SERVICE

The undersigned certifies that on June 22, 2015, a copy of the foregoing was served via

electronic mail to the following:

Teresa Morton Nyhart Robert M. Glennon T. Joseph Wendt Robert Glennon & Assoc., P.C. Jeffrey M. Peabody 3697 N. Co. Rd. 500 E. Barnes & Thornburg LLP Danville, IN 46122 11 South Meridian Street [email protected] Indianapolis, Indiana 46204 [email protected] [email protected] [email protected] Timothy L. Stewart David Stippler Bette J. Dodd Randall Helmen Anne E. Becker Indiana Office of Utility Consumer Counselor Joseph P. Rompala 115 W. Washington Street, Suite 1500 South Lewis & Kappes Indianapolis, Indiana 46204 One American Square, Suite 2500 [email protected] Indianapolis, Indiana 46282 [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]

. [email protected] [email protected] Courtesy copy to: [email protected] [email protected] Kevin Higgins John P. Cook, Esq. Energy Strategies, LLC John P. Cook & Associates Parks ide Towers 900 W. Jefferson Street 215 South State Street, Suite 200 Franklin, Indiana 46131 Salt Lake City, Utah 84111 [email protected] [email protected] Kurt Boehm, Esq. Jennifer A. Washburn Jody Kyler Cohn, Esq. Citizens Action Coalition of Indiana, Inc. Boehm, KUl1z & Lowry 603 East Washington Street, Suite 502 36 East Seventh Street, Suite 1510 Indianapolis, Indiana 46204 Cincinnati, Ohio 45202 [email protected] [email protected] [email protected]

~~ eremy Cornea tty. o. 6310-53

Investigation of IPL's Network - Cause Number 44602-

Report of Independent Consultant

Final- 22 Jun 2015

Prepared for the Indiana Utility Regulatory Commission

By:

O'Neill Management Consulting, LLC 1820 Peachtree Road, Suite 709 Atlanta, GA 30309 Telephone 404-603-9226 Website: www.oneillmc.com

O'Neill Management Consulting, LLC Consultants to the Utility Industry

O'Neill Management Consulting, LLC IURC Cause 44602 - IPL Network Investigation

Table of Contents

1 EXECUTIVE SUMMARY ........................................................... - 4 -2 BACKGROUND ............................................................................ - 7 -

2.1 Events leading to the present investigation .... .. .... ......... ........ .................. - 7 -2.2 System Overview- Update ......... ........ ..... .. ..................... ...... ........... ........ - 8 -2.3 System Design and Equipment Specification- Update ...... ... ............... . - 11 -2.4 System Condition - Update ................................ ....... .. .............. ........... - 13 -2.5 Maintenance Planned and Completed - Update .. ..... , ............ ............... - 16 -

3 RELIABILITY AND SAFETY OF THE IPL SySTEM ..... ; ... - 17 -3.1 Recent history of incidents in downtown Indianapolis ......................... - 17 -3.2 IPL' s Response to the Incidents and Public Concern ........................... - 21 -

4 SYSTEM PERFORMANCE ...................................................... - 25 -4.1 Measures of system performance-Update ....... .... .... ... .. ... ... ........ .. ......... - 25 -4.2 trends in performance-Update .. ........................................................... - 27 -4.3 Comparisons with other utilities-Update ..... ................. ................ ........ - 27 -

5 EMERGENCY MANAGEMENT .............................................. - 28 -5.1 Emergency Planning- Update ..... .......... ................... ................ ............. - 28 -

6 ASSET MANAGEMENT ............................................................ - 29 -6.1 Asset Management-Update ....... .............................................. ... ........... - 29-6.2 Capital spending and maintenance-Update ........................................... - 35 -6.3 Maintenance and replacement-Update ... ............................................... - 37 -6.4 Failure analysis-Update .. ........... .... ....................................................... - 40 -

7 ORGANIZATION AND STAFFING ........................................ - 43 -7.1 Work planning and resources-Update ...................... ........ ........ ... .......... - 43 -

8 TECHNOLOGY .......................................................................... - 45 -8.1 Technology and secondary networks ........ : ........ .... ... .......... .. ...... ..... .. ... - 45 -8.2 SCADA ......... .. ..... ......... ... ........ ... ........... .. ..... ...... ....... .. ... ... ........ .. ... .... .. - 45 -8.3 Loadflow modeling ... ....... ........ ... .......... . : .... ... ... ... .... ............ .. ..... ....... ... - 47 -8.4 GIS/facilities records ... .. .... ...... ..... ...... .. ... ...... ...... ....... ... ... ........ ... ......... - 48 -8.5 Thelmal imaging ...... .............. ........ .... ................ ......... .......................... - 48 -8.6 Fault direction indicators ...... .... ............ ... ........................ ................. .... - 48 -8.7 Vented, locking, or lift/locking manholes covers ...... .... ... .................... - 49 -8.8 Remote data capture for inspection results via handheld devices .... ..... - 50 -

9 FINDINGS AND RECOMMENDATIONS .............................. - 51 -9.1 Findings ........ ........ .. .. .... .. .... .......... ..... ....... .... ..... ... .. .. ...... ......... .............. - 51 -9.2 Obtaining commitment for implementation ........... .. ..... ..... ......... .... ...... - 54 -9.3 Recommendations ................... ... ....................................... ...... .... .. ....... . - 55 -

APPENDIX A - IndyStar List Of InCidents Since 2010 ..................... - 56 -APPENDIX B - IPL'S July 7, 2011 Response to the IURC ............... - 58 -

- 2 -


APPENDIX C - Map of CBD network incidents, 2005-2015 ............ - 79 -APPENDIX D - IPL responses to data requests cited ........................ - 80 -

- 3 -

o 'Neill Management Consulting, LLC IURC Cause 44602 - IPL Network Investigation

1 Executive Summary

In September, 2011, in response to a recent (at that time) increase in network incidents,

our firm, O'Neill Management Consulting, LLC, was engaged by the Indiana Utility Regulatory Commission (lURC) to audit the electrical network in downtown Indianapolis, which is owned and operated by Indianapolis Power & Light (IPL), a subsidiary of AES Corporation (AES). We delivered our report1 in December, 2011;

then we helped develop an action plan that was finalized in January, 2012; and then assisted the IURC in monitoring IPL's implementation of the recommendations over the first year of the action plan, 2012. During the rest of2012 there were two network incidents in the Central Business District (CBD) in July, then none the rest of the year,

and none in 2013.

In the period from March, 2014 through March, 2015, however, there were five incidents, including four that involved significant amounts of fue and/or smoke, and in one case a

complete shutdown of one of the four downtown secondary networks. Moreover, two of the five occurred in mid-March, 2015, only two weeks before Indianapolis was to host the NCAA Final Four Basketball Toumament in Lucas Oil Stadium, April 4-6, with visitors concentrated in the CBD. Thus, the IURC opened a new docket, Cause 44602, and we were asked to assist the IURC testimonial staff in the investigation.

Our work included a visit to the site of the recent (2015) incidents and meetings with IPL operating and regulatory personnel and their outside counsel, as well as six rounds of formal data requests. We analyzed the data, followed up with additional questions and data requests, and formed our independent conclusions. Our six Findings and three Recommendations are detailed below in Section 9, where the interested reader may want

to go next to get a deeper overview before delving into each section that precedes it. Here we summarize that section as follows:

1. The design, maintenance, and operation ofIPL's downtown electrical network are basically sound, needing only incremental improvements to bring them to the next

level. The risk to the citizens oflndianapolis is low, and will remain low as IPL follows through on its commitments to improve the system.

2. IPL has been very responsive to the events from 2011 through the recent spate from March 2014 through March 2015, and has not hesitated to expend resources on

discovering the causes and committing to, and (to date) following through on,

1 "Independent Assessment of Indianapolis Power & Light's Downtown Underground Network",

December 13, 2011. See: http://www.in .gov/iurc/files/IPL Downtown Network Audit Report - Final Report(2).pdf

- 4 -


programs to remediate any perceived deficiencies. See Section 3.2 for details of the commitments and actions which IPL has undertaken. See Section 9.1, Finding 5 below for how that responsiveness may need to be better grounded in IPL's asset management processes.

3. The root cause of recent incidents is not anyone cause, but four separate causes associated with four separate categories of equipment, and within each category the root causes are not generalized, such as age or wear, but specific problems with specific models or situations, each pointing to specific programs of risk mitigation. Again, we refer the reader to Section 9.1, Finding 3, and also to Section 3.1 Recent history of incidents in downtown Indianapolis, and Section 5.4 Failure Analysis.

4. IPL has committed to the programs which address the root causes of such incidents, either as part ofIPL's response to our December, 2011 RepOli or as part of the response to IPL's root cause analyses of subsequent events. In many cases the actions have already been completed or are scheduled for completion in a reasonably timely manner. See Section 9.1, Finding 4, and also Section 3.2 IPL's Response to the Incidents and Public Concern.

5. In the area of asset management, we note IPL's significant improvement in asset management methods and procedures since implementing our fifth recommendation of the December, 2011 Report (see Section 5.1 for that recommendation and further details). Neveliheless, we feel there is not sufficient transparency in IPL's process of asset management to allow the IURC and the concerned public to see how IPL's responsiveness is reflective of a systematic program of asset management. We recognize that IPL continues to improve its asset management process2

, yet we think the time has come to also document to the outside in some detail the process by which the asset management function serves to address the risk and performance of the system, all in the context of cost effectiveness. To that end, we recommend an audit of the asset management process. (See Section 9,1, Finding 5 and Section 9.3, Recommendation 2).

6. Notwithstanding the merit ofIPL's responsiveness to network incidents, the process whereby IPL responds to regulatory pressure every time that CBD network incidents raise public awareness is not a good long-telID solution. We encourage IPL and the IURC to develop a system of repOliing, with specific metrics, such as the system of

2 In the December, 2011 public meeting in which we delivered our December, 2011 Report, we described

IPL's asset management process as more 'aspirational' than operational. With the progress since 2011,

much more of the process is now operational, although IPL admits that there are still some aspects that

are conceptually envisioned but not yet fully developed, i.e., what we would call aspirational.

- 5 -

O'Neill Management Consulting, LLC lURC Cause 44602 - IPL Network Investigation

Service Quality Metrics that has been developed in Massachusetts,3 and with which we have direct experience in optimizing. If, through such a system, IPL executes an

agreed-upon process in a transparent way, it can give the IURC confidence that the risk to the citizens of Indianapolis is being effectively managed to an acceptably low level. (See Section 9.1, Finding 6 and Section 9.3, Recommendation 3)

Finally, as we did in December, 2011, we advise that the IURC and IPL agree on a specific, monitored action plan. (See Section 9.2 and Section 9.3, Recommendation 1).

We also thank IPL for its full cooperation in responding to our requests for information and in making available to us in a timely way its staff and its facilities for our interview and inspection.

We have attached to this report those IPL responses to the IURC Testimonial Staffs data requests which are cited in this report (Appendix D).

3 "Recommendations for Strengthening the Massachusetts Department of Public Utilities' Service Quality

Standards", 12-13-2012, By O'Neill Management Consulting, LLC, Prepared for the Massachusetts Office

of the Attorney General. See: http://www.mass.gov/ago/docs/energy-utilities/ago-sq-review.pdf .

- 6 -

o 'N eill Management Consulting, LLC IURC Cause 44602 - IPL Network Investigation

2 Background

2.1 Events leading to the present investigation4

In September, 2011, in response to a recent (at that time) increase in network incidents,

our firm, O'Neill Management Consulting, LLC, was engaged by the Indiana Utility Regulatory Commission (lURC) to audit the electrical network in downtown

Indianapolis, which is owned and operated by Indianapolis Power & Light (lPL), a subsidiary of AES Corporation. Our firm was selected by the IURC from a list of qualified vendors developed by a process of requests' for proposal and competitive bids.

We delivered our Report in December, 2011, helped develop an Action Plan that was fmalized in January, 2012, and assisted the IURC in monitoring IPL's implementation of the recommendations over the first year ofthe Action Plan, 2012.

While our report was being prepared there was an incident in November, 2011, and another in January, 2012, as we were reviewing the action plan. During the rest ofthat year, 2012, there were two incidents5 on the downtown network, in the sense we define them here, i.e., public exposure to smoke or fire on the downtown underground network. 6

Both incidents were in July, one of which would likely have been avoided by the

measures that were completed by the end of2012, and the other of which was relatively minor, involving no more than a report of an explosion sound and a subsequently found displaced manhole cover. In 2013 there were no network incidents.

In the period from March, 2014 through March, 2015, however, there were five incidents, including four that involved significant amounts of fire and/or smoke, and in one case a complete shutdown of one of the four downtown secondary networks.7 Moreover, two of

the five OCCUlTed in mid-March, 2015, only two weeks before Indianapolis was to host the NCAA Final Four Basketball Tournament in Lucas Oil Stadium, April 4-6, with visitors concentrated in the CBD.

4 Some of this section is necessarily repetitive of some of the first paragraphs of the Executive Summary.

5 A third incident, on July 15, 2012, was on E. 16th street, outside of the footprint of the secondary

networks that serve the area roughly bounded by North, South, East, and West Streets, and so would not

have been included in our original scope. It was, in any event, a relatively minor event, as described in

Section 3.1.

6 All such incidents were examined in detail by us with reference to the detailed root cause analysis

reports prepared by IPL and reviewed by us.

7 Edison West, covering an area in the northwest part of the Central Bus'iness District (CBD) south of North

Street and east of West Street.

- 7 -


Thus, the IURC opened a new docket, Cause 44602. As stated in the Order:

"The Indiana Utility Regulatory Commission ("Commission") is aware of recent incidents involving Indianapolis Power & Light Company's ("IPL") network

facilities. Based upon our concern over the persistence of such events and previous information received from both IPL and an independent auditor during public meetings in 2011 and 2014 regarding underground network failures, the

Commission finds it appropriate to commence this formal investigation into any and all matters relating to IPL's investment in, and operation and maintenance of, its network facilities."

After issuing the order, this consulting firm was once again asked to assist the IURC in investigating IPL's performance. This time, we were invited to assist the lURC

Testimonial Staff in their investigation. We also met with several parties, including the staff of the Office of the Utility Consumer Counsel (OUCC) to explain our investigation.

Our work included a visit to the site of the recent incidents and meetings with IPL operating and regulatory personnel and their outside counsel, as well as six rounds of formal data requests. We analyzed the data, followed up with additional questions and data requests, and formed conclusions. Our Findings and Recommendations can be viewed in Section 9 below.

Section 3.1 includes a detailed review of the 5 recent events as well as 16 more in the last 10 years8

, with an eye to common root causes and IPL's agreed-upon action plans to date.

Readers who are very familiar with the system overview, design, and equipment specifications from the December, 2011 Report may want to skip directly to section 3.1.

2.2 System Overview- Update

In this section we begin by reprising the System Overview section fl.-om the 2011 review, with updates of the various quantities. Also significant is that in 2012 IPL's Substation

Number 3 network, which had been a separate, fifth underground secondary network fed from Substation Number 3, was subsumed into the four existing networks (essentially, into the Edison East network).

The following are updated values for some key statistics regarding IPL' s Indianapolis downtown underground network (as distinct from the rest of the IPL system):

8 See Appendix 1 for the annotated list from Feb 2010 to Mar 2015 as reported by Jill Disis of the

Indianapolis Star on indystar.com, March 19, 2015. Note: that list does not include the 24 Nov 2014

event, a dislodged manhole cover, and the 20 Jun 2011 incident, again two dislodged covers. The

descriptions are useful to review to discern the extent of public awareness and concern about each event.

- 8 -

o 'Neill Management Consulting, LLC TIJRC Cause 44602 - IPL Network Investigation

• Approximately contained within the area known as the Mile Square, bounded by East, West, North and South Streets, the territory is approximately 9 blocks by 9 blocks, or a mile long on each side, which, if exactly 9x9 and a mile square, would have 20 named streets (10 N-S and 10 E-W), 20 miles of roadway, and 180 block-long

sections of roadway, each 586 feet long.

• Serves 2,600 customers, with a peak demand of 114 MW, 3.6 percent of total system load (average peak load of22.8kW per customer).

• 625 miles of concrete-encased conduits.

• 1,214 manholes in the network area.

• 140 network vaults.9

• 305 network transfOlmers (average 2.2 per vault, with the most common count being one, and many having two, three, or more).

• 381,216 feet (72.2 miles) of network primary cable (21 circuits, each averaging 3.4 miles).

• 198,000 feet (37.5 miles) of network secondary cable (three-phase).l0

• Divided into four secondary network areas fed from two substations:

Edison, off!' Massachusetts Avenue to the northeast

Gardner Lane, off!' Kentucky Avenue to the southwest

*Position not exact, for security.

Map of area served by the four secondary networks (Edison East and Edison West,

Gardner Lane North, and Gardner Lane South):

9 For explanations of some terms which may be unfamiliar, the reader may want to consult Question 1 in

Appendix B of this report, for IPL's description of its downtown underground network facilities, or see

Section 1.5 of our December, 2011 Report for a primer on underground electric distribution systems.

10 The original secondary cable miles number in 2011 was determined to have been the phase miles

instead of circuit miles. This resulted in the reported number being three times what it should have been.

- 9 -


Figure I - IPL Downtown Network Boundaries

L fl ~ffi

In addition, we recognize that most public officials, regulators, and parties interested in

Cause 44602 are most familiar with radial forms of electric distribution: overhead, underground residential distribution (URD), or a hybrid of both (overhead down major

streets to the neighborhoods, then URD via a 'riser' that dips underground at the edge of the residential development). In urban areas like downtown Indianapolis, many electric utilities, including IPL, use a different design: underground secondary networks, with radial underground primary feeders that each supply multiple underground vaults where

transformers convert the voltage to secondary (in most instances, house current, i.e. 120V phase to ground) which in tum feed a network grid which is not radial, with radial services tapped off of that grid. For a more detailed primer on underground secondary networks, including some of the details about how IPL implements the design, see

Section 1.5 of the December, 2011 RepOli.

Moreover, since this review's scope, while motivated by the downtown network events, was not as limited as the 2011 review, we include the following statistics on IPL's total

- 10 -


system: IPL provides electric service to approximately 470,000 customers in a 528-square-mile service tenitory, basically covering Marion County, Indiana (which itself is 403 sq. miles) . IPL has almost 1,500 employees.

The company owns and operates three coal-fired generating plants, combustion turbines, and has a purchase power agreement with a wind farm, which gives the company a total

nameplate generating capacity of3,513 MW of power. IPL's generation mix in 2017 will include 45% natural gas, 44% coal and 10% renewables as opposed to a historic mix dominated by coal. The additional natural-gas generation at IPL includes the new 671MW Eagle Valley CCGT (combined cycle gas turbine), which is under construction

and set for commercial operations in early 2017 near Martinsville, IN.

In November, 2011, the parent company ofIPL, AES Corporation, which is a global

electric power company headquartered in Arlington, VA, acquired DPL, the parent of Dayton Power & Light. Since that time, there have been some efforts to exploit synergies in terms of senior staff, but at a more basic operating level the companies are run fairly independently.

2.3 System Design and Equipment Specification- Update

For detail of the underground network system design as well as comparisons to the underground networks in other cities, see section 2 of the December 2011 report. Very little has changed in the overall design and equipment specification since then, except the following:

There are now only 4 secondary networks, not 5, as noted above with the integration of the Substation Number 3 network into (for the most part) the Edison East network. The number of network transformers/network protectors is now 303, and the number of dedicated network feeders is 21. Revised versions of some of the tables in that report are

as follows:

Edison West network Five 13kV feeders to network vaults Edison East network Five 13kV feeders to network vaults

Edison - Other Four 13kV feeders to two network vaults

& primary service to one customer Edison - Non-network Four 13kV feeders to non-network vaults Gardner Lane North Five 13kV feeders to network vaults

Gardner Lane South Six 13 kV feeders to network vaults; Six 13kV feeders to non-network vaults

Substation 3 Two 4kV feeders to non-network vaults

- 11 -

O'N eill Management Consulting, LLC IURC Cause 44602 - IPL Network Investigation

Grid Networks Network Transformers bv Size (kVA)

Subs Feeders 300 500 750 1000 1500 2000 2240 Total

Edison 14 6 76 18 18 24 9 4 152

Gardner 11 0 10 7 51 46 34 0 151

Sub#3 0 0 0 0 0 0 0 0 0

Total 25 6 86 25 69 70 43 4 303

Also, we reprise here our main conclusions about the system design from the December,

2011 Report:

In summary, our experience indicates that the design of the downtown

Indianapolis underground network is consistent with peers in the industry, with

four notable features:

• The manholes and ducts are drier than most • The exposure to steam is greater than most • The connection to customers is almost exclusively three-phase • The secondary cable is predominantly P ILC, not insulated with polyethylene or

rubber

As we discuss the performance of the system in the remainder of this report, these

features will be seen to be significant.

In terms of equipment specification, we can update our December, 2011 Report by noting the changes that IPL agreed to implement as a result of our recommendations and their own root cause analyses 11:

• Retro-fill all termination chambers of network transformers with a fire-retardant insulating fluid ("FR3" is a brand) and for all new or replaced network

transformers, fill the main tank with such fluid.

• For all new transformers, specify elbow connectors instead of termination chambers.

• Replace all pre-I985 Westinghouse CM-22 network protectors in 480V vaults.

• Replace all aluminum bus network protectors.

11 A more extensive list of IPL's commitments that have ramifications for the system design and

equipment specification can be found in Section 3.2 below.

- 12 -


• After testing for toluene (none was found), discontinue practice of venting network protectors before operating them.

• Specify all manhole covers to be of the locking/pressure relief (Swiveloc TM) type,

and replace all existing covers with the new design by the end of 20 15.

• Install primary fault detectors at select locations to facilitate faster restoration.

• Install debris shields on selected transformers prone to accumulation of debris

from above.

2.4 System Condition - Update

In the December, 2011 Report, in Section 3, we presented data and analysis regarding the

condition ofIPL's downtown Indianapolis underground network. In each of the

subsections we looked at:

3.1 Summary of age and type.

3.2 Evidence from inspections.

3.3 Condition by type of equipment.

3.4 Maintenance planned and completed.

In this section of our report we note the more significant changes that have taken place.

To begin with, we reprise here some of the key findings of the system condition section

of the December, 2011 RepOli:

• The age distribution of the equipment in the IP L underground secondary network

in downtown Indianapolis can be summarized in two graphs, one for the network

transformers and one for the network protectors (see updated graphs below). It is

a reasonable assumption that age distribution of the primary cable follows that of

the network transformers and the age distribution of the secondary cable follows

that of the network protectors.

• The network protectors and secondary cable are older than the network

transformers and primary cable, in part because of more aggressive replacement

of primary and transformers in the last ten years, but also because the conversion

from 4kV primary to 13. 2kV primary which occurred in 1970 's and 1980 's was

. done for the most part by changing out the primary cable and transformers and

leaving the existing protectors in place, without any need to replace the

- 13 -


secondary cable since its voltage did not change. This is evidenced by the very

high percentage of network protectors that are 50 years old or older.

To examine what has changed since the last report in this regard, we begin with a comparison of the age distributions of network transformers and network protectors, shown in the two charts below (Source: DR 6.7).

The most immediately obvious difference is that IPL has now done a better job of determining the age of its individual assets in its vaults, especially the network protectors, which in our December, 2011 Report (and the gray values in the charts below) showed 37 percent oflPL's network protectors were classified as of unknown age. From the 2015

data it appears that many of those protectors of unknown age were in fact relatively newer, e.g., 0-10 years old or 20-30 years old. It is still true, however, that the average

age of network transformers in 2015 (which is 22 years old) is ten years less than the average age of the network protectors in 2015 (which is 33 years old).

Age Distribution of Network Transformers: 2011 & 2015 40%

35% 35 %

30%

25%

20%

15%

10%

Sq'c, 0% 1% 0% 0%

0% 0-10 10-20 20 -30 30-40 40-50 50-60 60-70 70-80 Unknown

Age (years)

Figure 2 - Age Distribution of IPL Network Transformers

Age Distribution of Network Protectors: 2011 & 2015 40%

35%

30% 2011

25% 23% 23%

20%

15%

10%

5%

0% 0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 Unknown

Age (years)

Figure 3 - Age Distribution of IPL Network Protectors

- 14 -


That picture is likely to change radically, however, because IPL's commitments to replace all 480V network protectors and all pre-I98S Westinghouse CM-22 network

protectors will mean the age distribution of network protectors will shift significantly toward younger ages as these programs are completed over the next few years. Since we would be somewhat surprised if IPL were to not take advantage of a change-out of a protector to also change-out the transformer (unless the transformer is quite new), we can

also expect a shift toward younger ages in the network transformers. There is not, however, a corresponding shift in the age distributions of primary and secondary cable that would be associated specifically with these two network protector programs. (See

Section 6.3 on maintenance and replacement to see a discussion of replacement from an asset management viewpoint).

Additional conclusions from the December, 2011 Report included:

• The primary cable is generally 750MCM or less, with a mixture of Paper-Insulated Lead-Covered (P ILC) insulation method for the older cable and solid dielectric

insulation for the newer cable.

• The secondary cable is generally 350MCM or more for the mains or ties between

transformers, with the predominant material being PILC or varnished cambric lead

covered, because of its resistance to high heat as might be experienced in the

presence of steam line crossings.

• The physical environment had the advantage of having very few 'w~t' manholes or

vaults, i. e, ones where there was a water level that would reach the lower levels of the

equipment; it had the disadvantage of many 'hot'manholes, i.e., ones which had an

elevated ambient temperature in the hole due to proximity to a steam leak. The latter

was distinctive relative to other cities, and we noted that Indianapolis has the second

largest steam distribution system in the nation, after New York City.I2

• Based on our onsite inspections in late 2011 and our review of pictures taken in the "full sweep" inspections of 2011 and also the earlier inspections in 2005, we found

that the maintenance seemed to be focused more on corrective activity, i. e. repairs

needed to restore service or prevent imminent failure, than truly preventive activity

that, while not necessarily leading immediately to a failure, might tend to do so over

time. Examples we cited included:

Unsupported cable (not properly racked)

12 This led to recommendations of increased coordination between IPL and Citizens Thermal (Which was

only separated from IPL in 2000), which has been implemented through monthly meetings, about which

we will have more to say below.

- 15 -


- Missing mechanical stress protectors (rack arm saddles and duct edge shoes)

Oil leaks (from the oil-impregnated paper in the cable, its dielectric fluid) - Low oil in tanks that are part of the transformer

Transformer nitrogen blanket pressure too high (could cause gasket failure)

- Excessive cable bending

Uncapped and/or unmarked cable retired in place

Our update for this report is that after reviewing the more recent maintenance records in

some detail, we see evidence that these conditions are being addressed and a shift to more preventive maintenance is under way. We will have more to say below about how this

improved focus of the field crews and supervision needs to be reinforced with a more transparent process of analysis from the asset management function.

2.5 Maintenance Planned and Completed - Update

In our December, 2011 Report we detailed the planned maintenance on manholes and vaults, with specific activities noted (Section 3.4 of that report). We also noted that from our inspections and review of pictures provided with previous inspections we felt that the description of the planned maintenance and the actual results were not always consistent,

and we suggested that this might be due to a systematic failure on the part of the crews and their supervision to recognize conditions that were likely to lead to failure. We called it 'looking' but not seeing" (See Section 3.4 and Appendix B, both of the December, 2011 Report).

As part of our update to this part of our review, we asked IPL (through formal data requests) to supply additional information on trends in the frequency and scope of its planned maintenance. We reviewed those responses in detail, and followed up with further data requests on specific work orders that we thought looked potentially

problematic.

An example is shown in IPL's response to DR 5.4, which shows that of fifteen inspections we targeted, four were shown not to have been completed within the specified three-year cycle for vaults,13 and were only recently inspected because ofthe 'full sweep'

done after the March 19, 2015 network failure and before the NCAA Final Four. 14

13 IPL's response is worded that for those four inspections the "records could not be found". Our

standard, and that of other effective maintenance audits is one that also applies in hospitals, etc.: ((Not

documented, not done", by which we mean the system cannot be counted on to work if documentation

does not confirm the activity, so if there is no record, one should assume that the work was not done,

even if one thinks it probably was.

14 We note in passing that the 'full sweeps' done in 2005, 2011, and 2015 also complicate the evaluation

of the effectiveness of the maintenance cycle.

- 16 -


3 Reliability and Safety of the IPL System

3.1 Recent history o/incidents in downtown Indianapolis

In this section we review the recent incidents as well as those of the past 10 years to put the root causes of the recent incidents in the context of previous incidents and to examine how the previously agreed upon recommendations did or did not address the issues and

root causes. See Appendix C for a map of these incidents.

Table 1 - IPL Network Events in 2014-2015

Date Location (or vicinity) Event Cause

19 Mar 2015 N. Capitol Ave. & Manhole explosions, Secondary cable fire

E. North St. network outage in caused primary cable

NW downtown damage, loss of supply

16 Mar 2015 Mass. Ave. & Explosion in vault, Fault in termination

E. New York St. self-extinguished chamber of transformer

24 Nov 2014 327 E. New York St. Cover dislodged Secondary cable fault

13 Aug 2014 26 S. Meridian St. Explosion"s and fire, Internal fault in network

outage to building protector

14 Mar 2014 150 E. Market St. Explosion and fire Secondary cable fault

Although the 24 Nov 2014 event was not considered newsworthy, the other events caused

disruption of traffic as police, fire, and IPL personnel made the area safe and did work. Video clips taken by passers-by were posted to the internet, and the events were covered extensively by print and television media.

Looking back before March, 2014, we show in Table 2 below the history of events from

2005 through 2012, including one that OCCUlTed while the review was taking place in November 2011, and four during 2012. There were no events in 2013 . Similarly, there were no noteworthy events on the downtown network from 2006 through 2009, except the network shutdown on 10 Nov 2007 that was required by the Indianapolis Fire

Department ("IFD") to deal safely with a fire not related to the IPL network. See Appendix A of our December, 2011 Report, answers to question 2, for IPL' s brief description of each of the events from January 2005 through 1 May 2011.

- 17 -


Table 2 - IPL Network Events in 2005-2012

Date Location (or vicinity) Event Cause

15Jul2012 E. 16th St. Smoking manhole Primary cable fault

5Jul2012 300 block Mass. Ave Displaced cover Primary splice failure

4Jul2012 S. Capitol Ave. & Vault fire Termination chamber

Westin Hotel fault, no FR3 yet

16 Jan 2012 IPL Center Substation Explosion and fire, . Circuit Breaker failure at

area outages the substation

19 Nov 2011 24 S. Delaware St. & Explosion and fire, Secondary cable fault,

E. Pearl car damaged due to old steam leak

20 Jun 2011 326 E. New York St. Dislodged 2 covers Secondary cable fault in

duct

31 May 2011 46 N. Capitol, near Explosion in vault Termination chamber

Governors' Office fault

27 Apr 2011 20 W. Court St. Explosion & fire in Customer junction box

vault fault (no limiters)

30 Jan 2011 E. Michigan St. & NJ Explosions & fire in 5 Secondary cable faults,

St., near Athenaeum manholes, duct possible natural gas, too

19 Aug 2010 342 Mass. Ave. Explosion, manhole Primary splice failures

cover dislodged due to undersized cable

18 Aug 2010 535 Mass. Ave. Vault fire Termination chamber

fault

17 Feb 2010 215 E. Ohio St. & N. Vault explosion, fire, Customer transformer

Del. St. ('gold' bldg.) bldg. evacuation internal fault

10 Nov 2007 Post Office Bldg., Network shutdown, IFD request, to work a

Michigan & Meridian NW Indianapolis non-electrical fire

8 Sep 2005 Talbot St. & Manhole explosion, Feeder fault, protector

Vermont St. multiple cable repairs failed to open, causing

(NP at 117 E. MI St.) & replace NP/Xfmr secondary cable fire

8 Jan 2005 137 W. Market St. Manhole explosions Secondary cable fault

(Bookland Bldg.) & fire, CO explosion due to steam leak; duct

in bldg. basement seals to basement were

temporarily removed

4 Jan 2005 114 W. North St. Manhole explosions Secondary cable fault,

& fire replaced 2000' of cable

- 18 -


It may also be useful to reprise and update our preliminary comments about the incidents

from Section 1.2 of our December, 2011 Report. The text from the original 2011 Report

is italicized and indented, and our updated comments are flush left, not indented:

We will have more to say below about how these events fit into the overall picture

of the system condition and the company's maintenance and replacement

practices. For now, we simply comment on the following key aspects:

1) While three events in three months warrants the attention which the lURe has

given to this topic, 12 events over almost 7 years is not a particular high

frequency when compared to the experience of many other similar utilities.

Update on item 1: Similarly, with 3 major events within the six months from August

2014 through March 2015, and especially two within days of each other in March, and

considering that the Final Four was about to be hosted in the city (as the Super Bowl was

about to be hosted by the city in early 2012), it is quite understandable that the IURC saw

fit to revisit in a formal way their concern about the performance ofIPL's downtown

network. In our Recommendations of our December, 2011 Report, we stated:

In summary, the rate of underground network incidents (12 in the last seven

years) is relatively low, but obviously the recent rate of five per year in 2011 is

alarming and should be addressed to bring the rate back down to the long-run

average of less than two per year, or even less.

We continue to feel that an average rate ofless than 2.0 per year is an appropriate goal.

Obviously, a merely displaced cover is less severe than a vault fire shooting up through a

sidewalk grating, so we would not say that a simple count of events should be the sole

measure of performance. Nevertheless, it is noteworthy that since the Action Plan was

adopted in late January, 2012, there have been eight events in 3.5 years, or a rate of2.3

per year. If we consider the action plan to have taken a year to have its initial effect, we

can count five events in 2.5 years, or a rate of2.0 per year, including some significantly

newsworthy events. IfIPL experiences no fulther events for 2015, its rate would drop to

1.67 per year (5 in 3 years), which would be consistent with the goal we recommended,

although the severity of some of the recent events still warrants the scrutiny now being

gIven.

2) Three of the 12 events, or 25 percent, were the result offailures in customer

equipment, not IP L equipment or procedures.

Update on Item 2: None of the subsequent events are due to customer equipment failure.

That could be interpreted to mean that even if the rate of events were to decline, one

- 19 -


might still have a concern that fewer events can be 'excused' as being customeroriginated.

3) Another 3 of the 12 events were either caused by or exacerbated by the failure

of equipment from another public utility, in particular a steam leak for typo and a

probable gas leak for another.

Update on Item 3: One of the subsequent events, and the most serious, namely dropping the Edison West (Northwest Indy) network on 19 Mar 2015, was apparently due to steam leak-related damage to adjacent electric facilities (see the discussion of the root cause

analysis of that event below). Since the low-pressure steam distribution line that may have caused the problem was retired in 2012, the damage may have been incurred before the newly enhanced utility coordination procedures were adopted in 2012 as a result of our recommendations. On this item we would re-iterate that with Indianapolis having the

second-largest steam distribution system in the nation, continued vigilance in avoiding damage to the electric underground distribution system from the steam distribution system will pay dividends in incident reduction.

4) Another 2 were caused by failures in the primary termination chamber of

network transformers. We will have more to say about this cause and its potential

remediation in a later section of our report, but here we note that this is not the

classic 'aged cable' failure mode, since the faults were not in the duct, but in the

termination chamber and were likely caused by a combination of mechanical

stress from unsupported cable and inadequate taping on the termination. In that

sense, they were more akin to splice failures than deteriorated insulation due to age.

Update on item 4: Two of the subsequent events were caused by telmination chamber failure (4 Jul2012 and 16 Mar 2015). In this regard it is important to note that the severity of the second event was significantly mitigated by the presence of a fireretardant irisulating fluid ("FR3" is one brand) in the chamber, which was the direct result

ofIPL's implementation of our December 2011 recommendation that they replace the insulating fluid in all of the termination chambers of the network transformers. It is very

likely that had it not been for the FR3, the 16 Mar 2015 event would have been much more serious, with flames shooting up through the sidewalk grate for a fraction of an hour, as was true of the 4 Jul2012 event. In fact, a previous termination chamber failure in the same vault as the 4 Jul2012 event (at the Westin hotel on 50 S. Capitol) but on a different transfOlmer led to no public event at all because that transfOlmer's termination chamber had been retro-filled with FR3 and in that case the FR3 was even more effective

at avoiding any fire/smoke. (In vaults with multiple transfOlmers, continuity of service to customers dictates that only one feeder be taken out of service at a time; therefore, the

- 20 -


retro-fills are scheduled accordingly, and the second transformer had not yet beenrevisited). In addition, IPL has followed through on our recommendation to begin replacing the telmination chambers of its network transformers with elbow connectors in all newly installed or replaced network transformers.

5) One was caused by a network protector failure. We will also have more to say

about that cause and its remediation in a later section of the report.

Update on item 5: This item refers to an 8 Sep 2005 incident in which a network protector failed to open upon reverse current, thereby allowing the secondary network to

feed a primary fault, causing damage to both. One of the more serious subsequent events was the 13 Aug 2014 internal failure of a network protector which led to multiple explosions and smoke pouring out of the sidewalk grate in fi·ont of the Oceanaire restaurant for over a half hour (a 'meltdown'). We have already been asked to review

IPL's root cause analysis of that event, and are satisfied that IPL's action plan to replace all such network protectors that operate at the 480V secondary voltage will address that issue, as well as an OSHA-mandated concern about arc flash (the 480V protectors generate a much higher level of fault current when they fail).

6) Finally we noted no significant deficiency in the company's response to each of

these 12 situations. Therefore, our recommendations will focus on avoiding the

future occurrence of similar incidents, rather than improvements to how the

company prepares for or responds to such incidents.

Update on item 6: Other than our recommendation that IPL develop a written plan for dealing with a network emergency, which was formally executed in the 19 Mar 2015 event, we continue to feel as we did in 2011, that IPL's response to these events is acceptable and does not wan·ant additional improvement. We have expressed our

concern, however, that this response may be critically dependent on retention or effective replacement of key personnel with the knowledge of how to respond to network events. We will have more to say about that in the section below on staffing.

3.2 IPL's Response to the Incidents and Public Concern

IPL has been very responsive to regulatory concern about the increased frequency of incidents in the downtown underground network. This has been evidenced in a number of ways as described below:

1. IPL response to the increased frequency in late 2010-early 2011 (from the July 7, 2011 public meeting with the IURC):

- 21 -


One-time 'full sweep' (100%) inspection of all manholes (rate had been 20% per year)

Increased inspection cycle frequency (from 5 to 3 years for manholes, from 3 to 2 years for vaults)

Network SCADA (Supervisory Control and Data Acquisition, i.e. for remote

control and monitoring of conditions in each vault) to be completed in 2013

Primary cable fault detectors - to be completed in 2012

Install secondary cable limiters at select locations

Pilot, then implement hand-held thermal imaging devices

Pilot and selectively implement (75 locations at first) combination locking/pressure relieving manhole covers ("Swiveloc TM,,)

Agreed to fund an independent assessment of IPL' s downtown Indianapolis underground network, which resulted in the December 2011 report by this consulting firm

2. IPL response to December 2011 report of this consulting firm: IPL agreed to implement all 5 first tier and all 5 second tier recommendations, and to have this consulting firm monitor the implementation over the next twelve months. These included (fl.-om IPL's January 2013 Summary Report of work done in 2012 in

response to the December, 2011 Report):

500+ Inspection-generated repair work orders from 2011 inspections completed

202 Swiveloc ™ locking manhole covers were installed (152 before the Super Bowl, 50 later in 2012)

315 Termination chambers changed to fire retardant fluid

315 Network transformers tested for Dissolved Gas Analysis (sets ~ baseline for future tests)

154 Deflector shields installed over transfOlmer/protector (avoids cOlTosion)

Fault detectors installed on 43 of 44 circuits for faster feeder restoration (44th install

will occur after proactive cable replacement)

65 Network protectors checked for aluminum bus. Only 25 had aluminum bus. 7

have been replaced

- 22-


6 Network transformers acquired with new specifications (no termination chamber),

1 installed

18 Network transformers designated to be retrofit (3 already shipped)

10 Tablet computers acquired, software customized, installed, and users trained

244 Inspections completed using tablet computers, including many with pictures and

notes

6 Infrared imaging cameras purchased and users trained

549 Infrared checks completed and entered into Ivara asset management software

300+ Network protector relays replaced as part of SCADA project

5 SCADA Gateways installed, 6 more to be completed by end of2013-Q2 (allows monitoring NP's)

153 Steam-related work orders entered and all completed except recent entries still

being worked

35 (of the 153) steam-related work orders were generated by new information from

Citizens Thermal

13 Network protectors tested for toluene gas. All were found acceptable, so it is safe

to operate without venting

84 Network-related equipment incidents entered into new failure database on new

forms

All (5) Networks modeled on CYME computer modeling program (proven accurate in modeling recent actual fault)

3. IPL response to the event on August 13, 2014 at 26. S. Meridian (From the Recommendations in IPL's October 3,2014 Root Cause Analysis), IPL committed to:

Continue to replace all 480V network protectors, beginning with the 58 pre-1985 Westinghouse CM-22 network protectors. This action was already part ofIPL's plan to mitigate arc flash as a worker safety issue, as required by recent changes

in OSHA regulations. This program was to be integrated with the previously agreed upon replacement of aluminum bus network protectors.

Develop a fOlmal, written Network Emergency Response Plan (This plan was subsequently put into action on the March 19, 2015 event).

- 23 -


Met with IFD on September 10,2014 to ensure coordination ofIPL and IFD actions at network emergencies.

Develop a mitigation strategy that would limit the severity and consequences of a network transformer or network protector failure, e.g., revised relay settings.

Conduct a review of all SCADA gateway vault circuits for communications stability and frequency of telemetry en-ors.

Enhance the network protector inspection process to ensure attention to the issues cited in the root cause analysis (gray spool insulator cracking, etc.).

Continue to monitor, update, and implement IPL's succession plan for key staff responsible for network operations and emergency response (to ensure adequate

levels of experience and expertise).

Continue commitment to participate in industry forums and conferences where

other companies that operate underground secondary networks participate.

Take steps to improve the state of inspection and maintenance records by implementing periodic auditing to improve the retrieval and completeness of inspection and maintenance records so as to better support future root cause analysis of network equipment failures .

4. IPL response to the events in March, 2015 (and in preparation for the NCAA Final Four in Indianapolis - from IPL's presentation at the public meeting before the IURC

on April 1, 2015):

One-time 'full sweep' (100%) inspection of all manholes and vaults

Upgrade from partial implementation of locking/pressure relief manhole covers to

full (100%) implementation by the end of2015

We take this extensive list of actions as evidence that IPL has endeavored to respond to the severity and frequency of downtown network events and the accompanying public

scrutiny. We will have more to say below about the prospect of moving fl.-om a mode of 'putting out fires' (often literally, but we mean more figuratively in the sense of various remediation programs after the event) to a more proactive mode that will anticipate and .

avoid future failures by monitoring those key measures that can direct IPL' s efforts to reduce the frequency and severity of future downtown network events.

- 24 -


4 System Performance

4.1 Measures of system performance-Update

In our December, 2011 Report, we discussed at some length the industry's typical measures of system reliability (SAIDI, SAIFI, and CAIDI, with and without adjustment for major events like storms) and noted that for downtown underground networks such measures are inadequate. For one reason, such networks, with their built-in redundancy,

are typically highly reliable, contributing only a small percentage to the system total annual duration or frequency of interruptions. But such reliability is expected from the central business district, with its high-rises, elevators, and function as the core of a community's business (and tourism) activity. Moreover, any measure of downtown

network perfOlmance must obviously include the safety ofthe public in such heavily trafficked areas. As we said in our December, 2011 Report,

What many utilities do in this regard, and what we would recommend for IP L, is

to measure and analyze their performance in terms of the failure rate of the

network equipment and also the completion rate of their scheduled maintenance ...

In other words, for downtown underground networks, where failures can lead to

incidents involving explosions or fires, it is not sufficient to monitor such

catastrophic events and try to control them as they occur. Rather, the proper

approach is to monitor those conditions that are likely to lead to more serious

events, i.e., incipient faults, overloads, or stress-causing conditions, and to

prioritize such conditions and then mitigate them as appropriate in a timely way.

Looking beyond the downtown Indianapolis area, IPL has for at least the last five years shown first-decile performance on SAIDI and SAIFI. While we believe that such results are easier to achieve for a utility whose service territory is basically one county, it is still

an admirable performance, and one might be expected to prefer to be an IPL customer than any other investor owned utility in Indiana or indeed most other states. Still, with aging infrastlUcture and possibly changing climate, IPL should apply its best asset

management techniques to maintaining a satisfactory level of reliability.

As we find ourselves once again, for the third time in four years, called to serve the IURC

and IPL as an independent consultant reviewing issues in network performance, we are inclined to take our recommendations to the next level in telms of how the IURC and IPL might achieve a regulatory regime that did not require such active inquiry. We have noted in previous years that there is not sufficient transparency of IPL' s processes so as

to instill confidence by the IURC in how IPL is managing its cost and risk. To some

- 25 -


extent, this may be due to the technical nature of electric distribution in general and especially underground secondary networks in particular that leads to the relative opacity ofIPL's strategy, spending, maintenance, and operations. We think, however, that the screen may be pierced by a sufficiently detailed and well-thought-out system of

performance reporting that went beyond the existing measures such as SAIDI and SAIFI to include measures that had a more direct tie to some of the risks and satisfaction drivers that are inherent in IPL's business.

As it turns out, we speak from some experience in this regard, having not only observed

various systems of regulatory reporting in the several jurisdictions in which our clients operate (many of our clients being larger investor-owned utilities that operate in multiple states), but also having participated recently in helping to design and optimize regulatory

. h 15 reportmg sc ema.

For years, we have been intimately involved in client, regulator, and industry efforts to measure reliability in ways that properly reflect the different aspects ofthat sector of service quality, but we have also been involved in efforts to measure performance more broadly, in telIDS of customer service and satisfaction, public and employee safety, cost,

environmental quality, and community commitment.

We have seen elaborate and complicated mechanisms involving benchmarks, means and standard deviations, percentiles, dead bands, cross-sector credits, and both penalties and rewards. All of these concepts arise not for the purpose of analytical sophistication in itself, but as a means to capture and reward/penalize the right aspects of performance. In fact, we have seen these mechanisms employed in internal corporate employee group or

subsidiary performance measurement and incentive systems as well.

We think that the very effort itself to design such systems tends to bring greater cladty to

both sides of the regulatory equation in terms of what behavior/perfOlmance is desired and what is possible to measure. Accordingly, we state as our fifth Finding and our third Recommendation that the IURC and IPL should initiate a formal discussion of how such

a system might improve the regulatory relationship between the two entities and ensure a better outcome for the citizens of Indianapolis.

15 See footnote 3 for a reference to our work in Massachusetts. We have also been involved in efforts on

behalf of public utility commissions in Ohio, Pennsylvania, New Jersey, and Connecticut, and have assisted

clients in regulatory responses in New York, Illinois, Maryland, the District of Columbia, and Texas.

- 26 -


4.2 Trends in performance-Update

In our December, 2011 Report, we noted the trend in failures for underground network

equipment, which averaged approximately 30 per year. Our update for this report is that

the trend in such failures has been toward a lower figure, representing the favorable

effects of the various programs which IPL has implemented over the last four years.

Clearly, such trends must be continuously monitored and actions taken to reverse any

adverse trend. This is the job of asset management, about which we have much to say

below in Section 6.

4.3 Comparisons with other utilities-Update

In our December, 2011 Report, we devoted a section to comparisons of network

performance with other utilities, noting that there are not many publically available

sources, but that from our extensive work with other utilities and surveys we have done

for them, we have a good sense of what typical performance is. We have no update as

such to that earlier analysis, so after referring the interested reader to that report, we

would simply highlight here a few of our overall comparative statements:

What is especially notable when comparing IPL 's performance with that of other

underground systems is that it does not have many 'merely smoking' manholes.

Its incidents seem to more often fall into the category of fires or explosions

(" over-pressurization)} events). This may be due to the fact that it has relatively

dry holes and does not have much secondary cable with insulation made of

rubber or polyethylene. The latter can vaporize in the presence of electrical

.arcing, and create combustible gases which may at first merely smoke, but if allowed to migrate to a structure and collect there, may combust with more force.

An additional item worth noting is that in interviews with IP L personnel, when

asked if there were reports of electric shock coming from energized nenjlork

structures, they reported none. We have heard similar reports from southern

utilities that do not see much snow/salt, but northern utilities have recognized this

type of event as an issue, including ConEdison, where a customer fatality was

involved in such an incident in January of 2004, and Massachusetts, where a

study was done in 2005 (by a team led by Dan 0 'Neill when he was with Navigant

Consulting) after some reports of animals being shocked in Boston.

In summary, the rate of underground network incidents (12 in the last seven

years) is relatively low, but obviously the recent rate offive per year in 2011 is

alarming and should be addressed to bring the rate back down to the long-run

average of less than two per year, or even less.

- 27-


Clearly, we re-iterate that a rate of five incidents in the space of a little over a year

represents an alarming rate that warrants addressing. Cause 44602 addresses that need.

5 Emergency Management

5.1 Emergency Planning- Update

In our December, 2011 Report we described IPL's Emergency Planning function as

satisfactory, citing mainly its preparedness for storms and other threats to the overhead

system, but we also noted the experience with a requested network shutdown in

November 2007, and the close coordination with the Indianapolis Fire Department (lFD).

Subsequently, after the August 2014 incident we noted that there was no written network

emergency plan and we suggested one should be developed. IPL complied with this

recommendation and used that plan in the 19 Mar 2015 network shutdown. IFD felt the

coordination was good in that event.

- 28 -


6 Asset Management

6.1 Asset Management-Update

In our December, 2011 Report, in section 6.1, we described how the process of asset

management has begun to be adopted in asset-intensive industries like utilities, and how the typical asset management methods and procedures had relevance for IPL and specifically its downtown underground network in Indianapolis. One of our five top-tier recommendations was specifically focused at improving the asset management process at

IPL:

5) Improve the process of asset management by dedicating additional resources to development of equipment databases and processes that facilitate effective failure analysis and resource planningfor condition-based equipment maintenance and

replacement that goes beyond imminent failure.

At that time, we described IPL' s Asset Management functionality as more' aspirational'

than 'operational'. The IURC has expressed interest in the details of IPL' s asset management process and has received various snapshots of information since 2012,

including:

• In its May 4,2012 response to an April 4, 2012 letter from the IURC with eleven specific multi-part questions about IPL's root cause analysis of the Center Substation circuit breaker failure that occurred on January 16,2012, IPL provided 19 pages of answers and 20 pages of exhibits, most of which related directly to IPL's asset management process.

• On July 12, 2012, an additional 6 pages were provided in answer to the IURC's follow-up questions, and IPL delivered a PowerPoint presentation with 25 slides including a timeline for the development of the process (See Table 3 below)

• On September 20,2012, IPL responded to the IURC's August 20, 2012 letter requesting infOlmation about IPL's asset management process, specifically six questions about how equipment with operating restrictions is managed and also how maintenance work is managed. IPL's six-page response included some graphs used for work tracking in IPL's EMPAC and WMIS software systems.

• On October 19, 2012, IPL responded to the IURC's letter of October 4,2012 in which the IURC followed up on the six questions it had asked on August 20. IPL's response included 28 pages of answers and 13 appendices, the latter being

- 29-

o 'Neill Management Consulting, LLC IURCCause 44602 - IPL Network Investigation

exhibits from IPL's most recent (9-9-2012) monthly Performance Analysis Report (PAR), and also the PAR for the week ending January 29,2012.

• On December 14,2012, IPL responded to the IURC's letter of November 14,

2012, in which the IURC asked three multi-part questions about IPL's asset and risk management. IPL responded with a three-page direct answer to the questions and a 30-page paper on IPL's Asset and Risk Management which IPL referenced in its three-page direct answer.

• On January 25,2013, IPL presented its 47-page report to the IURC on the status of its implementation in 2012 of the recommendations of our December, 2011 Report. This included 4 pages on the status ofIPL's implementation of our

recommendation number 5, asset management.

Table 3 - IPL's Timeline for Asset Management Development - July, 2012

2008 2009 2010 2011 2012 2013 2014

Power Prioritization larry Conrad Asset Asset Health Oelivery Asset model first Full time Management lv.uCl indexinapf MassCisset Management developed consultant group collection of key substation analysis in

Initiative and lIsed added expanded indicators assets Ivara

lvara assets Analysis of loaded and Incorporate Residential Expanded current failure

Meetings with Cable. Sub analysis to practice Storm Improve tracking key AM Transformers, resource reviews response integration of .malysisin

consultants etc. requirements developed analysis databases lvara

POC.pEx Smart Grid Adding UsingAHI budget Utilities plan for Revised CBO criticality review replace

centralized International Distribution inspection analysislo I refurbish under AM engagement Automation process lvara strategies

Analysisof EMPACto

Expanded create a Work order (apEx scoring analysis to manageable Interface and budgeting Add smaller

analyze number of developed viewable on individual resource work between lvara IPlIntr.net assetsinto

requirements -buckets· and EMPAC website Ivara

Continued Analyslsof Field Trial of review and

work lvara AM group tablet development -buckets· purchased expanded computers orAMKPI's

Prioritization model

enhanced to Ivara KPI inciudeO&M Development

Storm response analysis

One might think that, with such extensive questioning and responses, the IURC would feel confident that it knows and understands IPL' s asset and risk management process. Actually, that is not the case. This is likely due to two factors: First, the process is a moving target, as acknowledged by IPL and evidenced in Table 3. Second, the answers

provided by IPL failed to satisfy the IURC's questions, and in fact, often raised more questions than they answered, as in the IURC's letter to IPL of October 4,2012, and echoed elsewhere in the four-year timeline since 2011,

- 30 -


"Thank you for your responses to our follow-up questions which were sent to you

on August 20,2012. Unfortunately, your answers raised additional questions.

Please address the following:" ...

From our own reading of the process of development oflPL's asset management process,

we have noted considerable progress from what we observed when we first gathered data

in 2011 . At that time, we were disappointed to see:

• The function was minimally staffed, with the manager of asset management doing

much of the analysis himself, assisted by a consultant, interns, and sometimes

borrowed time from other functions. This would suggest a half-hearted

commitment to the development of this important process.

• The software used to accomplish the function, which was centered on the Ivara

application which was fed by two different work management systems, EMP AC

and WMIS, was relatively new at IPL (See Table 3) and was not in an advanced

state of integration and implementation. Many operations personnel who might

have been users were not trained on and had no direct access to Ivara, relying on

the manager of that process to supply any information and analysis, much of

which might be done by special routines written ad hoc by that manager.

• The systems for capturing data on system condition were paper-based and not

well integrated in such a way as to provide timely asset condition information to

those who might need it most. We noted specific instances of failure to be able to

produce records needed for failure analysis, as well as a tendency of inspections

to undeneport conditions which could lead to failure. 16 We recommended

implementation of a tablet-based data entry system that could provide intelligent

business rule-based aids to data entry.

• The process that suppOlis failure analysis was almost non-existent. We helped

IPL develop a form to capture failure infOlmation in such a way as to support a

formal root cause analysis where warranted, e.g., for explosions or other

newswOlihy events. We also encouraged other practices such as segregation of

damaged equipment in a way that it could be examined for potential causes of

failure (as opposed to being thrown on to a scrap pile with other refuse).

• We failed to see evidence of a coherent, condition data-driven replacement strategy for major classes of equipment. What we saw was a pilot for one

16 One of the many examples was evident in the chart that showed the average age of network

protectors, where over 30 percent were unidentified by age. IPL subsequently addressed this, as

evidence by the updated chart in Section 2.4.

- 31 -


category of equipment (URD cable) and the software on which the pilot was based was not subsequently purchased or implemented. Again, this suggests a

half-hearted commitment to the process.

On the plus side, we saw a predilection for piloting new technology, some of which could be very useful, including winning a Department of Energy grant to implement a SCADA system for the downtown network, and an interest in pilots for locking/pressure-relieving

manhole covers, underground feeder fault indicators, inexpensive portable thermal imaging equipment, loadflow modeling, a geographic information system, and the handheld data capture technology mentioned above. Our report recommended adoption of many of these technologies and IPL enthusiastically embraced those second-tier

recommendations.

Nevertheless, as we look at the progress made in the last three years, we still see a considerable opportunity for improvement in the application of asset management at IPL. Specifically, while we have definitely seen progress made in all of the areas noted above, we still see the following (matched to the five items listed above):

• While the staffing of the asset management function has increased significantly, it still appears to us to depend heavily on the ad hoc information processing ability of the manager in charge of the function and a handful of employees, contractors

and interns.

• The implementation of the Ivara software has clearly advanced significantly, and the development of an internal asset management website has made the data available to more personnel without their having to become trained users on the I vara system. Still, that user access is accomplished through customized code

(SQL queries and Visual Basic macros) developed by a handful of talented staff/consultants/interns, which means the users' access to the data is limited to the specific programs or queries written to deliver data through the web portal.

• Clearly, the system has advanced considerably from the paper-based version we saw in 2011, including the implementation of the MobilFrame tablet-based system for data capture of inspection infOlmation. There were many hiccups along the way, such as an aborted attempt to use an Ivara front-end as the tabletbased application, and also some problems with docking the tablets that caused

data to be lost without it being apparent until the data was needed for failure analysis or similar needs. While we acknowledge the progress made, we still sense that additional integration of disparate systems is needed.

- 32 -


• We have been pleased to see some high-quality failure analysis reports now coming from IPL. (Although we, like IPL, would prefer not to have a need for such repOlis, nevertheless, as long as noteworthy failures occur, we expect to see

a competent root cause analysis scaled to the severity of the event). What we still have not seen is a systematic failure database that would serve as a basis for asset management strategies, which leads us to our next point.

• We have been pleased to see the draft of the 66-page AES Downtown Underground Network Asset Life Cycle Plan dated March 23,2015 (confidential attachment in response to DR 6.4). It would be unfair to be too critical ofthis document, since it was acknowledged by IPL as only a draft of a work in progress!7; yet we must point out that while it lists a number of recommendations

for specific changes in programs, the analysis that would ground those programs in the details of asset failure rates, failure impacts (risk) and program costs is either non-existent or not very convincing. This could leave one with the sense that IPL's responsiveness is based more on a desire to be doing something to

respond to events rather than from a systematic approach to strategic options in asset management. In that regard, we look forward to a more extensive review that may reveal additional analysis or judgment that simply has not yet been put to writing.

• We have noted specific instances where it seems to us that problematic conditions at a few specific sites have 'slipped through the cracks' as a result. To some extent, this sort of thing happens at other utilities, as all systems are vulnerable to a celiain degree of human error, whether by the users or the programmers. Yet we feel that IPL is perhaps more vulnerable than it needs to be in this regard, and while it has not severely affected perfOlmance so far, specifically, none of the recent incidents can be traced to a failure of asset management as such, we think it

prudent to bring the system to a higher level of performance through a more thorough audit of its current functionality and by a program of making its functionality more systematic.

In discussions with IPL asset management staff, we have been confronted with the assertion that IPL is doing asset management as well as any other utility in Indiana, and as well as some of the best in the country. We cannot effectively evaluate the claim relative to Indiana, but we would affirm that IPL is making good progress toward being a leading practitioner of asset management. Yet, the fact that it is still in process, and that

17 For example, there are sections in it with questions marks for missing data, and sections with notes like

"We need to add analysis", and "In future updates to this ALCP we will analyze the financial data with

much more rigor and detail". Yet other sections provide detailed analysis of selected alternatives.

- 33 -


some things still seem to fall between the cracks as the system continues to be modified,

leaves us and the IURC Testimonial Staff with questions as to what part of the process is

mature and solid versus what is still aspirational. In response to such a question from the

IURC, IPL in its December 14, 2012 response produced the following table:

Table 4 - IPL Asset Management Maturity Assessment - December, 2012

OH UG Vee Attrlbu~e CSO Subs T~lin e

Di rt Dist ..... ~mt

Clear rotes and aturc at uro;- Miltur.:- ,ature Matur~ Mature-

r~ sponsi iii i~~ .

Well d -'ln~d insp -c t i atu e A~tu rc? M.ature t.I,atufe> M tUff~ M lUI'~ and maintenance plan.

t

Risk bas.~d priorftizaUo-n atur e lature Mature tature Mature Mature methods

Failure t raeh.ing and atu r ','ture "at re 20!3 - 013 Matur~ rep r tl n~

Mature l ion ou~g E' defect

013 201 3 ~ t r for ' fm r.

0 13 M- ture reportln~ nd tr ckin~ 013 01'

orea ers

Ann ual revi e-vi of programs ,I.\atu n~ Mature "".ature J.,',ature Mat urE' Matu re

Fmanel al mE't ries pr - ' ~ntive/(orr t i'J~, :10 - 2013 2013 201 3 2013 20 13 20H IIAY

Monit ori ng and I 00 Is in tu r~ atur - 20!3 20 13 10' ture managt" ",nt .)r i.'Icklo g place

Indivi dual ~..sset Criticality atur e \at ure 2013 2013 2013 Mature Ra ini

Individual A5:$et Health atur- \ tu r~ t ur~ 2013 20 3 M tur I nd~x

This would suggest that the process for the CBD (and in fact, for the whole company)

was to be considered mature by the end of 20 13. We do not judge that to be the case, as

is evident from the current state ofthe Asset Life Cycle Plans (Table 8, below).

What is needed is not an updated Table 4, since the definition of maturity is probably a

matter of opinion, but rather a clear picture of which processes aTe currently working at a

satisfactory level of proficiency and which are still under significant development (and a

schedule for their completion in terms that can be verified). We believe this could be

accomplished by a focused audit of the asset management process itself, with the intent

of clearly revealing and communicating the state of its development. It is possible that

such an audit would generate a pleasant surprise and demonstrate a higher level of

maturity than we currently suspect, but only a more detailed review would allow such a

- 34 -


conclusion. In any event, such an effort would, if done properly, improve the transparency of IPL' s asset and risk management process to the IURC and the public, thereby ensuring greater confidence that IPL is managing the assets and risk.

6.2 Capital spending and maintenance-Update

In our December, 2011 Report we commented on two tables with the ten-year trends in capital spending and underground maintenance spending, both for the downtown network and the whole transmission and distribution system. There we noted, and more recent

data confirms, that capital and operations and maintenance (O&M) spending on the downtown network make up a small portion of the total T &D spending - approximately five percent. The number for O&M cannot be calculated exactly because the group that does the downtown network O&M also does substation operations and maintenance for

the whole system.

The table below shows the projected O&M budget for IPL transmission and distribution ("Power Delivery"). The portion attributable in 2015 to Cost Center 124 - Substation and Network is only $3.94 million, or 6.3 percent, including substation O&M not related to

the downtown network. This does not reflect any 'starving' of the spending on the downtown network, but only that the downtown network is a small part ofIPL's total transmission and distribution system.

In the past, we have been asked whether financial difficulties at AES might cause IPL to have to tighten the belt on its spending, and that this could cause the downtown network to suffer. Our sense in 2011 and still today is that there is a lot larger pool of potential economies outside of the downtown network and that IPL has shown little to no reticence to spend money on addressing the causes of public events on the downtown network, including ordering 'full sweep' inspections of downtown manholes and vaults by outside

contractors (each costing almost $1 million, all of which would be considered discretionary spending and probably not originally budgeted), and increasing the frequency of regular inspections, as well as the many other items listed in Section 3.2 -IPL's response to Response to the Incidents and Public Concern.

Table 5 - O&M Budget Trend ($OOOs) From DR 1.7

2015 2016 2017 2018 2019

Total T&D I I I I - - -Sub & Network - • • • • The story is similar for capital spending, where the budget trend is shown in the table below. We note that the 2015 budget is approximately $20 million higher than the out

- 35 -


years because of some specific non-recurring growth-related projects, e.g., $10.8 million for electric vehicle chargers for French battery/car maker Bollore's electric car-sharing

project in Indianapolis. The portion of this budget attributable to the downtown underground network is closer to ten percent, but we note that over half of the Central Business District ("CBD") spending in 2015-2016 is due to the replacement of the 480V

network protectors, and the network capital maintenance for 2017-2018 is $5 million more than usual in those two years.

The table of CBD capital investment from our earlier study showed spending that was cut

in half the year after the acquisition (2000) to approximately $1 million per year for 2001-2003, but it ramped up in 2004 and peaked at $3 million in 2005, when there were three network incidents, including two in January, then averaged $3 million per year for 2006-2009, enhanced by some growth-related spending (Cultural Trail, etc.), then

doubling to $6 million in 2010, when there were multiple network incidents, and has remained even higher since then. Again, while it may have appeared in the years just after the acquisition that IPL was under-spending on its downtown network, it is clearly spending at a much higher level in response to recent network incidents, and also in

response to growth opportunities in downtown Indianapolis.

Table 6 - Capital Budget Target ($OOOs) From DR 1.8

2015 2016 2017 2018 -2019

Total T&D - - -eBD - - -In short, we can re-affirm the statement made in section 6.2 of our December, 2011

Report:

While the company in its response did not make mention of the pattern in the

early 2000s, it is clear that spending dropped considerably from the year 2000 to

2001, and remained at a low level for 2002 and 2003, rebounding only in 2004

and 2005 back to its previous level and beyond. We are aware of the events that

may have contributed to that pattern, including the acquisition of IP L by AES in

2000 and the attempts by AES to take cash out of the utility operations, especially

in the financial exigency of 2003. While further discussion of the events of that

era might be interesting to some and quite colorful, we would relegate that to

historical interest and conclude our discussion of this area with the comment that

the current level of capital spending seems quite adequate for a network of this

size.

- 36 -

O'Neill Management Consulting, LLC . IURC Cause 44602 - IPL Network Investigation

6.3 Maintenance and replacement-Update

In our December, 2011 Report, we presented an extensive discussion ofIPL's maintenance and preventive replacement, all in the context of the industry's aging

infrastructure and best practices for secondary networks. We noted that

A common observation is that companies whichfail to do enough preventive maintenance find themselves ultimately overwhelmed with the volume of corrective maintenance that is required Therefore, striking an optimal balance

between the two is a key asset management decision.

At IP L the volume of corrective maintenance is sufficiently low as to suggest that preventive maintenance has not been systematically curtailed

Another term we have heard in this context is that a utility might adopt a practice of "run to failure" rather than a policy of monitoring assets with the potential for failure and

replacing them preventively in a systematic program that targets the potential worst offenders. In that regard it is worth examining IPL's preventive replacement of equipment in its downtown underground secondary network. 18

Table 7 below shows that over the last four years from 2011 through 2014, IPL has

replaced its network transformers at a rate of7.5 per year, and its network protectors at a rate of 8.25 per year. Considering that the total population of each in 2011 was 315, this represents an annual rate of approximately 2.5 percent, or at a 40-year rate. In 2015, IPL

embarked upon a program to replace all 140 of its 480V network protectors over the next three years, i.e., a 33 percent replacement rate for the 480V protectors as a group. That also escalates the overall replacement rate for network protectors to 12 percent.

Table 7 - IPL Replacement of Network Equipment

i Network Transformers ·L Network Protectors ~

Year Replace New Replace New ~ ~

2011 I 6 0 8 0 -- -

2012 8 0 7 0 i----

2013 8 0 7 0 --

2014 8 2 11 2 -- - -

2015 YTD 3 0 4 0 -

2015 Planned 0 0 38 0

Source: IPL response to DR 1.9

18 We focus our comments here on the downtown network, but we also reviewed replacement data for

the overall system and found evidence of systematic replacement rather than a "run to failure" approach .

- 37 -


As we commented in 2011, a typical industry failure rate is approximately 0.5 percent, and in any event a 12 percent replacement rate is clearly not a "run to failure" policy.

Now, given that, as we noted in 2011, IPL had allowed its network protectors to age more

than its network transfOlmers, one could argue that this represents a catching up for past under-replacement, but at that rate it goes considerably past catching up and will leave IPL with a relatively young fleet.

For network transformers, the average age in 2011 was 29 years old, with less than 6 percent of the population older than 60 years. A policy ofreplacing 2.5 percent per year would, if targeted at the oldest transformers,19 be expected to reduce the average age by

about a half-year per year, thus mildly counteracting the 'aging infrastructure' effect.20

We note that in 2015 IPL' s planned replacement of transformers is only 3 per year. This is somewhat understandable, given the significant investment in network protector

replacement over the next few years, although we wonder whether the replacement of those protectors might be the occasion for replacing at least some of the attached network transformers at the same time if their condition warrants it.

For both the network transformers and the network protectors, we would expect to see, as part of the asset management process, a document that lays out the replacement strategy

based on condition, cost, and risk. We have requested that in DR 4.5, and IPL has answered that such plans are under development as follows:

The Asset Life Cycle Plans are in the process of being developed and

implemented These plans take a holistic look at the asset class life cycle and are

expected to address items such as:

~ Description of the assets

~ Asset performance

~ Asset condition and risk (health and criticality)

~ Operating and maintenance strategies

~ Expenditures and refurbishment or replacement strategies

~ Information systems

~ Innovations and best practices

19 In fact, the policy should target those in the worst condition, not the oldest, but as there is likely a

correlation between those categories, the calculation is worth examining for the insight it provides.

20 Elsewhere, we have presented the case for developing an optimal replacement policy along these lines,

but shifting to failure rates rather than age. See: "Drawing a Line in the Sand - A 'Get No Worse' Asset

Replacement Program," EEl Transmission. Distribution & Metering Conference, Kansas City, October 13,

2009. See: http://www.oneillmanagementconsulting.com/papers/EEI%20TD&M%20Fall%2009%20Drawing%20a%20Line.ppt

- 38 -

o 'N eill Management Consulting, LLC lURC Cause 44602 - IPL Network Investigation

A meeting has been scheduled in June 2015 for the implementation leads to

discuss the implementation schedule for the asset classes shown in the table

below:

Table 8 - Status of Asset Life Cycle Plans

Asset Class Implementation Status

Wood Poles Draft completed - 12/4/2014

Relay System Protection Draft completed - 11/25/2014

Circuit Breakers Draft completed - 12/15/2014

Power Transformers Draft completed - 2/27/2015

Downtown Network Draft completed - 2/26/2015

Underground Residential Data gathering and draft under development - 2015 Q3 Cable (URD) Overhead Distribution Lines Data gathering and draft under development - 2015 Q4

Transmission Structures Data gathering and draft under development - 2015 Q4

Meters TBD

Substation Batteries TBD

Transmission Lines TBD

Substation Communications TBD

Distribution Transformers TBD

System Control and Data TBD Acquisition (SCADA)


We requested (DR 6.4) a copy of the document for the downtown network, which the response in the above table indicates was completed in February, 2015. We received a

draft dated March 23,2015 which was still obviously incomplete, although it had 66 pages of content (as a combined DP&L and IPL document, with separate discussion of each).

With respect to the two other major categories 'of downtown network equipment, primary

and secondary cable and splices, the replacement data for the last four years is as follows:

Table 9 - IPL Underground Network Cable and Splice Replacement

Year 2011 2012 2013 2014 2015 Est.

Primary Cable (feet) 24,229 20,521 24,821 27,491 20,000

Secondary Cable (feet) 20,292 4,364 3,712 4,052 10,000

Primary Splices 607 257 430 528 600


- 39 -


For primary cable, for 2011-2014, the average replacement was 4.6 miles per year, which on a population of 69.5 miles represents a 6.6 percent annual rate of replacement, or a 15-year program. For secondary cable, the replacement has been more lumpy, with large amounts of replacement in 2011 and planned for 2015, and an average level of .76 miles

per year for 2012-2014. On a population of 122.8 miles of network secondary cable, .76 miles represents a paltry .6 percent rate of replacement. This would essentially constitute a "run to failure" program. At 10-20,000 feet per year, or 1.9 to 3.8 miles per year, the rate rises to 1.6 to 3.1 percent per year. We look forward to seeing the asset management

strategy that determines the rate of secondary cable replacement based on failure, cost, and risk.

6.4 Failure analysis-Update

In the Executive Summary of our December, 2011 Report we provided an analogy of how good failure analysis, followed by the appropriate remedial action should work in a utility just as it does in other public safety contexts:

An analogy can be made to modes of transport such as driving a car, riding in an airplane, or walking the sidewalks. In those modes, the risk is managed by a process that, when it works well, investigates every significant instance of equipment or operator failure, tracing its root cause, and then mandating appropriate remedial action, which might be a traffic citation, an equipment recall,

or a mandated change in the design of cars, airplanes, or sidewalks and crossings. As new modes of failure are discovered, new remedies are put in place.

Our findings, detailed below, indicate that, based on our examination of the recent and past incidents and our audit of the current system condition and practices in comparison to those of other utilities, there is an immediate need to improve the process by which IPL finds, documents, and remediates failures in its electrical system in downtown Indianapolis. The recommendations contained in this repmi,

if acted upon, can ftniher reduce the risk of service interruption, collateral damage, and possible injury associated with the system. [Emphasis in original]

We went on to discuss the history and current state of IPL' s failure analysis as it peliained to the underground secondary network, citing examples involving network protectors and toluene gas and also aluminum bus protectors and the danger of hydrogen gas. We then went on to say:

When we began this study, we asked IPL for detailed analyses of the previous failures. We were told that they did not exist in a useable fmID, other than the write-up presented to the IURC at the July 7 meeting (Appendix A). This is a

- 40-


situation that needs to be remedied, and we planned early on to include that as a finding in our report. As it turns out, just before the report was due, we had an opportunity to observe firsthand the current IPL process for researching and documenting the root cause of a significant incident.

This was the 19 Nov 2011 incident at 24 S. Delaware and E. Pearl St., where a secondary cable fault due to an old steam leak caused an explosion and fire, and damage to a parked car (with no one in it at the time). As this happened while we were conducting our audit,

we had the opportunity to observe directly IPL's root cause analysis process as it was beginning to be improved?l We commented at length on this particular incident in our December, 2011 Report.

Since then we have reviewed IPL's root cause analysis of each subsequent network

incident and we have seen a marked improvement in the quality of the analysis.22 Each report is scoped in accordance with the severity of the event, and the process of gathering the necessary evidence has been what we might expect.23 While it is not always possible to definitively proclaim the ultimate root cause failure that causes a network incident, since some of the evidence is destroyed as a result of the incident, we believe IPL has

learned to do a goodjob of identifying the most likely candidates and the appropriate remedial program response.

We feel that this capability should go a long way toward producing the kind of effective, transparent process that can give the IURC and the public a sense of confidence that the process being followed to manage the risk of network incidents is like that described in our 'modes of transport" analogy. As each incident is examined and the evidence made public, another degree of confidence is established. Of course, the true test will be when the remedial programs are effective in reducing the number of incidents and their severity.

What we would expect to see eventually obtain is a situation where, as the saying goes, "the exception proves the rule". That is, we would hope to see that any new incident

21 In preliminary discussions with IPL during our audit, IPL personnel recognized that we would be

recommending an improved failure analysis process, and had begun to formulate one.

22 For the root cause analyses, such as they were, for events prior to May 1, 2011, see IPL's answer to

question 2 in Appendix B ofthis report (IPL's July 7,2011 meeting with the lURe). For later root cause

analyses, see the IURC website page: http://www.in.gov/iurc/2825.htm. and click on the links to

documents. For the events in 2015, see supplements Sl and S2 in Jim Sadtler's supplemental testimony in

Causes 44576/44602.

23 The one exception is that IPL has not always been effective at sequestering the material recovered and

labeling it in such a way as to permit subsequent analysis IPL is aware of this shortcoming and has

pledged to improve in that regard.

- 41 -


would be so idiosyncratic in its nature that it would demonstrate that any normal event would have been avoided. To some extent, we see something like that in the 19 Mar 2015 event, where the steam damage to the secondary cable was not the result of a steam leak that the remedial program established after the 19 Nov 2011 incident was designed

to prevent, i.e. the joint IPL-Citizens Thermal monthly review of hot manholes and steam leaks, but rather was the result of a previous leak that occurred before the steam line was retired.

- 42-


7 Organization and Staffing

7.1 Work planning and resources-Update

In our December, 2011 Report we noted the history of the current manner in which work

on the downtown secondary network was organized, namely,

For many years IPL had a dedicated set of crews to work on the downtown

underground network system. In 1994 IP L developed a program to create a

position, beginning with an approved apprenticeship, of a substation mechanic

who would be qualified to work in the substation or in the underground network

system. While this approach gave the company some flexibility in staffing major

work, the sense we got is that there is still a significant specialization among the

crews in terms of which mechanics work on the downtown underground network.

In the section that typically works in that area, there are 14 people, including one

section leader, six crew leaders, and seven mechanics, who typically work as 4

three-person crews, and sometimes as two-person crews for network protector

maintenance. In addition, there are three more sections with crews qualified to

work on the underground if needed. For the most part, they work on substations

throughout the IP L system.

For our update, we asked for the current level of staffing and received the following data

(from DR 1.11):

Table 10 - Staffing levels for substations and downtown network

Year Skilled Craft Exempt Total I-

2011 47 7 54 I- - ~--

2012 47 9 56 - --

2013 48 9 57

2014 48 9 57 r-

2015 (as of 4/15/15) 47 9 56 - -- - -The Skilled Craft count is currently 47 but will return to 48 on Monday, April 20th due

to an internal job transfer.

From this we judge the staffing to be virtually unchanged, although this has been

accomplished with some tumover and re-assignment.

In the discussion of the root cause analysis for the August 20,2014 incident at 26 S.

Meridian the IURC raised a concem that the transitions could have. been foreseen and the

training of replacements could have been made more effective by having the retiring

- 43 -


employee shadow the replacement. IPL asserted that it had done a staffing study, which

was in need of updating after the merger with DP&L, and that the issue would be addressed. The IURC should follow up to make sure this commitment is met.

In our December, 2011 Report we also did analysis of the work requirements to

accomplish inspections versus the availability of experienced, dedicated crews, and we concluded the staffing was sufficient. In addition, we noted that IPL makes use of contractor crews for much of its new construction and special project maintenance, like the "full sweep" inspections done· in 2005,2011, and 2015.

- 44-


8 Technology

8.1 Technology and secondary networks

In our December, 2011 Report we noted that while'underground secondary networks in

Indianapolis and the rest of the nation are based upon technology that is for the most part many decades old, 24 there are a number of relatively new innovations that represent

opportunities to apply technology to the design, operation, and maintenance of

underground secondary networks. At that time we listed the following seven areas worth

investigating, and we gave our recommendations for each:

• SCADA or RMS

• Loadflow modeling

• AMlFM or GIS

• Thelmal imaging

• Fault direction indicators

• Vented, locking, or lift/locking manholes covers

• Remote data capture for inspection results via handheld devices

IPL has generally shown an interest in applying technology to its system in order to

improve performance, often with a side benefit of potentially reducing ongoing costs as

well. Accordingly, IPL agreed with all of our recommendations regarding the application

of technology in these seven areas and began or continued its programs for adoption.

In the sections immediately below we provide an update on the progress of each. See the

December, 2011 Report for a more extensive discussion of the purpose and benefits of

each. We have retained the original order here, but would note that what we called then

the lift/locking manhole covers, and which we now refer to as locking/pressure relief

covers, are probably the most significant item in this list. (See Section 8.7)

8.2 SCADA

SCADA (Supervisory Control and Data Acquisition) is a technology whereby utilities

remotely monitor equipment conditions and remotely control operation of devices like

switches to respond to conditions. While SCADA has been widely deployed by utilities

for years at the substation level, only some utilities had deployed it to monitor the

. 24 Even brand new equipment is often based on technology that was developed in the first half of the 20th

century and refined only minimally in the second half, with the exception of the items we mention in this

section,

- 45 -


network transformers and network protectors of underground secondary networks. In 2011, IPL won a Department of Energy grant to assist IPL in the deployment of SCADA on its secondary network. That system is now fully deployed. In the August, 2014

network event at 26 S. Meridian it became apparent that telemetry errors sometimes prevented that system from providing the full extent of monitoring and control for which it was designed, and IPL agreed to a monthly monitoring of telemetry errors. In one of

our data requests (DR 5.8), we asked for data on those errors and received the following:

Cumul.tive Telemetry Error Rate By Month XIl6 ,------ --------------------

i 8 . ~~o I---------~----------------------

i a i~o ~~-------~--------------------

i !.w!O i g

11· 3CIlo &. D ~ !~ r_--~r_r_~~I----~-,r----~r_------~

~ F-----~~~-~-----~~--r_----~-~~

We were pleased to see the error rate greatly reduced after we called IPL' s attention to the problem in reviewing the August, 2014 network incident.

In our December, 2011 Report, we recommended that IPL's asset management group

take advantage of secondary network SCADA data to achieve certain efficiencies, specifically, we listed the following:

• Understand, resolve and leverage lessons learned from any connections issues associated with the current roll out. Currently IPL has 190 network relays upgraded/installed with 68 currently providing SCADA communication.

• Opportunities to create real time reports. Examples we have seen at other utilities include rep011s revolving around transformer temperature. For example Tier 1 rep011s indicating which units are >20 degrees below nameplate rating with notification via email, Tier 2 reports which are operating at or above name rating, with notification via text to cell phones. Also utilities have created almIDs points around protect01;s pumping (open and closing several times a minute - indicating a faulty relay)

- 46-


• Metrics and scorecards on real and historical conditions. E.g. number of protectors not closed or opened in the previous six months, current loading conditions, number of manholes with water, etc.

• Other information, driven by system failure rates and system operating conditions. Such as Voltage, Current, Open/Close, but also Temperature, presence of water, intemal component pressure (Transformer/Protector), etc.

• Real time modeling ofthe system. For example during summer heat waves or during heavy rain periods.

• Optimize the maintenance/repair process via creating new targeted maintenance programs. For example some utilities that have a corrosion issue on the bottom of the transformers monitor manhole moisture along with intemal transformer pressure/oil levels to determine certain transfOlmer locations that require additional field visits. For IPL, this may be monitoring water presence in the protector, protector open/close counts, and loading of adjacent transformers and closed protectors to determine a prioritized list of locations to be inspected.

• Opportunities, cost and benefits of extending the remote monitoring to within the secondary grid e.g. to those secondary/primary manholes exhibiting characteristics of previous failures such as proximity to high temperature sources (steam). Currently the effort at IPL has focused on the transformer/protector vault locations.

As we said in the previous section, we believe that IPL's asset management function has

incorporated some of these opportunities but is not fully utilizing the SCADA capabilities

in terms of translating the potential for asset management into systematic reporting and processes to act on the information provided.

8.3 Loadflow modeling

In our December, 2011 Rep0l1, after discussing the industry's eff0l1s to apply loadflow

modeling to secondary networks (which are intrinsically more difficult to model than

radial distribution), we recommended the following:

IPL's current eff0l1s to use CYMEDIST to model its networks should be

encouraged.

It is our understanding that IPL did continue to work on this objective, but has not yet

succeeded in getting a full set of acceptably accurate network loadflow models. We note

that they are not alone in that status, as other utilities we are aware of have also found it

difficult to make the CYMEDIST model accurately model their secondary network flows.

- 47 -


We continue to encourage this effort, and IPL has indicated its intention to do so, especially with the personnel currently in charge of network engineering.

8.4 GIS/facilities records

As was the case for network loadflow modeling, in our December, 2011 Report we cited the difficulties other utilities were having in getting their secondary networks accurately portrayed in their geographic information systems (GIS's). We made a similar

recommendation:

Nevertheless~ as technology has become easier to apply, at this time almost all major companies have begun to put their underground secondary networks on GIS,

and IPL should continue its efforts in this regard.

And we make a similar recommendation now that IPL be encouraged to continue to make

progress toward this objective.

We do note that in the area of facilities records, IPL has improved its records for nameplate data on its network protectors, as evidenced by the much lower percentage

with unknown dates of installation/manufacture.

8.5 Thermal imaging

In our December, 2011 Report we noted with approval that IPL was beginning to take advantage of the less expensive hand-held devices for thermal imaging. We quoted from

IPL's July 7, 2011 report to the lURe:

As the thermal imaging technology has matured, the size of the equipment has decreased significantly. There are now hand-held devices about the size of a large

flashlight that can be used in confined spaces. IPL has purchased two hand-held infrared cameras, which it is piloting in the full sweep of manholes and vaults. Test inspections with the cameras began on June 3, 2011. Initial results have shown that the cameras can be effective on connections and terminations.

We now note with approval that IPL has integrated the use of such devices into its vault

and manhole inspections.

8.6 Fault direction indicators

In that same July 7,2011 report, IPL committed to a program of piloting and, if successful, implementing fault direction indicators on network primary feeders in order to

accelerate fault location. We again quoted IPL's commitment:

- 48 -

O'Neill Management Consulting, LLC lURC Cause 44602 - IPL Network Investigation

"The purpose of this initiative is to install a device on the primary network cables that detects when fault current has passed through it. This device is called a fault detector or fault indicator. Fault detectors are used in an underground system to help determine the location of the fault. IPL will be able to read the device

remotely from street level without having to enter the manhole. This will result in a considerable time savings to locate a cable fault and will reduce overall feeder restoration time. This project is in the design phase. Equipment purchase is

expected to commence first quarter 2012."

IPL went on to fully deploy fault direction indicators on all of its network feeders, and has noted their effectiveness in reducing the time to locate feeder faults.

B.7 Vented, locking, or lift/locking manholes covers

In our December, 2011, we reviewed the industry's history on the issue of using manhole cover design to improve the performance and safety of underground network systems, and ended with a recommendation that IPL selectively adopt the Swiveloc™ cover. We noted that there were potential issues with the normal operation of the cover that might need to be worked out, and since the technology was actually relatively new and, while having been thoroughly tested and proven effective in improving safety, were still not widely deployed in the industry, which might warrant some caution and selective

deployment.

After deploying 75 covers in the area of Lucas Oil Stadium and Georgia Street prior to the Super Bowl in February, 2012, IPL did find a problem in operation. The problem was not in the operation ofthe pressure relief aspect which is used to improve safety, but in the locking mechanism used for security. Crews had a tendency to apply too much

force in setting the lock, which actually only needed a hand wrench. IPL worked with the manufacturer to change the design and retrofit (at the manufacturer's expense) the

improved design to existing covers.

IPL has now committed to deploying the Swiveloc ™ manhole cover on all of its

manholes in the CBD by the end of 20 15. When it does that, it will be the only company in the nation to have them deployed on 100 percent of its CBD manholes. We would have been content for IPL to adopt a three-to-five year implementation plan, such as the five-year plan adopted by Pacific Gas & Electric in San Francisco and Oakland, but we

can hardly fault IPL for being overly aggressive in this safety-related program.

- 49 -


8.8 Remote data capture for inspection results via handheld devices

In our December, 2011 Report we recommended that IPL adopt a technology for

electronic capture of inspection data using intelligent software on hand-held devices. IPL

agreed with the recommendation, but chose at first to attempt implementation using a

non-intelligent Ivara data entry screen. By the end of2012, however, IPL had switched

to the MobilFrame platform that we had cited as an example of the type of software we

were recommending. That system has worked well. In the section on asset management

above we note that we suspect that even more integration and intelligence can be built in,

but we are pleased to note the progress thus far.

- 50 -


9 Findings and recommendations

9.1 Findings

In this section we suinmarize our key findings and conclusions, taken from the previous sections of the report (and as further summarized in the Executive Summary).

1. We continue to feel that the basic design, maintenance, and operation ofIPL's downtown electrical network is sound, and that the risk to the citizens of Indianapolis

is low. As we said then:

"There is risk, but that risk has been, until recently, acceptably low."

The caveat, "tmtil recently" that we voiced then is, of course, at issue once again, and our further findings address that concern, in that they examine whether the recent

higher frequency represents in our view a permanent shift to a higher level of risk or a temporary run of occurrences occurring against a background of low risk. We judge it to be the latter, for reasons explained below, but we judge that there are two actions, in addition to those to which IPL has already committed, that we would

recommend in order to allow the IURC to be able to have confidence in and to monitor that acceptably low level of risk.

2. We feel that IPL has been very responsive to the situation, and has not hesitated to expend resources to investigate the incidents and their possible root cause and to commit to action plans that will address concerns that they had and which we shared. We see no evidence that IPL at this time is shrinking from making the necessary commitments due to their intrinsic cost or perceived financial concerns at IPL's parent company, AES Corporation. We feel that the process by which some of these

commitments were made deserves further transparency, as we elaborate in our fifth finding below.

3. The root cause of the most recent incidents (five from March 2014 through March 2015) is not anyone problem, but rather four separate problems involving separate

aspects of the underground network:

a. Network transfOlmers (especially, their telmination chambers),

b. Network protectors (particularly, the 480V models used for spot networks),

c. Secondary cable (the 120V grids in the streets of the CBD, especially the oldest cable if it has been exposed to steam and high levels of road salts to prevent ice

caused traffic hazards).

- 51 -


The fourth major component, primary cable, is performing well, except when it is collaterally involved in one ofthe other categories listed above, and represents a low

risk because of the way that the feeder breaker relays at the substations greatly limit the amount of energy that primary faults can involve, almost always keeping the damage confmed to an underground structure without spilling over into the public space.

4. The solutions to the root causes of the recent incidents lie in programs to which IPL has already committed to implementing with reasonable speed. These include:

a. Filling the termination chambers of network transformers with fire retardant

insulating fluid as we had recommended in 2011 and which IPL completed by the end of2012. As a result, the 16 March 2015 event that involved a failure in the termination chamber resulted in only a momentary release of energy, enough to make a noise and raise the manhole cover, but not involving an extended episode of smoke and fire. In that regard, the event was comparable to the faults on

underground primary mentioned in the previous paragraph which occur at IPL and other utilities with some regularity but with little significant risk to the public. Moreover, the small risk that might be involved in the displacement of the cover itself is being addressed by IPL through a program of installing locking/pressurerelief manhole covers in all of its CBD manholes by the end of2015, making it, to our knowledge, perhaps the first utility of size in the nation to have 100 percent of its manholes so protected.

b. Replacing in the next five years all of the 480V network protectors with newer models that will greatly reduce the risk of failure for that type of equipment. Since that equipment inherently involves higher energy than the 120V network protectors, it is appropriate that these should be prioritized. As it turns out, their replacement also addresses revised OSHA rules for operating equipment that has

the potential to produce an arc flash under certain circumstances while an employee is working on or near the equipment.

c. Secondary cable explosions and fires will have their frequency addressed by an enhanced program of looking for elevated temperatures in ducts where they cross steam lines that may be or may have been leaking, causing damage to the insulation of secondary cable; and the severity of secondary cable incidents will be significantly reduced by the program of locking/pressure-relief manhole covers

that have been shown by independent research to significantly reduce the magnitude of underground explosions by controlling the flow of oxygen to the explosion; and, of course, the locking aspect keeps the covers from flying through

- 52 -


the air and injuring someone as they fall. In addition, IPL has budgeted a

reasonable amount of preventive replacement of secondary cable each year.

5. Notwithstanding our sense that IPL has been quite responsive to the situation and is

taking appropriate steps, we do see room for improvement in two areas: asset

management and regulatory performance monitoring. With respect to the former, we

note IPL's significant improvement in asset management methods and procedures

since implementing our fifth recommendation of the December, 2011 Report.

Nevertheless, we feel there is not sufficient transparency in IPL's process of asset

management to allow the IURC and the concerned public to see how IPL's

responsiveness is reflective of a systematic program of asset management. We

recognize that IPL continues to improve its asset management process, yet we think

the time has corne to also make the effort to document to the outside in some detail

the process by which the asset management function serves to address the risk and

performance of the system, all in the context of cost effectiveness.

6. Our final finding is that we feel that the process whereby IPL responds to regulatory

pressure every time that the number of CBD network incidents raises public

awareness is not a good long term solution. In saying this we want to reiterate that

we laud IPL's responsiveness and much prefer this situation to one in which the

utility has to be coaxed and goaded into even being responsive to public events. Still,

we feel that in the long run both the public and IPL would be well served to have a

regulatory reporting process in place that would allow the IURC to have confidence

in IPL's system and management enough to not have to react to each event as a crisis.

The IURC has been duly deliberate in responding to events, being content to see a

thorough root cause analysis for events that seem isolated and not severe, but it has

had to exercise that restraint and good judgment without the benefit of a reporting

mechanism that could more effectively support a trust-and-verify relationship

between the regulator and the utility.

In the Executive Summary of our December, 2011 Report, we drew an analogy which

may be worth repeating for its insight into the CUlTent situation:

An analogy can be made to modes o/transport such as driving a car, riding in an

airplane, or walking the sidewalks. In those modes, the risk is managed by a

process that, when it works well, investigates every significant instance of

equipment or operator failure, tracing its root cause, and then mandating

appropriate remedial action, which might be a traffic citation, an equipment

recall, or a mandated change in the design of cars, airplanes, or sidewalks and

crossings. As new modes of failure are discovered, new remedies are put in

place.

- 53 -


We continue to encourage IPL and the IURC to develop their relationship such that as

IPL executes an agreed-upon process in a transparent way, it gives the IURC confidence

that the risk to the citizens of Indianapolis is being effectively managed to an acceptably

low level.

As we did in December, 2011, we advise that the benefit of our having done the work on

which we report here will only be reaped if the IURC and IPL agree on a specific,

monitored action plan to implement those commitments which IPL has made as well as

any that flow from our current report and which IPL finds it can wholeheartedly adopt (or

modify by mutual agreement).

Finally, we affirm as we did in December, 2011 that IPL cooperated fully in responding

to our requests for information and in making available to us in a timely way its staff and

its facilities for our interview and inspection. We wish to have placed into the record of

this docket those IPL responses to IURC data requests which are referenced in this report.

9.2 Obtaining commitment for implementation

In section 9.2 of our December, 2011 Report, we commented:

Typically, after a report of this type, regulators and utilities agree on a quantified

list of commitments and a schedule of status reporting on the progress of the

commitments. No doubt such a process will take place in this instance as well,

which will allow the recommendations made in this report to improve the

performance of the Indianapolis downtown underground system.

That expectation was met, in that the report was followed by a subsequent development

of a detailed action plan that was monitored by us and the IURC over the year following

January, 2012, with a final report by IPL in January, 2013.

We have noted that IPL has made many commitments to various programs which will be

implemented over the near-term and longer-term future. We would expect that IPL and

the IURC will continue to monitor IPL's progress in meeting its commitments.

We also look forward to working with IPL in the next few months to address the

concems addressed in our fifth and sixth findings regarding the transparency ofIPL's

asset management process and the metrics by which IPL' s performance is measured.

- 54 -


9.3 Recommendations

We conclude this report with three recommendations:

1. IPL and the IURC should continue to document and monitor IPL' s commitments to implement actions intended to address the concerns of this investigation. The

list in Section 3.2 is a good place to start, plus any more recent commitments, such as those from the root cause analyses of the events in March, 2015, or

commitments made in respons~ to data requests from this investigation.

2. The IURC should order that a further investigation (audit) be done ofIPL's asset

management process in order to clearly document which aspects of that process are fully developed and operational versus which aspects are still in progress. For those items in progress, a schedule of intended implementation should be obtained. It is hoped that this greater transparency will lead to the IURC's being

able to have confi~ence in IPL's execution of its asset management process.

3. The IURC should order that IPL enter into discussions with the IURC concerning the design of a set of performance metrics which could be used to avoid the process of opening new investigations with every incident of perceived poor performance. Such metrics could ultimately be part of an explicit incentive mechanism, although we suspect that a certain amount of reporting and revision

may be necessary before the metrics would be stable enough to become part of such a mechanism, and in any event the mechanism itself should be open to modification and revision over time as experience is gained with it.

- 55 -


Appendix A - IndyStar list of incidents since 2010

14 underground fires or explosions downtown since 2010

Jill Disis,[email protected] 8:26 p.m. EDT March 19,2015

At least 14 underground explosions or fires have been reported in Downtown Indianapolis since 2010. Here's a look at those incidents.

March 19,2015: A series of explosions Thursday morning sent several manhole covers

hurling into the air near Capitol Avenue and North Street. Power was shut off to up to six square blocks, including the heavily trafficked Meridian Street Downtown. IPL officials said the explosions were caused by a short circuit and not because of aging infrastructure. Utility regulators called an emergency meeting to address the explosions.

March 16,2015: Police closed down a portion of Massachusetts Avenue near East New

York Street after reports of an explosion in front of Starbucks Coffee. IPL said the problem apparently occurred in a box in front of the transformer called the primary termination chamber. That's where cables from under the street enter the transformer and begin the process of reducing power from 13,000 volts to 120 volts.

Aug. 13,2014: A series of underground explosions rocked Downtown Indianapolis on South Meridian Street between Washington and Maryland streets. IPL officials said the source of the problem was identified as the failure of an underground network protector. While no one was injured, power had to be shut off to many nearby buildings so repairs could be made.

March 14,2014: A manhole explosion was reported near Delaware and Market streets downtown, causing traffic to be blocked off.

July 15, 2012: An underground cable in a manhole shorted out on East 16th Street Downtown. When firefighters aITived, they found smoke coming from the manhole.

July 5, 2012: Pressure from an underground cable caused a manhole lid to slide in the

300 block of Massachusetts Avenue Downtown.

July 4, 2012: Flames erupted fi·om an underground electrical vault near the Westin Hotel Downtown, causing flames to shoot up from a sidewalk grate on South Capitol Avenue.

- 56 -


Jan. 16,2012: An explosion and fire broke out at IPL's Center Substation near West and Morris streets. The fire was caused when a breaker failed, and caused nearly 10,000 IPL customers to lose power.

Nov 19,2011: A series of underground blasts near Bankers Life Fieldhouse, then named Conseco Fieldhouse, caused flames and pillars of smoke to shoot up from manholes. The

fire heavily damaged a vehicle parked near a manhole, and a cast-iron manhole cover was blown several feet into the air. IPL officials said they spliced underground cables to repair the damage, which did not result in any customers losing power. The blasts sparked concerns over whether Downtown would be safe enough for visitors to

Indianapolis during the 2012 Super Bowl game. IPL officials said they purchased 100 manhole covers to lock in place so they would not dislodge. Fifty of those new covers were placed along Georgia Street, the home of the Super Bowl Village.

May 31,2011: An explosion outside Gov. Mitch Daniels' office rattles windows at the Statehouse. IPL blames a faulty underground cable to a transformer at 46 N. Capitol Ave., which ignited a small reserve of oil in a connector box. No injuries are reported.

April 27, 2011: An underground electrical explosion blows a manhole cover into the air

Downtown on Court Street. IPL says a 900-foot stretch of electrical cable apparently short-circuited and burned. Smoke is reported in the basements of at least two nearby buildings. No injuries are reported.

Jan. 30, 2011: Multiple underground explosions rock the 400 block of East Michigan Street near the Athenaeum, blowing at least five manhole covers off their bases. IPL says electrical cables caught fire but does not know whether they were the cause or merely a

casualty of the explosions. No injuries are reported.

Aug. 18,2010: An underground explosion in the 500 block of Massachusetts Avenue sends flames flying 25 feet into the air. IPL says a transformer exploded, caused by oil burn-off. No injuries are reported.

Feb. 17,2010: An underground fire shoots flames high into the air Downtown near the entrance of Charlie & Barney's restaurant at Delaware and Ohio streets. Witnesses say they heard a loud boom. The fire forces the evacuation of about 1,000 workers from the nearby Gold Building - some of them walking down 20 flights of stairs. Fire officials sayan electrical transformer near the restaurant entrance short-circuited arid caught fire.

The privately owned transfOlmer is not linked to the IPL grid. No i~uries are reported. Damage is estimated at $75,000.

Star researcher Cathy Knapp contributed to this report.

- 57 -

O'N eill Management Consulting, LLC IURC Cause 44602 - IPL Network Investigation

Appendix B - IPL's July 7,2011 Response to the lURe

Response of Indianapolis Power & Light Company ("IPL") to Indiana Utility Regulatory Commission Questions for July 7, 2011 Public Meeting

1. As part of your presentation on July 7TH please explain how the downtown network system works, in terms of system protection. Explain the various components and their characteristics (cables, transformers, circuit breakers, etc.) and what happens when various components fail, as · well as what causes explosions, fire, and smoke. Explain the interface with customer-owned equipment.

IPL Response:

A secondary network system has been recognized for many years by the utility industry as an economical way to provide a high degree of reliability to downtown areas with a high concentration of loads. A secondary network consists of a grid of cables operating at the same voltage level with multiple sources feeding into the grid. In IPL's case this is 120/208 volts. This grid is energized at numerous supply points from network transformers served from multiple 13,800 volt primary feeders . The system is designed so that the loss of anyone component will not cause a loss of service. The key to the design of the secondary network system is its redundancy.

Components

The components that make up a secondary network system include:

Manholes: These are used as junction/splicing points for the underground cables. Workers physically enter these structures, which vary in size but are typically 5 feet wide by 10 feet long. Manholes are considered a confined space and require additional safety precautions to be followed for entry.

Conduit system: This is the pathway between manholes and vaults for the underground cables; it is also referred to as a duct line. These are typically fouror five-inch diameter conduits made of clay tile, fiber material, or PVC plastic. The conduits are arranged in groups of four to twelve individual conduits. The group of conduits is then encased in a concrete envelope for added physical protection.

- 58 -


Transformer vault: These structures house the underground network transformers. Typically a vault consists of one to four compartments also referred to as bays. The size of the bay is typically 10 feet by 20 feet. Each bay generally contains one network transformer. A grating in the vault roof provides ventilation and an access door allows personnel entry into the vault. Most vaults contain openings between the bays that allow a worker to move from one bay to another. There is a collector bus comprised of individual copper bars to which cables connect · that runs the length of the vault through each bay. The bus is supported from the ceiling of the vault by insulators. The output of the network transformer connects to this collector bus. Service cables to customers and network secondary cables between vaults also connect to the collector bus. The vault is considered a confined space and requires additional safety precautions to be followed for entry.

Primary cables: IPL's primary network cables operate at 13,800 volts. These are installed in the manholes and conduits from the source substation to the underground network transformers in the vaults.

Secondary Cables: These cables are operated at 120/208 volts and connect transformer vault collector buses to other transformer vault collector buses through manholes and the conduit system. Customer service cables may also connect to the secondary cables in the manholes through which secondary cables pass.

Cable limiter: This is a protective device similar to a fuse. It is used to isolate cables that have experienced a failure from the secondary network system. IPL also uses cable limiters on certain size service cables that connect to the secondary cables. These may be installed in a vault or in a manhole. Cable limiters will clear short circuit conditions or faults that have high sustained cunent flowing through them. Cable limiters will not reliably clear an arcing fault where the cunent is fluctuating rapidly from high to low.

Network transfOlmer: The underground network transfOlmer brings the voltage down fi:om the primary voltage of 13,800 volts to the secondary level of service voltages of either 120/208 volts or 277/480 volts depending on the application. Network transformers can range in size from 500 KVA to 2,000 KV A. There are three pm1s to a network transfOlmer. First there is a primm'y termination chamber where the underground primm'y cables connect to the network transformer. Right below that is a primary switch compartment, which contains a three position nonload break switch. Those positions are Open, Closed, and Ground. The ground position is used for safety when personnel are working on the primary cables.

- 59 -

O'Neill Management Consulting, LLC lURC Cause 44602 - lPL Network Investigation

The third compartment is the main tank, which contains the core and coil assembly of the transformer. All three compartments contain insulating oil and are separate from each other.

Network protector: This device connects to the low voltage secondary terminals of the network transformer. It acts like the circuit breaker in a horne except it operates on the direction of power flow rather than on the magnitude of fault current. The network protector is designed to open if power tries to flow from the secondary network system back through the transformer into the primary cables. This flow could occur when a primary cable fails and the substation circuit breaker opens to de-energize the primary cable. The network protector opens to stop reverse cun'ent flow from the vault collector bus back through the transformer and out through the primary cable to the fault. There are also fuses in the network protector that will open and isolate a fault on the secondary side of the network transformer to prevent a possible failure of the network transformer. The network protector is a critical component that allows a secondary network system to function.

Secondary network vault: IPL's secondary network vaults are operated at 1201208 volts and connect through secondary cables that tie multiple network transformers together.

Spot network transformer vaults: These vaults operate at 277/480 volts and serve a single building. Typically, the vault collector buses are not connected to other 277/480v buses.

System Configuration

The Indianapolis Downtown Network system is divided into five independent secondary network areas. Each of these secondary network areas is served by four to five primary network feeders from one of three substations that serve the downtown area. There are no ties between the secondary network systems. They each operate independently from one another. There are no ties between the primary network feeders, which operate independently from each other.

Causes of IntelTuptions

The primary and secondary cables used for the Downtown Network system are very reliable. The insu~ation systems of these cables have shown they will perform well for more than 50 plus years with some lasting over 100 years. Because ofthe long, expected

- 60 -


service life, the cable manufacturers do not publish an expected service life value. The connection points or splice points are inherently the weakest points in any underground system. As such, these are the areas where failures are most likely to occur. These are the weak points because these connections are made in the manholes or vaults. While workers take great care to keep contamination and moisture away from the splice work being performed, they are still working underground in a harsh environment. The splicing technology has improved over time. Since the early 1990's, IPL has used heat shrink technology for most of its splices. These splices are easier to install and have greater tolerance for field conditions.

Voids (air pockets) or contaminants in the splice are common causes of splice or termination failure. Either of these will cause areas of high electrical stress in the splice and, over time, eventually lead to a failure of the splice.

Downtown Indianapolis also has an active steam heat pipe system, which contributes to failures on the electrical network due to excessive heat. Heat radiating from steam lines with deteriorated insulation or steam leaking from steam pipes near electrical facilities will cause degradation of the components of the electrical network system. Excessive heat causes splice materials and cable insulation to deteriorate and eventually lose their insulating properties, resulting in an electrical failure.

Road salts and other ice melting chemicals can cause corrosion and deterioration of the electrical .components as well. This can allow moisture to infiltrate the electrical components and can lead to a failure.

Failure of customer owned equipment connected to IPL's secondary network system can also lead to problems and failures on IPL's components. This can be at the primary voltage level of 13, 800 volts or at the secondary voltage· level.

Event Sequences

The network system is designed with redundancy in order to provide continuous service to customers when a system abnormality, such as a primary cable failure, occurs.

When a primary cable failure occurs, the protective relays at the source substation detect a problem and signal the substation circuit breaker at the beginning of the primary cable circuit to open. For underground network feeders the circuit breaker opens typically within 0.1 seconds and stays open. The network protectors on the transformers connected to the faulting primary cable circuit will immediately open on reverse power flow from the secondary network back toward the primary network. This isolates the

- 61 -


faulted feeder from the rest of the network system and results in no customer outages. The System Operators in the Transmission Operations Control Center are signaled through the Energy Control system that the circuit breaker opened. They dispatch the

appropriate personnel to respond.

When a network secondary cable failure occurs, fault current will flow toward the fault from both directions. The fault current will melt the cable in two at the short circuit point thus clearing the fault. Most secondary cable faults are isolated without incident.

Infrequently, secondary cable faults can result in a fire or an over-pressurization event.

An over-pressurization event can occur when the levels of carbon monoxide and other combustible gases build up in the manhole above their lower explosive limit. The

combustible gases can include natural gas, methane gas from the sewers, and other gas created from the smoldering or burning of the cable insulation. The most prominent gas generated from burning cable insulation is carbon monoxide which has a lower explosive limit of 12 percent. If there is an ignition source such as arcing at the short circuit point,

the combustible gases in the manhole can be ignited. The pressure in the manhole will build rapidly and can dislodge one or more manhole covers.

Interface to Customer-Owned Equipment

IPL owns and maintains the network primary and secondary cables, vault, network transformers and other associated equipment, including the metering equipment. The customer is required to extend their secondary service cables to an IPL manhole or vault. The demarcation point between IPL and the customer is the point where the customer's

cables connect to the IPL distribution network. The details on service requirements for customers are provided in IPL's Electric Service and Meter Manual (a.k.a., "The Gold Book,,).25 This book provides details about IPL's requirements for service, the

customer's responsibilities, and how the metering equipment will be installed. IPL

requires an inspection celiification from the City of Indianapolis Electrical Inspector before a service will be connected and energized.

2. Please provide the root cause of each over-pressurization event with underground utility plant that occurred on Janumy 8, 2005 and any other event that has occurred through May 1, 2011, including events on August 17, 2010, August 19, 2010, January 30, 2011, and April 27, 2011. Also, please explain. For plllposes of identifying the actual number of pressurized events in the downtown area during this

25 This publication is available at http://apps.iplpower.com/goldbook/Goldbook.html

- 62 -


period, an event should be defined as a single occurrence on a single day at a single location. Please provide a description of the root analysis methodology employed.

IPL Response:

The infonnation below provides details and analysis of over-pressurization manhole and vault events that have OCCUlTed from January 1,2005 through May 1,2011.

1) January 4, 2005 - 114 W. North Street. The network secondary cables short circuited and resulted in a sustained secondary network fire. There were multiple manhole covers that were dislodged in this event. It was necessary to de-energize that portion of the secondary network to contain the fire and minimize further damage to the network system. There were customer outages associated with this event until the damaged area could be isolated and the secondary network reenergized.

IPL actions: IPL was not able to detennine an exact cause for the failure of the secondary cable due to the fire damage. IPL replaced more than 2,000 feet of network primary and network secondary cables that were damaged as a result of this incident.

2) January 8, 2005 - 137 W. Market Street. The network secondary cables between two IPL manholes in the 100 block of West Market Street failed from excessive heat coming from a leak in a Citizens Thennal steam line located directly below the duct line. A secondary network fire occurred resulting in damage to primary and secondary cables in the area and an over-pressurization event occurred in the basement of the Bookland Building at 137 W. Market Street. Carbon monoxide gas from the secondary network fire entered the basement of the building through the service entrance conduits. The fire depmiment investigation did not detelmine the ignition source, but the pilot light on a gas water heater in the basement may have been the source that ignited the carbon monoxide gas.

IPL actions: A leak in a steam line directly below the IPL duct bank caused a deterioration and failure ofIPL network secondary cables at the point of the steam leak. Cm'bon monoxide gas from the faulted network secondary cables was able to enter the basement of the building at 137 Market due to the lack of duct-sealing materials in the service conduits to the building. The duct-sealing materials had been removed to facilitate a service upgrade that was in progress, and they had not yet been replaced.

- 63 -


IPL works closely with Citizens Energy when gas or steam incidents occur to coordinate service restoration. Citizens Energy made repairs to their steam line. After this event, IPL inspected the downtown manhole system with an emphasis on inspecting and repacking service ducts as needed to generally prevent water

and gases from entering customer premises through IPL structures.

3) September 8, 2005 - Talbot and Vermont Street. A network primary cable on Edison Feeder UG 431 faulted in the duct line at North and Capitol. The network

protector on Edison Feeder UG 431 at 117 E. Michigan Street failed to open on reverse power flow. The secondary network continued to supply fault current to the faulted feeder. The secondary network tie cables between the 117 E. Michigan Street Vault and the 114 E. Vermont Street vault failed from carrying

fault current for an extended period of time. The secondary cable failure damaged Edison Network Feeder UG 412, causing it to relay out of service. One manhole did have an over-pressurization event during this incident.

IPL actions: The cause of the initial primary cable fault was not determined. The cable did fail in the duct line between manholes. The subsequent events were

the result of the network protector failing to open on reverse power flow. IPL replaced the UG 431 transformer and network protector in the 117 E. Michigan Street Vault. Network protectors in the adjacent vault were all tested to confirm that they would operate correctly. All damaged primary and secondary cables were replaced.

4) Febmary 17, 2010 - 215 E. Ohio Street. A customer owned transformer at this

location suffered an internal electrical failure. The tank on the transformer mphn~ed resulting in an oil fire that was contained to the transformer vault. There

was no damage to any IPL facilities.

IPL actions: IPL did not perform a root cause analysis for this incident because it was customer owned equipment that failed and IPL' s facilities were not damaged in the event. The customer did not require IPL to install temporary facilities. They were able to repower their building from the two . remaining

transformers, and the transformer involved in this event was replaced by the customer.

5) August 18, 2010 - 535 Massachusetts Avenue. The question referred to an incident on August 17, 2010, but the actual date was August 18, 2010. A failure

occurred in the primary telmination chamber on the network transformer

- 64 -


connected to Edison Substation Feeder UG 422 in this vault. A small flash fire occurred and flames came through the vault grating.

IPL actions: The network transformer was replaced.

6) August 19, 2010 - 342 Massachusetts Avenue. At 4:18 a.m. Edison Substation Feeder UG 459 relayed out of service for a splice failure. There was no manhole over-pressurization event associated with this failure. At 8: 16 a.m. Edison

Substation Feeder UG 449 relayed out of service from a splice failure. There was a manhole over-pressurization event associated with this second splice failure with the manhole cover becoming dislodged. When Feeder UG 459 relayed out of service from the first splice failure, the load current it had been carrying was picked up on Feeder UG 449. Our investigation revealed an undersized section of underground priinary cable in the Feeder UG 449 circuit where the second splice

failure occurred. We believe that heating from the higher load current flowing through this section of cable from the first splice failure contributed to the second

splice failure.

IPL actions: An undersized section cable resulted in overheating of the cable. This heat was transferred to the splice causing it to fail. The section of cable was replaced with larger cable. IPL inspected the adjacent sections of cable to verify

that the cable data records matched the cable that was in the field. No other discrepancies were found.

7) January 30, 2011 - E. Michigan Street & New Jersey Street. At 6:36 a.m. the Indianapolis Fire Depmiment was dispatched on the report of a gas odor in the vicinity of the Athenaeum on the southeast comer of Michigan and New Jersey. At 6:38 a.m. a manhole over-pressurization event occurred in the intersection of

Michigan and New Jersey. IPL determined that a secondary cable failed in that manhole. Over the next few minutes two additional manhole over-pressurization events OCCUlTed. One OCCUlTed while a fire tmck was pm'ked over a manhole and

the second was in a manhole next to the Athenaeum. There was no damage to the fire tmck, but a window was broken at the Athenaeum. IPL believes that a combustible gas, possibly natural gas, was present in the manhole/duct line system and contributed to the two later manhole events. We do not believe that a sufficient quantity of carbon monoxide would have been generated from the original secondary cable fault to sustain the latter two manhole events.

IPL actions: IPL installed new secondary cable and splices.

- 65 -

o 'Neill Management Consulting, LLC IURC Cause 44602 - .IPL Network Investigation

8) April 27, 2011 - 20 W. Court Street. Around 6:00 a.m. a failure occurred involving customer owned cables in the customer's junction box. This failure on the customer's service cables led to a failure of IPL secondary cables in a manhole just outside the building. When the IPL secondary cable failed, a manhole over-pressurization event occurred. The IPL secondary cables failed from the high amounts of current flowing through the cable to the faulted cables in the customer's building.

IPL actions: A customer-owned secondary cable failed between the customer's Junction Box and his main switchgear. Fault current flowing to this fault lead to a failure ofIPL's network secondary cable in a manhole just outside the customer's building. IPL replaced 300 feet of secondary cable and installed cable limiters on the service cables going to 20 W. Court. IPL also required the customer to install cable limiters on his cables where they connect to the main switchgear.

In most cases, IPL performs an investigation to determine the root cauSe of an event. These investigations are conducted by either Operations personnel or Network Engineering personnel. The investigation methods include a site visit to examine conditions at the location of the event, visual inspection of the faulted equipment, and a review of system data available from the Energy Control System, including event messages, sequence of events, equipment status, recorded voltages and currents. In some cases, faulted equipment may be sent to a lab or a vendor for further analysis. When damage to the affected electrical components is too severe, or when outside factors clearly caused the event, we proceed to replacing the damaged components.

3. Please provide an analysis of the incident that occurred on May 31,2011 on the Capitol Street side of the Indiana Statehouse.

IPL Response:

On May 31, 2011 at 4:59 p.m. IPL's underground Feeder UG 641 relayed out of service. A few minutes later IPL received a report from the Indianapolis Fire Depatiment of a possible electrical fire at 46 N. Capitol on the west side of the Indiana State Capitol Building. IPL personnel were dispatched to the scene to meet the Fire Depatiment. Upon arrival at the scene, IPL personnel determined that a failure had occurred in the Primary Termination Chamber on the West Transformer in the North Bay of the 46 N. Capitol vault. The transformer in question is connected to the UG 641 circuit that relayed.

- 66-


Upon further investigation, IPL determined that the stress termination on the center phase in the termination chamber failed. The cable faulted to ground in the chamber with some collateral damage to two of the three cables. The arc from the failure likely heated the mineral oil in the chamber, breaking the oil down into various gases. The pressure in the termination chamber likely built to the point that the cover ruptured, releasing the gases and hot oil, which likely would have been ignited from the arc. There were approximately 10 gallons of mineral oil in the termination chamber. This likely resulted in a brief flash fire until the oil was quickly consumed and the fire went out. Confirmed accounts by conversation with a credible eyewitness indicated duration of the flash fire above the grating was less than 10 seconds.

The exact reason for the failure of the stress termination cannot be determined because it was completely consumed in the arcing from the fault. The likely cause for this type of failure would be some type of void in the stress relief tape. The void would have created an area of high electrical stresses which would have led to the failure. The oil level in the termination chamber or possible contamination, such as moisture in the mineral oil, would not cause this type of failure.

IPL actions: IPL will be replacing the network transformer involved in this event. We are working to schedule this change-out. IPL also plans to conduct oil quality tests and Dissolved Gas Analysis on all 315 network transformers. This testing requires that the transformer be de-energized, which requires taking the network feeders serving the transformers out of service. Ambient temperatures and loading on the network feeders must be lower than current conditions before we can schedule this work. We will commence work on this project as soon as we can reliably do so. 4. Please provide information related to the number of over-pressurization events with underground plant experienced at utilities similarly situated to IPL. The lURe is most interested in underground facilities in urban downtown areas in similarly-sized Midwestern cities over the past five years. We would like a minimum of three additional examples for comparison purposes.

IPL Response:

IPL has not identified a publicly available study of the number of "over-pressurization events" for the past five years in similarly-sized Midwestern cities.

IPL's research indicates inconsistent definitions of network events and the date ranges for such events. Because of this inconsistency, IPL recommends caution in using this information to draw definitive conclusions. In communications with IPL, industry experts and Electric Power Research Institute ("EPRI") staff echoed these concerns about

- 67 -

o 'Neill Management Consulting, LLC lURC Cause 44602 - IPL Network Investigation

consistency of the available information. Subject to these caveats, the information IPL was able to find from the District of Columbia and Massachusetts is presented below.

Summary of Other Utility Manhole Events

State Companies Date Range Manhole # of Manholes # of Vaults # yrs of Events data

DC26 Pepco Jan 2000- Dec 2000 48 nla nla 1

Jan 2001- July 2001 46 nla nla 0.58 MA27 NSTAR July 2004 - Dec 2005 44 38000 800 1.5

National Grid Aug 2004 - Dec 2005 20 20735 1675 1.4

WMECO June 1999- Dec 2005 30 3750 250 5.5

Unitil 1998- Dec 2005 0 192 30 8

5. Please provide details regarding IPL's process for qualifying customers for interconnection witlt IPL underground facilities and wltetlter any periodic inspections are performed by IPL or if any certifications are maintained on tlte condition, of tlte said interconnectionfacilities and customer equipment. What training and/or certification is 1) required of and 2) provided by IPLfor customers wlto interconnect witlt IPL via its undergroundfacilities?

IPL Response:

In order to take service from IPL, customers are required to have a qualified electrician install the service facilities and obtain the necessary permits from the City of Indianapolis. The electrician must install the customer's equipment according to the current version of the National Electric Code (NEC) and meet IPL's requirements for service as outlined in IPL' s Electric Service and Meter Manual. F or all new services or

service upgrades downtown, a member of IPL's Network Engineering group will meet with the customer's electrician to review the proposed work. The application for a permit from the City of Indianapolis triggers an inspection of the customer's electrical service

equipment by the city electrical inspector. Once the electrical inspector determines the installation is compliant with all applicable cunent codes, IPL installs the meters and makes the final connection to the IPL underground system. While the NEC is updated

26 The repOlt "The Assessment of Underground Distribution System of the Potomac Electric Power

Company Final Report", dated December 7, 2001 and referenced in May 2007 Filing No. 991-E-218 can be viewed at http://www.utilityregulation.com/contentireports/DCSWFinaIReport.pdf

27 The report "Independent Assessment of Dislodged Manhole Covers Prepared for The Commonwealth of

Massachusetts Department of Telecommunications & Energy Massachusetts Report", dated December 9, 2005 can be viewed at http://www,mass.govlEocaJdocs/dte/distribution_system_safety-'piansl12905siemensreporLpdf

- 68 -

O'Neill Management Consulting, LLC lURe Cause 44602 - IPL Network Investigation

every two years, it is not retroactive. As customers upgrade electrical systems to

accommodate changes or building additions, any new facilities must meet the current NEC requirements. Customers are not required to modify or upgrade existing facilities not involved in the upgrade. As mentioned in the response to Question 1, IPL's service requirements are posted on its website as Electric Service and Meter Requirements ("The Gold Book").

6. Is IPL aware of the extent to which other peer group electric utilities regularly inspect customer-owned equipment that connects to the utility's underground facilities?

IPL Response:

IPL is not aware of any peer electric utility that regularly inspects customer owned equipment connecting to its underground facilities.

7. Please provide, to the extent IPL is aware, details of the requirements that utilities from other states follow that address issues related to the over-pressurization of underground utility plant.

IPL Response:

Electric utilities strive to utilize best practices to provide safe and' reliable service to their customers. Utilities throughout the United States are required to abide by the following codes and agency rules:

• NESC - National Electric Safety Code o Clearances and operations

• Occupational Safety & Health Administration (OSHA) o Adequate training for electrical line workers o Confined Space entry procedures o Signing, signaling, flagging

• Environmental Protection Agency (EPA) Rules o PCB oil testing and containment o SPCC (Spill Prevention, Control, and Countermeasures)

• Local Electrical Pelmitting Practices o Connect only after inspection is complete and approved by

appropriate pelmitting authority (such as the City of Indianapolis) IPL researched regulatory reporting requirements from other states and reports the

following: - 69 -


California

Connecticut

Massachusetts

New York

CA PUC began an investigation into PG&E underground network events in late spring 2011.

In April 2007, the PUC required CL&P to continue a four-year inspection program of manholes and to replace a portion of its secondary network over a five-year period and to report project status quarterly. (Docket No. 06-10-21) In addition, the utility is required to file transmission and distribution maintenance plans on a two-year cycle. (Docket No 86-12-03) CL&P agreed to highlight network maintenance in future reports.

The Depmiment of Telecommunications and Energy ("DTE") initiated an "Independent Assessment of Dislodged Manhole Covers" in 2005 and required the four subject utilities to implement the following recommendations of the report: 1) create a technical working group - including DTE staff - to standardize definitions and event reporting; 2) complete inspections on a fiveyear cycle; 3) report qumierly findings; 4) document and log splicing activity; 5) complete failure analysis; and 6) file annual reports.

Following a 1999 power outage, the New York DPS initiated an investigation of Con Edison underground incidents (Case 99-E-0930). On March 15, 2000, DPS Staff issued a repOli that contained 44 recommendations for the company to implement. The Commission ordered Con Edison to implement all of the recommendations and required Con Edison to file quarterly status reports on its plans to implement the recommendations. Con Edison subsequently filed a Petition requesting modification to allow reporting on a semi-annual basis (Nov 1 and May 1), which the Commission approved. In the first two years following the May 2000 order, Con Edison implemented 32 of the recommendations. As of May 1, 2011, Con Edison has completed all but one of the 44 recommendations.

District of Columbia The DC PSC initiated an investigation and independent assessment following a series of manhole explosions in the late 1990s. Following independent repOlis by Stone &. Webster and Siemens, the PSC ordered PEPCO to inspect manholes on a four year cycle,

- 70 -


test new technologies such as remote monitoring and manhole Covers that limit displacement in pressurization events, complete seasonal modeling and report quarterly and annual reports to the PSC.

8. For each year since 2000, what has been IPL's total capital investment in underground facilities? Please provide information that details the amount spent on the downtown area in particular.

IPL Response:

The table below shows the capital investment made in underground facilities for the years 2000 through 2011 year to date ("YTD"). The column labeled Downtown Underground Network Area is the area that comprises the secondary network area. The column labeled Underground Non-Network area is the balance of the IPL service territory served from the general distribution system with underground facilities. These values represent capital investment in new facilities to serve new or load additions, system upgrades, and replacement of facilities.

Capital Investment

Downtown UG UG Non-Network Total UG Capital Year Network Area Area Investment

2000 $2,298,445 $18,200,740 $20,499,185

2001 $798,012 $19,525,405 $20,323,417

2002 $1,010,730 $19,943,938 $20,954,668

2003 $1,024,313 $26,023,139 $27,047,452

2004 $1,869,913 $18,213,189 $20,083,102

2005 $3,218,865 $19,221,403 $22,440,268

2006 $2,260,853 $19,988,977 $22,249,830

2007 $2,618,414 $20,302,278 $22,920,692

2008 $4,593,963 $16,751,397 $21,345,360

2009 $3,997,232 $15,428,193 $19,425,425

2010 $6,071,673 $13,584,576 $19,656,249

2011 YTD $5,224,799 $6,791,042 $12,015,841

Downtown UG Network Area - Work on the Cultural Trail began ramping up in 2007 and has continued through 2011. This project has required relocation and replacement of underground facilities including duct lines, primary cables, and secondary cables. In 2005, there were 4 major additions to the network including the Comad Hotel, the Simon Office Building, Hudson Condos, and Homewood Suites Hotel. In 2008, the increase in spending is attributable to providing service to Lucas Oil Stadium. The increase in 2010

- 71 -


and 2011 YTD is attributed to the start of relocation work for the Georgia Street upgrades led by the City of Indianapolis.

UG Non-Network Area - The large increase in spending in 2003 can be attributed to the burial of several substation exit circuits for the interstate 1-70 entrance to the new Indianapolis International Airport. In addition, 2003 saw a larger than normal expenditure for new UG commercial projects. The reduced expenditures that started in 2007 and continue to present are directly related to the slowdown in the economy. 9. For each year since 2000, what have been IPL's total maintenance expenditures for underground facilities? Please provide information that details the amount spent on the downtown area in particular.

IPL Response:

The table below shows the maintenance expenditures made in underground facilities for the years 2000 through 2011 YTD. The column labeled Downtown Underground Network Area is the area that comprises the secondary network area. The column labeled Underground Non-Network area is the balance of the IPL service territory served from the general distribution system with underground facilities. The values represent expenses for repairs of facilities, inspections, routine maintenance, and any noncapitalized work.

. Maintenance Expenditures Total UG

Downtown UG UG Non-Network Maintenance Year Network Area Area Expenditure

2000 See Note 1 See Note 1 $4,349,746

2001 See Note 1 See Note 1 $4,353,497

2002 See Note 1 See Note 1 $3,420,738

2003 See Note 1 See Note 1 $3,361,098

2004 $1,l29,017 $2,953,399 $4,082,416

2005 $1,759,071 $3,048,405 $4,807,476

2006 $1,373,129 $2,597,986 $3,971,115

2007 $1,844,163 $2,647,764 $4,491,927

2008 $1,470,764 $2,981 ,070 $4,451,834

2009 $1,529,371 $2,872,771 $4,402,l42

2010 $1,036,397 $3,243,648 $4,280,045

2011 YTD $477,334 $1,179,689 $1,657,023

Note 1: For the period 2000 through 2003 the specific maintenance costs for the Downtown Underground Network cannot be broken out from the total dollars spent on underground maintenance. The total UG maintenance expenditures are relatively

- 72 -


consistent throughout the period. We believe the split in underground expenditures

between the downtown and the other underground areas in 2000-2003 would likely be similar to that in 2004 and beyond.

10. Does IPL have a long-term maintenance plan? Has IPL changed its underground maintenance practices over the last 10 years? If yes, please explain the specific changes and the reasons for each change.

IPL Response:

Yes, IPL has a long-term maintenance plan that includes the following features:

• A five-year inspection cycle of its manholes in the downtown area. The industry standard for good utility practice is to inspect manholes on a four-to-ten-year .

cycle. This means we enter and inspect about 260 manholes each year. [~ 1 ,266

manholes in inspection cycle]

• We do a visual inspection of the structural condition of the manhole, looking for any water infiltration issues and any other conditions that

warrant further investigation.

• We inspect for damage to the fireproofmg tapes on the cables, which help

protect the cables from external heat.

• We visually inspect splices for leaking and swelling, which indicates they

need to be replaced.

• We visually inspect for the presence and condition of duct sealing

material, which prevents gases in the manhole from entering customer

buildings.

• We take current readings on all secondary cables, making note of any

cable with 0 current or greater than 200 amps of CUlTent for further

investigation. Here, we are looking for cables that may have experienced

a short circuit and are open, or cables that are approaching their maximum

capacity. Further review is performed by the crew on those circuits

indicating 0 current to determine if a cable has opened and appropriate

action is taken. If these conditions exist, Network Engineering follows up

and corrective actions are taken.

• Network protectors are on a three-year inspection cycle to make sure they

operate cOlTectly and to verify their operating settings. A general vault

inspection is also made at the same time as the network protector testing.

- 73 -


Additional aspects of the plan incorporated within the last 10 years include:

• In 2011, an infra-red (IR) camera was added as an additional preventive maintenance testing tool. The camera is used to look for potential hot spots on the network protectors, primary oil switches, and incoming primary cable feed to the transformers.

• Vault cleaning has been expanded to include an inspection of the vault structure and a visual inspection of the equipment. The structure is inspected to determine wall, roof, and drainage issues. The transformer is checked for potential leaks, rust, oil or other abnormal conditions. Ladders, vault lighting circuits, bus work, steam penetrations are also recorded conditions of the inspection. IR temperature guns (digital numeric display of temperature) were added for temperature recording in 2009.

• Beginning in 2005, manhole inspections included inspecting and repacking of service ducts as needed.

11. IPL, as part of its Summer Preparedness Presentation, listed five new initiatives it was taking in regard to the downtown network system. Please explain each of these in more detail as to cost, effectiveness, and retro-fit applicability and provide a timeline for the implementation of each.

IPL Response:

IPL has undertaken five initiatives tojmprove the Downtown Network system. These initiatives were briefly discussed at the 2011 Summer Preparedness meeting and are described in more detail below. In addition, IPL is in the process of completing a full inspection sweep of all manholes in the Downtown Secondary Network area.

• Network SCADA Projected Completion Date: 4/1/2013

The purpose of this initiative is to install a control system, known as Supervisory Control and Data Acquisition ("SCADA") that will provide two-way communication and control with all of the network protectors on the Downtown Network. This system will provide various analog values such as voltages and CUlTents at the Network Protector. The system will also provide for remote control of the Network Protector. This will allow IPL's System Operators to Open or Close a Network

- 74-


Protector remotely and eliminates the need for an IPL employee to enter the vault to perform that task.

The project involves installing an underground fiber optic communication system. The fiber optic cable will be installed utilizing existing IPL conduits. The protection relay in the network protector will be replaced with one that has two-way communication capability. The relay will be connected to the fiber optic communication system. The network SCADA system will be operated through IPL's existing Energy Control System in the Transmission Operations Control Center.

Approximately 44 percent of the required fiber optic cable has been installed, and 177 of 315 network relays have been installed for this project.

• Primary Cable fault detectors Projected Completion Date: 12/31/2012

The purpose of this initiative is to install a device on the primary network cables that detects when fault current has passed through it. This device is called a fault detector or fault indicator. Fault detectors are used in an underground system to help determine the location of the fault. IPL will be able to read the device remotely from street level without having to enter the manhole. This will result in a considerable time savings to locate a cable fault and will reduce overall feeder restoration time. This project is in the design phase. Equipment purchase is expected to commence first quarter 2012.

• Install cable limiters at select locations Study Completion Date: August 1,2012

The purpose of this initiative is to assess the feasibility of installing additional cable limiters at certain locations on the secondary network system to provide an additional level of protection. These additional cable limiters could help to isolate faulted secondary cables and reduce the risk of secondary network events. Care must be taken in the placement of these additional cable limiters in order not to increase secondary network events. The determination of the possible locations for placement of these cable limiters requires detailed models of the secondary network system, as well as performance of both power flow and fault CUlTent studies. Work has begun to develop the necessary electrical models needed for the studies to detelmine if and where additional cable limiters should be installed. A timeline for installation of additional cable limiters will be developed after the modeling and analysis IS

complete. Our plan is to complete the study and analysis phase by August 1, 2012.

• Evaluate Thermal Imaging Projected Completion Date: 08/01/2011

- 75 -


The purpose of this initiative is to evaluate the effectiveness of thermal imaging through infrared cameras on components of the Downtown Network system. IPL has been using thelmal imaging on the overhead transmission and distribution systems and on substation facilities for a number of years. These efforts have proven to be effective in finding problems before they become a failure and potential outage. Early thermal imaging cameras were very large and bulky and did not lend themselves for use in the confined spaces found on the Downtown Network system. As the thermal imaging technology has matured, the size of the equipment has decreased significantly. There are now hand-held devices about the size of a large flashlight that can be used in confined spaces. IPL has purchased two hand-held infrared cameras, which it is piloting in the full sweep of manholes and vaults. Test inspections with the cameras began on June 3, 2011. Initial results have shown that the cameras can be effective on connections and terminations.

• Combination Locking and Pressure relieving manhole covers

Project Completion Date: 912011 to take delivery of the first covers

The purpose of this initiative is to install and evaluate the effectiveness of a manhole cover that can be locked in place to prevent unauthorized access to the underground facilities, while, at the same time, allow gases in the manhole to vent in case of an over-pressurization. Until recently, no such product existed. A Michigan-based company working with EPRI has now developed a manhole cover that is locked in place through a mechanism attached to the underside of the cover. This mechanism will allow the cover to rise three to four inches and vent the pressure that can build up during an over-pressurization event. This locking manhole cover was used by Detroit Edison when Detroit hosted the Super Bowl in 2009. The security protocols for the Super Bowl require that manhole covers within the security perimeter be secured. With Indianapolis hosting the 2012 Super Bowl, IPL has been investigating the use of these covers.

IPL is working with the supplier to finalize the design of the manhole cover so that it will fit IPL's existing manhole rings. IPL expects to take delivery of the first of these covers by early September of 2011. Approximately 75 of these covers will be installed around Lucas Oil Stadium and along Georgia Street prior to the Super Bowl.

12. If IPL were to design and install a downtown network today, would the technology be similar to the existing network, or would it be different? If different, can

- 76-


any new plant added today take advantage of the new technology?

IPL Response: The basic components/theory of a network would remain the same, i. e.} vaults, manholes, network transformers, network protectors, cable, single contingency design, concrete encased duct banks, etc.

An underground network built today would also have a SCADA system. This technology facilitates remote control and monitoring of the network system. As mentioned in response to Question 11, IPL is in the process of installing such a system for its Downtown Network.

13. Please explain the extent to which IPL coordinates with or shares information with Citizens Energy in regard to underground plant and emergency response in the downtown network system area.

IPL Response:

IPL has a very open relationship with Citizens Energy and Thelmal. The utilities all respond in a timely manner to field events and emergency conditions to protect the public and maintain reliable services. Concerns we have with each other receive a timely response. When steam leaks are found that could jeopardize IPL's electric facilities, Citizens Thermal is contacted and they address the issue.

IPL has a good rapport with all utilities occupying the same area in the downtown Indianapolis.

14. Please explain how IPL coordinates with emergency responders in the event of a downtown underground explosion or fire.

IPL Response:

IPL and the Indianapolis Fire Department (IFD) have established specific protocols to respond to Downtown Network incidents. Whenever IFD makes a run on a repOli of a manhole event or other electrical issue in the downtown area, they notify the IPL Distribution Operations Control Center (IPL DOCC). This callis made via a direct phone line between the IFD Dispatch Center and the IPL DOCC. IPL immediately dispatches a truck to the repOlied location. IPL DOCC will immediately notify the IPL

- 77-


Transmission Operations Control Center (IPL TOCC) of the event. IPL TOCC will then notify the appropriate Network Operations leadership of the event so they can also

respond to the scene. Upon arrival on the scene, IFD is to secure the area and take a defensive position. They are not to enter manholes or vaults and they are not supposed to

put water o~ chemicals on any fire in a manhole or vault until IPL personnel arrive on the scene and determine the safe way for IFD to proceed. The safety of the public and the firefighters is our number one priority.

- 78 -


Appendix C - Map of CBn network incidents, 2005-2015

Note: The map does not include the Center Substation incident in January, 2012, nor the

E. 16th St. incident in July, 2012, as neither of these was actually an incident on the CBD secondary network as such, and also because including them would stretch the scale of

the map. ~

"'" o c:

'" o a: Z

WNor lhSt

'0. ~ z I

IV Wa lnu l SI

o iii ' ~

WEmmett st 1

] VlN~h SI '0. en co u z

ro E '" .D

Amel ican ~eglOn ~.II l1iley Towers Apartments' ~

Wl'la lnUI SI

c co

:0 WNorthSI '~

:2 z

NorthSt

c: o '" .. ! "

'E E Wainul St (i) co <11 > ~

~ ~ c co fi Qj a. a z North SI Z

E 7th St

E SI Clair SI

en EWalnut SI

I~ V)

r;;; -,

~ Norlh SI i

en 0: 2i

vel~S Memonal Plaza Old Nalionnl Centre ... c 0

W Michigan St Z 11

E Michigan St

Indiana I'/ar Memorial Gl ~

<11 ...J

E Michig

~ z

en >-" i:? I

EVe [ Vermonl St '

in k [Vermont SI <11 -, en W vermont St 1'1 Vermonl SI

'I '

I z West New York Street

rl

India na Stnte tit pitol Tour Office -

WOhio St

WCtSI

I r ' I , • P k Ippeea" oe S\ Umverslly. ar

<ii c: o V)

.~ a.

(i) co 'c '" > >-(I)

c C <11 a. Z

'~ ~

~ Z

II

en c o

~ Indianapolis Ci ty Mar,ket _ ... onlUlJe

, '0" ':>1' I Soldiers'nnd g, too Hil ton Gnrdeffi , "

Indianapolis Oy(tmtown Sailors Monument I

I

I I

,1

II

~ en '" E '" .D

.!;1

..:

I en ., E

(i) Conrad Indianapolis ... E Cour t St

", .D ... '«

W r¥a shington 5t E Washing ton 5 t

Hyatt nege"cy ... Indianapolis

W Ma r.yland St

Circle'Ce"ue 4\

~~~~,~?~.~s:"'!.z!. 4\

en '?, \B) , 0

(i) 0 ~i~ 'u ~ E Pearl St I c (f)

"1"" co en ~

(i) E Maryland S' Gi :2 ~ (f) co

~

- 79-

:: " Z Z

,t

E Ohio 5 1

oS' -,...'11<:'

'11~

en '0 c:

'" §! <11 U z en '0 c:

'" Qj > <11 U z

V)

Gl ~i '" Lu

ENew

en '" '" Lu

E WabaSh SI I

~

Harrison College

E Washil

~ E Pearl St


Appendix D - IPL Responses to Data Requests Cited

The following IPL responses to IURC Testimonial Staff Data Requests are cited in this

report and are therefore included with this repOli. The following table shows: • The Data Request number,

.If the response contained attachments, the name of the attachment included in Appendix D,

• On what page within this document the response was cited.

Data Request Attachment(s) Page(s) where cited

Staff DR 6-7 Confidential

6.7 Attachment 1 14

5.4 N/A 16

Staff DR 6-4 Confidential

6.4 Attachment 1 33,39

Staff DR 1.7 Confidential

1.7 Attachment 1 35

Staff DR 1.8 Confidential

1.8 Attachment 6 36

1.9 N/A 37

4.5 N/A 38, 39

1.10 N/A 39

1.11 N/A 43

St'aff DR 5.8 5.8 Attachment 1 45

- 80 -

Data Request Staff DR 6 - 07

Please provide an updated database of network transformers and network protectors with dates of installation, (or manufacture, specifying which it is), or labeled 'unknown' if not reasonably known. For clarity, Testimonial Staff understands that subsequent to the December 2011 O'Neill Report, IPL did a survey of nameplates on network protectors that greatly reduced the number of those with unknown dates of installation or manufacture.

Objection:

IPL objects to the request on the grounds and to the extent it seeks specific detailed design information that constitutes Critical Energy Infrastructure Information ("CEll"). Such CEIl is highly sensitive and the disclosure of which could endanger the public welfare and security. IPL further objects to the Request on the grounds and to the extent the request seeks information that is confidential, proprietary, competitively-sensitive and/or trade secret. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

See Staff DR 6-7 Confidential Attachment 1 for the updated network transformer and network protector information. There is a worksheet for transformers and another worksheet listing protectors. Please note that the feeder numbers have been replaced with "XXX" to protect CEIl information.

14

Response to Staff Data Request 6.7, Confidential Attachment 1 (Updated

Network Transformer and Network Protector Information)

CONFIDENTIAL - REMOVED FROM PUBLIC VERSION


In reference to IPL's response to IURC Testimonial Staff data request 1.6 confidential attachment 1, please state whether the following 14 vaults have been inspected since 2010 and if they comply with IPL vault inspection cycle of 3 years (2 years post 2011). Asset # : NV501 V51NW NVS502M33NW NV505V22NW NVS502M44NW NVS501I11NW NVS503M21NW NVS501I41NW NVS506M21NW NVS501K32NW NVSEC501 V11NW NVS501M43NW NVS501M55NW NVS50 1 V32NW NVS502K21NW NVS502K43NW

Objection:

Response:

Please see the table below. For the assets provided in this question, all 15 of the locations have been inspected since 2010 . The first two locations (NV501V51NW and NV505V22NW) have been retired and there will be no future inspections . Although records to confirm that four of the locations were inspected in 2014 could not be found, all four locations were inspected in 2015.

10

Inspected Compliant Asset # Location since 20107 with cycle? Comments Inspected

NV501V51NW Retired in Ivara 12/12/14,

429 E. Market Yes Yes Empacshows idle 1/22/13 10/1/10 and 7/9/11 NV505V22NW 24 S. Oelaware Yes Yes Retired in Ivara 12/12/14 11/5/10,7/9/11, and 4/23/12

NVS501IllNW some dates from NV5 instead 10/11/10,6/11/11,7/13/12,

12W. Market Yes Yes of NVS5 10/9/14, and 3/26/15

NVS501I41NW some dates from NV5 instead 10/18/10,7/9/11,2/6/12, and

311 W. Ohio Yes No of NVS5 3/26/2015

NVS501K32NW some dates from NV5 instead 11/2/10,6/17/11,4/9/12 and

202 W. Maryland Yes No of NVS5 4/1/2015

NVS501M43NW some dates from NV5 instead 10/11/10,7/22/11, 2/8/12,

326 E. New York Yes Yes of NVS5 6/19/14 and 3/28/15


535 Massachusetts Yes Yes of NVSs 11/11/14 and 3/26/15

NVS501V32NW some dates from NV5 instead 10/28/10,7/9/11,5/8/13,

217 E. Pearl Yes Yes of NVSs 5/15/14, and 3/28/15

NVS502K21NW some dates from NV5 Instead 10/21/10, 6/12/11,7/24/12,

120W. Washington Yes No of NVS5 and 3/26/2015

NVS502K43NW some dates from NV5 instead 11/2/10,7/9/11,6/24/13,

301 W. Maryland Yes Yes of NVS5 12/23/14, and 4/1/15


333 Massachusetts Yes Yes of NVS5 11/24/14, and 3/28/15

NVS502M44NW some dates from NV5 instead 10/29/10,7/9/11,4/20/13,

428 Massachusetts Yes Yes of NVS5 10/15/14, and 3/26/15


108 E. Market Yes No of NVS5 2/25/15 and 3/28/15


107 E. Ohio Yes Yes of NVSs 11/5/14 and 3/28/15

NVSEC501VllNW Secondary System 9 E. some dates from NV5 instead 10/1/10,7/9/11, 1/29/13, Market Yes Yes of NVS5 9/4/14 and 3/25/15

11


Please provide the most recent Documents, drafts or otherwise, related to IPL's asset management strategy of its underground networks.

Objection:

IPL objects to the Request on the grounds and to the extent the request seeks information that is confidential, proprietary, competitively-sensitive and/or trade secret. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

See Staff DR 6-4 Confidential Attachment 1 for the latest draft of the downtown network asset life cycle plan. As explained in IPL's response to Staff DR 4-5, the asset life cycle plan is in the process of being developed and implemented.

10

Response to Staff Data Request 6.4, Confidential Attachment 1 (Downtown

Underground Network Asset life Cycle Plan)

CONFIDENTIAl- REMOVED FROM PUBLIC VERSION

Data Request 44602 Staff DR 1 - 07

Please provide the 2015 - 2019 capital and expense budget by program for the transmission and distribution system, and identify those pertaining to the underground network system.

Objection:

IPL objects to the Request on the grounds and to the extent it is overly broad and unduly burdensome, particularly to the extent it seeks information for the entire transmission and distribution ("T&D") system. IPL's T&D system covers 528 square miles and includes approximately 480,000 retail customers. The vast majority of the T &D system is irrelevant to the operation and maintenance of the downtown network. IPL further objects to the Request on the grounds and to the extent the request seeks information that is confidential, proprietary, competitively-sensitive and/or trade secret. IPL further objects to the Request on the grounds and to the extent the request seeks a compilation, analysis or study that IPL has not performed and to which IPL objects to performing. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

See Staff DR 1-8 Confidential Attachment 6 for the capital budget for 2015 through 2019. The capital budget pertaining to the downtown network is shown on tab "Capital", Excel rows 62-70. The downtown underground network capital is primarily represented by two budget items: Network Capital Maintenance and New Facilities.

See Staff DR 1-7 Confidential Attachment 1 for the overall Transmission and Distribution operations and maintenance (O&M) expense budget. Typically, IPL does a detailed O&M budget for the next budget year and escalates this for future years. Any known significant items for out-years will be identified at a high level. Similar to the capital practice, the O&M budget is an active and iterative process throughout the year.

In Staff DR 1-7 Confidential Attachment 1, the tab "T &D 5 Year O&M Budget" has the five year total T &D budget. The second tab "2015 O&M Detail By Description" shows these budget items grouped by program descriptions. For example, programs such as Line Clearing are shown in Excel row 41. The third tab "CC124 Sub and Network" shows the substation and downtown network detail. Items highlighted in yellow are associated with the downtown network.

16

Response to Staff Data Request 1.7 Confidential Attachment 1, (Transmission

and Distribution operations and maintenance expense budget)

CONFIDENTIAL - REMOVED FROM PUBLIC VERSION


With reference to IPL's response to IURC Docket Entry dated March 24, 2015, question #9, discussing IPL's capital and operations and maintenance budget setting process, please provide: a. the year the screening tool was first implemented, b. a copy of the Excel screening tool, and c. Five year (2011-2015) model results with associated work activities described, model input values, prioritization ranking of that activity and ifit was funded in that particular year

Objection:

IPL objects to the Request on the grounds and to the extent the request seeks information that is confidential, proprietary, competitively-sensitive and/or trade secret. IPL further objects to the Request on the grounds and to the extent the request seeks a compilation, analysis or study that IPL has not performed and to which IPL objects to perfom1ing. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

a. IPL began using a Transmission and Distribution Excel based screening tool in the beginning of summer 2010 for the 2011 CapEx budget. The screening tool has been improved and revised numerous times since summer 2010.

b. The latest screening tool is called the Project Authorization, Scoring & Evaluation Tool (PASE) tool. This was first used in the summer of 2014 for the 2015 CapEx budget. See Staff DR 1-8 Confidential Attachment 1 for the latest version of the PASE tool.

c. See Staff DR 1-8 Confidential Attachments 2 through 6 for snapshots from the CapEx modeling for 2011-2015 and the legend table below to identify which projects passed the screening tool. Each "snapshot" represents the results of the CapEx modeling as of a specific date. Since the majority of the capital budget is driven by customer requirements, these budget items and individual projects are continually adjusted throughout the year. It is important to emphasize that this is a dynamic process so there is variance between these budget snapshots and what was actually funded. For example, a project that did not pass the screening tool as of a particular date may have subsequently been funded, either during that year or in a later year.

17

~. , . ,. .

Budget -:-.:.-.-."

... '.'. File Name ' Legend

.,.

.. . . ".--·' year ··.· .,

2011 Staff DR 1-8 Attachment 2 Items that were not funded in the snapshot are highlighted in light blue on the "Priority Summary" tab.

2012 Staff DR 1-8 Attachment 3 Items that were not funded in the snapshot are indicated with a "0" in column "C."

2013 Staff DR 1-8 Attachment 4 Items that were not funded in the snapshot are indicated with a "0" in column "c."

2014 Staff DR 1-8 Attachment 5 Items that were not funded in the snapshot are listed on the "Project_ View_Detail" tab below Excel row 130.

2015 Staff DR 1-8 Attachment 6 Items that were not funded in the snapshot are listed on the "2015-2016 Project List (2015)" tab below Excel row 32.

18

Response to Staff Data Request 1.8 Confidential Attachment 6, (2015-1019

Capital Budget)

CONFIDENTIAl- REMOVED FROM PUBLIC VERSION


Please describe the replacement or addition levels by year of any network protectors or network transformers for the past five years including the planned 2015 levels.

Objection:

Response:

The table below shows the network transformers and network protectors replaced or added to the downtown underground network system over the past five years, including 2015 YTD through March 31, 2015. The planned replacements for 2015 are also included in the results.

Network Network ProteCtors

Transformers Year . Replace New Replace .New 2011 6 0 8 0 2012 8 0 7 0 2013 8 0 7 0 2014 8 2 11 2

2015 YTD 3 0 4 0 2015 Planned 0 0 38 0

19


In reference to IPL' s response to IURC Testimonial Staff data request 1.13, please explain the "Asset Life Cycle Plan CALCP)" that is being developed, including implementation schedule by asset class.

Objection:

Response:

The Asset Life Cycle Plans are in the process of being developed and implemented. These plans take a holistic look at the asset class life cycle and are expected to address items such as:

• Description of the assets • Asset performance • Asset condition and risk (health and criticality) • Operating and maintenance strategies • Expenditures and refurbishment/replacement strategies • Information systems • Innovations and best practices

A meeting has been scheduled in June 2015 for the implementation leads to discuss the implementation schedule for the asset classes shown in the table below:

Asset Classes Implementation Status

Wood Poles Draft completed 12/412014 Relay_ System Protection Draft completed 11125/2014 Circuit Breakers Draft completed 12/15/2014 Power Transformers Draft completed 2/2712015 Downtown Network Draft completed 2/2612015 Underground Residential Cable (URD) Data Gathering and Draft Under Development - 2015 Q3 Overhead Distribution Lines Data Gathering and Draft Under Development - 2015 Q4 Transmission Structures Data Gathering and Draft Under Development - 2015 Q4 Meters TBD Substation Batteries TBD Transmission Lines TBD Substation Communications TBD Distribution Transformers TBD System Control and Data Acquisition

TBD (SCADA)

11


Please describe the replacement or additions levels by year of network primary or secondary cable and splices for the past five years including the planned 2015 levels.

Objection:

Response:

The table below shows the historical underground network cable and primary splice usage along with expected 2015 levels.

Year 2011 2012 . 2013 . ·201.4 2015 Est. Prmary Cable (feet) 24,229 20,521 24,821 27,491 20,000 Secondary Cable (feet) 20,292 4,364 3,712 4,052 10,000 Primary Splices 607 257 430 528 600

20


Please describe any changes in staffIng as a result of the Dayton Power and Light merger with AES Corp in 2011, with any affect on inspection, maintenance, and construction of distribution facilities.

Objection:

IPL objects to the Request on the grounds and to the extent it is overly broad and unduly burdensome, particularly to the extent it seeks information related to the entire distribution system. IPL's transmission and distribution ("T&D") system covers 528 square miles and includes approximately 480,000 retail customers. The vast majority of the T &D system is irrelevant to the operation and maintenance of the downtown network. IPL further objects to the request as overly broad and unduly burdensome to the extent it requests IPL to describe "any changes" in staffIng. IPL's response below is for the skilled craftsmen who perform the physical work activities of inspections, maintenance, and construction on IPL's distribution facilities. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

Downtown Network Distribution Facilities

The IPL Substation & Network Field Operations group remained unchanged as a result of the acquisition of Dayton Power & Light by the AES Corp in 2012. As previously reported in IPL' s response to Question 3a. of the IURC's 10/2/2014 Data Request, the information shows the staffing levels for the IPL Substation & Network Field Operations group (Note the table has been updated through 4115/2015):

Substation Network Group - Historical Personnel Count

Year Skilled Craft Exempt Total

2011 47 7 54

2012 47 9 56

2013 48 9 57

2014 48 9 57

2015 (as of 4115115) 47* 9 56

*Currently the Skilled Craft count is 47 but will return to 48 on Monday, April 20th due to an internal job transfer.

21

IPL utilizes contractors in downtown network for construction activities such as:

• Duct bank construction • Manhole construction • Upgraded manhole cover replacements • Large cable replacement projects (primary & fiber) • Programmatic network protector replacements

General Distribution Facilities

The IPL Distribution Field Operation group remained unchanged as a result of the acquisition of Dayton Power & Light by the AES Corp in 2012.

• No changes were made to IPL staffing. • IPL continues to manage both internal employees and contractor distribution

resources on an ongoing basis determined by: work load, retirements, internal job transfers, and terminations.

• IPL has added 32 apprentices in its 4 year distribution apprentice program. Class #1 - 12 apprentices planned graduation date in 2016. Class #2 - 20 apprentices planned graduation date 2019. Based on normal workforce planning.

22


In reference to your IPL ' s response to IURC Testimonial Staff data request 3.3 discussing the rate of telemetry errors on the network SCADA, please provide the error rate by month stmting with January 2012 segmented by root cause (e.g. out of service due to maintenance, etc.)

Objection:

IPL objects to the Request on the grounds and to the extent the request seeks a compilation, analysis or study that IPL has not performed and to which IPL objects to perfOlming. Subject to and without waiver of the foregoing objections, IPL provides the following response.

Response:

Staff DR 5-8 Attachment 1 is a graph showing the rate of telemetry errors on the network SCADA by month since January 2012. IPL does not have the telemetry data broken down by root cause but estimates that maintenance contributes approximately 2-3% to the overall telemetry error rate. During the first two and a half years, IPL was installing additional relays and troubleshooting communication equipment and the error rate was fluctuating significantly from month to month.

15

Staff DR 5-8 Attachment 1 IPL - Cause Nos. 44576/44602

Page 1 of 1

r· ... ·-··--·--... ···-·-·-··-... ----... ·-·· ...... -·· ... ·-···-·-· ... - ... --.. -- --...... - ...... ---... ---.--...... ----.-.---...... ---.-.--... -1

VI -c c a

70%

~ 60% '" ..... c a a.

Cumulative Telemetry Error Rate By Month

~ 50% ,. I , -c ro E L. a

...::.. 40% '" -c c a u QI

~ 30% +-- 1 -1--\ ';; I '\ i \ a a. ro ..... ro

~ 20% -a L. L.

~ QI ..... ~ 10% L. a .... L.

W

~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ 0 ~ ~ ~ ~ ~ 0 ~ ~ ~ ~ ~ 0 ~ ~ ~ ~ ~

\7> ~7> ~7> 'IV ,?e~ ~o \7> ~7> ~7> 'IV ,?e~ ~o \7> ~7> ~7> 'Iv. ,?e~ ~o \7> ~7> ~7>

- error rate