Prepared by Opinion Dynamics Corporation May 2004

Prepared by Opinion Dynamics Corporation May 2004

Opinion Dynamics was contracted to conduct a primary research effort to measure Market

Participant perceptions of ERCOT’s performance with respect to meeting its responsibilities.

Results allow for comparisons between market perceptions and operational realities.

Introduction

Three Phases:

Methodology

Phase 1: 9 in-depth interviews and 2 focus groups with ERCOT staff

Phase 2: In-depth interviews with 17 Market Participants

Phase 3: Survey of ERCOT’s Market Participants

ODC developed a sample of 1,157 unique Market Participants using the following lists provided by ERCOT staff:

Appropriate points of contact at market participant firms provided by MP’s via CSR’s

ERCOT Board members from 2003 and 2004

Current Committee members from 2003-04

Attendees of the 2003 IT forum

Survey Sample

Response Rate429 completed surveys from a sample of 1,157 Market Participants (37%)

ERCOT Board Members n=16

Committee Members n=112

Market Participant Staff n=301

Classification of respondent based on self-selected descriptions – QA1

By Market Participant Firm Type

Response Rate

Consumer/Other14%

Generator/PP16%

Comp. Retailer/Rep.24%

IOU13% Co-op

11%

Power Marketer9%

Muni13%

Survey Approach: 10 point scale

Many questions based on a 10 point scale: 1-3 = negative response, 8-10 = positive response.

Mean responses will trend toward the middle of a 10 point scale – only those with passionate opinion are likely to provide a rating in top or bottom 3.

In general, mean responses of 6.6 or above are favorable ratings, 7.5 and above are extremely positive responses.

Background and ContextMarket Participant Opinions Regarding

ERCOT Staff’s Role In Developing Market Rules

Findings

ERCOT staff should participate

in market rules development

ERCOT staff should only

administer the market rules Undecided

Board Member (n=16) 38% 50% 13%

Committee Member (n=112)

67% 28% 5%

Market Participant Staff (n=301)

53% 23% 24%

Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure

Findings

100%

0%

94%

6%

78%

22%

0%

20%

40%

60%

80%

100%

Board Member (n=16) Committee Member(n=112)

Market Participant Staff(n=301)

Group Respondent Feels is Most Responsible for Introducing Market Changes

Market Participant Committees ERCOT Staff

Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure (cont.)

Findings

81%

19%

0%

43%

55%

2%

37%

52%

11%

0%

20%

40%

60%

80%

100%

Board Member (n=16) Committee Member(n=112)

Market Participant Staff(n=301)

Group Respondent Feels is Most Responsible for Approving Market Changes

ERCOT Board Market Participant Committees ERCOT Staff

Background and ContextMarket Participant Interactions with ERCOT Staff

Findings

CSR’s Subject Matter Expert

ExecutiveMgt.

Board Member (n=15) 27% 20% 47%

Committee Member (n=112)

38% 48% 4%

Market Participant Staff (n=299)

66% 16% 1%

Background and ContextInterest in Future Training

Response Rate

No/Not sure33%

Yes67%

Overview of Perceived Strengths

Performance of ERCOT staff, officers and directors

Personalized contact with Market Participants-- particularly CSR contact

Timeliness and accuracy of data provided

Providing effective training

Findings

Color Key

6.5

7.4

10

0 2 4 6 8 10

Very Good

Good

NeedsAttention

YellowBlueGreen

ERCOT Staff Performance: Corporate Objectives(10 point scale, means shown)

Findings

Ensuring Reliability/Adequacy

of Grid

6.8

6.9

7.9

8.0

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Nondiscriminatory access to transmission/

distribution

Accurate accounting of electric production &

delivery

Timely information about customer’s

choice of REP

Corporate objectives: Non-discriminatory access/Market Participant Registration

(10 point scale, means shown)

Findings

Level of knowledge/ expertise displayed by the client service rep

Communication and distribution of necessary information and forms

Following procedures/ protocols for market

participant registration

Board Member (n=16)

6.9 7.1 7.5

Committee Member (n=112)

7.4 7.8 8.0

Market Participant Staff (n=301)

7.9 7.9 7.9

ERCOT Staff Performance(10 point scale, means shown)

Findings

7.16.9

7.0

7.17.0

7.27.2

7.47.3

7.37.6

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

ERCOT Officers & Directors ERCOT Staff

Consistency

Attitude

Industry expertise

Responsiveness to Market Participants

Overall Performance

Management of ERCOT organization

(Officers & Directors only)

ERCOT Staff Performance: CSRs(10 point scale, means shown)

Findings

Timely response

Knowledge/ Industry Expertise

Response accuracy

Direction of inquiries

Accessibility

Attitude/Willingness to resolve problem

Overall expectations

7.4

7.6

7.6

7.7

7.8

7.8

8.2

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

ERCOT Staff Performance: Functional Areas(10 point scale, means shown)

Findings

6.3

6.6

7.2

7.4

7.5

7.7

7.9

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Market Participant Registration

Systems Testing

Retail Transaction Processing

Scheduling

Grid Operations

Settlements and Billing

Settlements Dispute

Resolution

Communications(10 point scale, means shown)

Findings

Amount of information

Clarity of ERCOT Staff’s Messages

Written Communication

Verbal Communication

6.6

6.7

6.7

6.8

6.9

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Timeliness

6.6

6.9

7.2

7.3

7.5

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Communications: Functional Areas(10 point scale, means shown)

Findings

Asset Registration

Systems Changes

Progress on Market Projects

Bidding

Systems Planning

5.9

6.2

6.3

6.5

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Communications: Room for Improvement(10 point scale, means shown)

Findings

Impacts of PRRs

Grid ops decisions

When behind schedule for

systems changes

When systems are down

Communicating…

Timeliness & Accuracy of Data(10 point scale, means shown)

Findings

6.7

6.8

7.3

7.2

7.57.3

6.97.5

6.8

7.6

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Accuracy Timeliness

Settlements Bill

Transmission Congestion Rights

Renewable Energy Credits

Data Extracts

Metered Data

Effectiveness of Training (10 point scale, means shown)

7.2

7.3

7.3

0 1 2 3 4 5 6 7 8 9 10

SystemOperationsSeminars

Settlement andDispute Seminars

Retail TrainingSeminars

Overview of Areas for Improvement Portal reliability

Spending priorities

Systems and tools for communicating with the market

• Website navigation

• EMMS

• IT Technical Helpdesk

Functional Performance

• congestion management

• data extracts

• settlement dispute resolution Findings

6.0 5.6

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

10.0

(10 point scale, means shown)

ERCOT PortalLevel of Agreement with Statements About ERCOT IT Systems/Staff

Findings

The Portal is effective The Portal is reliable

5.7

5.8

6.2

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

ERCOT Spending Practices(10 point scale, means shown)

Findings

Spending Funds Equitably

Spending Funds on Things that are

Important to Your Company

Spending Funds Cost Effectively

4.2

5.5

5.5

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

Communications with the Market(10 point scale, means shown)

Findings

Navigation of the Web Site

Usefulness of IT Help Desk

Understanding EMMS

5.7

6.0

6.0

6.1

1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0

ERCOT Functional Performance(10 point scale, means shown)

Findings

Timely and Effective Implementation of Systems Changes

Timely Resolution of Settlements Disputes

Providing Data Extracts with Necessary

Content

Addressing Congestion Management issues

Market perceived areas of strength:

Grid reliability

Systems are providing timely and accurate data

ERCOT staff performance including personal interactions

Conclusions

Market perceived need for system improvements with:

Web site navigation

Data extracts (content)

Web portal

EMMS

Conclusions

Market perceived communication gaps in:

Impacts of PRRs

Systems changes

Spending priorities

Grid operations decisions

Conclusions (cont.)

Specific areas for strategic consideration by ERCOT Board:

Role of committees in setting spending priorities and introducing market changes

Defining and communicating ERCOT staff’s market function

Market is interested in more training--More research on potential types of training seminars is necessary

Conclusions (cont.)