CIP Spot Check Process

CIP Spot Check Process

Gary Campbell

Manager of Compliance Audits ReliabilityFirst Corporation

August, 2009

04/21/23 2

Presentation Goals

The audience should be :

Aware of the ReliabilityFirst CIP Spot Check Process to be used for review of the thirteen requirements for Table 1 entities or CIP Spot Checks in general

Cognizant of differences between an audit and spot check processes

Have an understanding of the auditors perspective in performance of the audits/spot check

04/21/23 3

Compliance Audits

ReliabilityFirst performs compliance audits:

Once every three years for BA, TOP, RC, TO/LCC Once every six years on all other functional designations starting

from 2008 Proper notice as per standard or CMEP Unscheduled as required to monitor compliance Can be on-site or off-site CIP standards audit intervals have not been determined at this time

At this time , assume a three /six year interval for applicable functions

Public and Non-Public Reports sent to NERC, Registered Entities, FERC and maintained on file at ReliabilityFirst

04/21/23 4

Spot Checks

RFC performs spot checks

Proper notice as per standard or CMEPPerformed as discussed in CMEPCan be triggered by an event, concern, trend, NERC or

FERC request, etc.Verify/confirm self certification, self reporting, data submittalsAny functional designations or registered entities can be

subject to spot check

Report maintained on file at ReliabilityFirstRegistered Entity receives copyNERC does not receive a copy, at this time

04/21/23 5

ReliabilityFirst Audit & Spot Check Goals

To be Performed: To the highest standard

Government auditing standards. CMEP, NERC RoP

Professionally Consistently

Auditor tools – QRSAWs, Surveys, RFI’s Regional agreed upon practices

Credibly With reasonable assurance, sufficient and appropriate

evidence to substantiate the findings

04/21/23 6

Audit Team Member Goals

The audit team will strive to be:

Consistent and fair Cooperative Professional Substantiate their findings

Providing credibility for their findings Findings which can withstand scrutiny of review Develop a complete record of its findings

• Documentation

• Notes

04/21/23 7

The Audited Entity

The audited entity should present Just the Facts by providing the evidence through documentation to meet the requirements of a standard as :

A complete record and understanding demonstrating compliance to a standard

Evidence that is valid

Evidence that can be substantiated?

And evidence which can withstand the scrutiny of the auditor and the public

04/21/23 8

Compliance Advice

The ReliabilityFirst staff and audit teams can not :

Tell an entity how to be compliant

Specify which practice, process to implement

Provide assurance of being compliant outside of the audit process

The staff or audit team can: Listen and provide guidance Direct registered regional entities to seek the assistance of a

consultant if the staff cannot direct the person to available documentation addressing the question

04/21/23 9

Confidentiality Agreements

Audit Team members are: Bound by their Code of Conducts or applicable Confidentiality

Agreements provided to the Audited Entity

NERC staff falls under the statement of NERC's obligation on the ROP (Section 1500) and code of conduct

FERC is bound by its agreements

Regional staff fall under their Code of Conduct and confidentiality statement per our delegation agreement

Contractors and industry volunteers will sign regional confidentiality agreements

Regional staff shall not sign an entity specific confidentiality agreement

04/21/23 10

Team Member Review of Information

The team will:

Have a conference call with the entity 85 days before the spot check review

Clear up an items of concern or understanding in the process

Have a team meeting to discuss the audit teams review of submitted information approximately 2 weeks before the review date

Request additional information for clarification or understanding Discuss preliminary requirement findings This effort allow auditors to focus on those areas of importance, lacking

information or understanding at the review.

04/21/23 11

CIP Spot Check Scope

The current CIP Spot Check Scope: For Table 1 entities - 13 requirements identified for review by

NERC for the period xxxxxxxxxxxxxxxxxxxxxxxxx

After July 1, 2010 – Table 1 and 2 entities – 41 requirements Not yet determined to be a spot check/audit

04/21/23 12

CIPS Compliance Review Team

Consist of:

Usually at least 3 – 4 members with experience with CIPS, IT and Operations

Lead (RFC Compliance Staff)

NERC observer or participant (@ NERC’s discretion)

FERC participant (@ FERC’s discretion)

04/21/23 13

Audit Team Members Roles

Team Members: Utilize technical experience

Exercise professional judgment

Gather data and information

Perform Interviews

Determine validity of the evidence

Substantiate the evidence

04/21/23 14

Objection to a Team Member

A Registered Entity can object to an team member

On the grounds of conflict of interest, or the existence of other circumstances that could interfere with the teams impartial performance of their duties

Objection must be in writing to the Compliance Enforcement Authority no later than 15 days prior to the start of the audit or spot check

ReliabilityFirst will make the final determination if the member can participate in the audit or spot check

NERC and FERC staff can not be limited in their participation on an audit or spot check

04/21/23 15

The Spot CheckProcess

The Spot Check Process consists of

Initial Notification and Request for information Conference Call with entitiy Spot Check Team Review of Information Spot Check Review on site Preparation of Spot Check Assessment and Report Distribution of Sport Check Report

T

04/21/23 16

Initial Notification

Initial Notifications will be:

For the 13 requirements, will be sent at least 90 days before the scheduled the scheduled review date of a spot check or audit. CMEP requirement is 20 days for a Spot Check and 60 days for an audit.

Contains Notification Letter

• Request for information• Background info on the process• Audit Preparation Guidelines• Audit Team Bios, Confidentiality, and COIs

An agendaSpot Check WorksheetQuestionnaires/Reliability Standard Audit WorksheetsPre-Audit Questionnaires

04/21/23 17

Audit Agenda

ReliabilityFirst will provide an agenda which:

Covers the expected days to complete the audit

Provide Audit sub-teams if appropriate

Schedule for standards to be audited and time allotted for presentations

Interview and group meeting schedules

Spot Check Worksheet

The worksheet will: Provide listing of all standards to be

addressed in the spot checkFor your use to track progress on standards

04/21/23 18

04/21/23 19

Questionnaires/Reliability Standard Auditor (QRSAWs)

QRSAWs:

Must be completed and returned 30 days before your audit your scheduled review date

Provides guidelines concerning the requirements

Does not add additional requirements

Posted on NERC Website

Could be used by internal compliance programs

Pre-Audit Questionnaires

The Pre-Audit Questionnaires request:Entity ProfileLogistical Information Request

• Hotel, airport, and travel information

Security Considerations• Identification Requirements• Restrictions• Escorts

04/21/23 20

04/21/23 21

The On-site Review

and

Post Monitoring Reporting

04/21/23 22

Typical Audit

The audit consists of:

Opening Briefing Review of requirements with SMEs and entity

personnel Any site visits as necessary Exit Briefing

The CIP Spot Check will consist of the same basic steps

04/21/23 23

Opening Briefing

Opening Briefing with management and participants of the review process: For audits and spot checks combined the 693 and CIPs topics will be

discussed together

Allows audit team to:

State Objective and Scope Explain process of the audit Discuss Confidentiality and COI Set the tone for the audit Provide the roles of the audit team and audited entity Opportunity to seek clarification on issues from RSAWs and any other preliminary

information submitted.

Allows registered entity to:

Provide overview of the their system and operations To provide logistic and security information Seek clarifications on scope of the audit

04/21/23 24

The Review

The Compliance Review of evidence to the requirements is completed: According to the Agenda With entity personnel as they designate

SME, PCC, other personnel With an opportunity for the team to additional

information, clarification and obtain an understanding of the entities evidence and approach

Should lead to a team finding on compliance

04/21/23 25

Exit Breifing

Exit Breifing with management and all participants of the audit to: Will perform with similar organization of the opening

briefing Provide the preliminary findings

Review the scope of the auditProvide the findings and the team’s basis for the findingsDiscuss Confidentiality

Discuss the report process and timeline Request completion of feedback forms

Reports

CIP Spot Checks will Have an assessment and report created ( Audits do not have a

documented assessment) Assessment is the compilation of information contained in the

completed QRSAWs, not sent to the entity. Spot Check Reports are a condensed version of the audit report

containing:• Executive Summary• Scope• Requirement Findings

Draft report will sent to the entity for comments Final Spot Check Reports will be sent to the entity and kept on file at

ReliabilityFirst. • Will not be sent to NERC at this time

04/21/23 26

The Audit Team Lead develops a

draft report

The Audit Team Lead receives

comments from the Audit team

Audit Team provides

comments

The Audit Team Lead transmits the report for audit team

review

20 Business days

The Audit Team conducts an exit briefing

with the Registered Entity with preliminary

findings

Audit Team Lead sends the draft

report to the Audit Team for their

review and comments

The Audit Team Lead sends the

draft report to the Registered Entity for their review and comments

Audit Team Lead revises the draft

compliance report

The draft report is edited upon receipt of

Registered Entity

comments

Audit Team Lead

revises the report upon receipt of

Audit Team’s

comments

Final report sent to RFC VP and

Director of Compliance, Registered

Entity, NERC & FERC as

applicable

Audit/Spot Check Report Timeline

20 business days

10 business days

5 business days

5 business days

Registered Entity reviews and

provide comments

Revision of the draft report

Audit Team provides

comments

5 business days

Audit Team Lead

completes final

compliance report

5 business days

04/21/23 28

Questions ?

Gary Campbell

ReliabilityFirst Corporation

Senior Consultant – Compliance