Kinetix Quality Services Internal Audit Report Purpose

Kinetix Quality Services Internal Audit Report

KQS-2020-07-FDM

Page 1 of 3

I. Purpose

The purpose of the audit was to perform the 2020 Annual Internal Audit of:

Kinetix Quality Services, LLC (KQS)

318 S. Main Street

Albion, NY 14411

II. Scope

The scope of the audit included verifying the KQS Nuclear Quality Assurance Program and its

implementation for adequacy and effectiveness and for compliance with the requirements of

10CFR50 Appendix B, 10CFR Part 21, and NQA-1:2008. The audit was conducted using the

Nuclear Procurement Issues Committee NUPIC Audit Checklist, Rev. 21. The following

portions of the checklist were found to be applicable.

Audit Section Section Description

1 Contract Review

9 Document Control

10 Organization/Program

11 Nonconforming Items/Part 21

12 Internal Audit

13 Corrective Actions

14 Training/Certification

15 Field Services

16 Records

III. Method

The audit was conducted remotely due to COVID-19 travel restrictions by reviewing records of

program activities and interviewing personnel responsible for program development and

implementation. Emails, photographs and web teleconferencing were used, as applicable, to

conduct the audit. A demonstration of the Service Order entry system was done via web

conference.

IV. Personnel Contacted

Name Title/Function Pre-Audit

Meeting

During

Audit

Post-Audit

Meeting

Chris Kulwicki Owner/President X X X

V. Audit Team

Frank D. McGee, Lead Auditor

Mark A. Mohar, P.E., Auditor/Lead Auditor in Training

VI. Pre-Audit and Post Audit Meetings

Pre-Audit Meeting:

A Pre-Audit meeting was held the morning of November 9, 2020. Introductions were made, and

the audit scope, schedule and logistics were discussed. Following the meeting, the investigative

portion of the audit was conducted until the audit exit meeting.

User

Typewritten text

QA RECORDS REVIEW


KQS-2020-07-FDM

Page 2 of 3

Post Audit Meeting:

The Post-Audit meeting was held the afternoon of November 16, 2020. The audit

recommendation was discussed to ensure a common understanding.

VII. Audit Summary

The audit resulted in one (1) recommendation which is not a finding but an opportunity for

improvement.

Below are highlights of the audit results from each of the checklist areas reviewed:

1. Contract Review

Recommendation #1: KQS-204, Rev. 1 Section 3.8 and subparts neither specify that

examinations are required, nor do they say are optional per course. Course #03 Bettis

APL NQA-1 Overview did not have an examination to measure learning objectives.

Revise this section to clarify certain courses do not require an examination and therefore

the QA record is not applicable.

9. Document Control/Adequacy

No Findings or Comments were initiated as a result of reviewing this NUPIC Checklist

Section.

10. Organization/Program


Section.

11. Nonconforming Items/Part 21


Section.

A comment in this section from the 2019 Internal Audit was adequately resolved.

12. Internal Audit


Section.

13. Corrective Action


Section.

14. Training/Certification


Section.

A finding in this section from the 2019 Internal Audit was adequately resolved.


KQS-2020-07-FDM

Page 3 of 3

15. Field Services


Section.

16. Records


Section.

A comment in this section from the 2019 Internal Audit was adequately resolved.

VIII. Follow-up, Closeout, and Effectiveness of Previous Corrections

The 2019 Internal Audit had one finding which was addressed using the corrective action

procedure in CAR #19-01. The CAR was closed out in a timely manner and has been adequately

resolved through the corrective action process. The corrective action process is evaluated to be

adequate.

IX. Adequacy and Effectiveness of Quality Assurance Program

Based on the audit results and discussion during the Post-Audit meeting, the KQS Nuclear

Quality Assurance Program is found to be adequate and effective. The audit is considered closed.

END

AUDIT REPORT KQS-2020-07-FDM

Prepared by: ___________________________________________ Date:_______________________

Frank D. McGee, Lead Auditor

X. Attachments

Audit Plan

Completed Audit Checklist

11/18/2020

NUCLEAR PROCUREMENT ISSUES CORPORATION AUDIT CHECKLIST

Supplier Name: Kinetix Quality Services, LLC (KQS) Document # 7

Audit ID #: KQS-2020-07-FDM Rev. 21

Page: 1 of 23

SUMMARY SHEET

SUPPLIER INFORMATION

SUPPLIER NAME: Kinetix Quality Services, LLC (KQS) NUPIC SUPPLIER #: N/A

PRIMARY ADDRESS: 318 S. Main Street

CITY: Albion STATE: NY ZIP CODE: 14411

ADDITIONAL LOCATIONS: N/A

PRIMARY TELEPHONE #: (585) 283-4062 COMPANY WEBSITE: www.kinetixquality.com

PRODUCT/SERVICE: Training, Qualification, and Supply of Nuclear Lead Auditor Personnel to Clients, and Supply of Training Services

ASME CODE STAMP, AUTHORIZATIONS, AND EXPIRATION DATES: N/A

QUALITY PROGRAM COMMITMENTS

10 CFR 50 Appendix B ANSI/ASME NQA-1 (Yr.): 2008 ASME Section XI IEEE (specify):

10 CFR Part 21 ANSI N45.2 ASME NCA 3800 SNT-TC-1A

10 CFR (other) ANSI (other) ASME NCA 4000 CP-189

Other

SUPPLIER CONTACTS

SENIOR COMPANY OFFICER: Christopher J. Kulwicki TITLE: President PHONE: (585) 283-4062

SENIOR QA OFFICER: Kyle G. Robinson TITLE: QA Manager PHONE: (585) 414-8471

EMAIL: [email protected]

AUDIT INFORMATION

MEMBER AUDIT ID #: KQS-2020-07-FDM NUPIC AUDIT ID #: N/A AUDIT DATES: November 9-16, 2020

AUDIT TEAM INFORMATION

AUDIT TEAM: NAME/NUPIC MEMBER

DESIGNATOR: PHONE: EMAIL:

CHECKLIST

SECTIONS

AUDITED:

TEAM LEADER Frank D. McGee 803-873-1943 [email protected] 14

TEAM MEMBER Mark A. Mohar 612-791-1927 [email protected] 1, 9, 10, 11, 12, 13,

14, 15, 16

TECHNICAL SPECIALIST N/A

(Specify Discipline)

AUDIT TEAM LEADER DATE

NUPIC REPRESENTATIVE DATE

11/18/2020

http://www.kinetixquality.com/

mailto:[email protected]




Page: 2 of 23

SUMMARY SHEET

SUPPLIER QUALITY MANUAL(s): Kinetix Quality Services Nuclear Quality

Assurance Manual REVISION: 9 DATE: 1/1/2020

ADD. SUPPLIER QUALITY MANUAL(s): N/A REVISION: N/A DATE: N/A

Audit Section

Section Description Previous Audit Finding(s)/

Deficiency(s) Section Status

Comments/Findings/Deficiencies

1 Contract Review S Recommendation #1

2 Design N/A

Kinetix Quality Services provides no

design services and performs no

design work.

3 Commercial Grade Dedication N/A

Kinetix Quality Services does not

procure any commercial grade

material requiring dedication and

provides no commercial grade

dedication services.

4 Software Quality Assurance N/A Kinetix Quality Services does not use

NQA Software for quality work.

5 Procurement N/A Kinetix Quality Services procures no

safety related services or items.

6 Fabrication/Assembly Activities, Material

Control and Handling, Storage and Shipping N/A

Kinetix Quality Services performs no

fabrication/assembly or associated

activities.

7 Special Processes N/A


special processes nor the associated

activities.

8 Tests, Inspections, and Calibration N/A


tests or inspections and requires no

calibrated equipment.

9 Document Control/Adequacy S

10 Organization/Program S

11 Nonconforming Items/Part 21 S

12 Internal Audits S

13 Corrective Action S

14 Training/Certification S

15 Field Services S

16 Records S

SECTION STATUS KEY

S – SATISFACTORY F / D – FINDING / DEFICIENCY N/A – NOT APPLICABLE (document basis)

NOTES: An audit section status identified as “F” indicates that one or more attributes in this checklist section were found to have inadequate documented or implemented controls and resulted in the section being unsatisfactory.




Page: 3 of 23

SUMMARY SHEET

An audit section status as “D” indicates that one or more attributes in this checklist section were found to have inadequate documented or implemented controls but did not result in the section being unsatisfactory based on the supplier adequately identifying these conditions within their corrective action program.




Page: 4 of 23

SECTION 1 – CONTRACT REVIEW

METHOD OF VERIFICATION

1.1 Within the assessment/summary section of each checklist question, record the procedures/instructions/drawings including the revision/date used to verify implementation.

1.2 Verify that measures are established and implemented for the translation of customer purchase order/contract technical and quality requirements into the supplier’s control documents.

References:

Appendix B/ANSI N45.2 - (3/4)

ASME Section III

NQA-1 Requirement 3

RESULTS: SAT FINDING(s)/DEFICIENCY(s) N/A Describe basis in box below.

FINDINGS/DEFICIENCIES:

(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to provide the

measures for translation of purchase order/contract technical and quality requirements into supplier’s control documents:

KQS NQA Manual Rev. 9, Section 5.0, Instructions, Procedures, and Drawings

KQS-701, Rev. 1, 12/1/2017 Order Entry and Contract Review

b. Describe implementation of the supplier’s measures (who, what, how) for translating customer purchase

order/contract requirements, including:

1. Correct translation of technical and quality requirements into the control documents

2. Documentation, customer notification, and customer approval of any exceptions to the purchase order/contract technical and quality requirements, including design changes.

KQS-701, Rev. 1 provides instructions for the review of customer contracts and/or purchase orders. The Owner/President is responsible for ensuring the bids/quotes are processed; the PO/Contract technical and quality requirements are translated into work documents; and documenting exceptions to customer purchase order requirements.

The Owner/President is authorized to respond to RFQs. Bids are processed by the Owner/President by reading through the original documents, entering the customer into the customer database, review for completeness and clarity, contacting customer for more information or clarifications, and contacting the customer for modification of problematic requirements, if any.

An observation of the customer service order entry was performed via videoconference on 11/12/2020. Six Service Work orders were reviewed as part of the audit, five of the Service Work Orders were closed at the time of review.

Service orders are kept in a database. The database contains links to customer purchase orders, relevant correspondence between the Owner/President and customer, Service Orders, technical and quality requirements and other relevant information to the order.

Each Service Order is assigned a unique number starting with the year and sequential number based on the date received. The Customer Contact, KQS Quote number, Customer Purchase Order number, and PO date are logged. A description of service is included which translates the customer requirements into a work order. Service and administrative details are listed based on items found in the customer RFQ or purchase order. The database also contains the date of work issue and the date completed.

There were a few cases where there were changes requested to the customer contract requirements. The original customer PO and revised PO were kept in the database record. Email correspondence between Kinetix and the customer were also retained in the database. In each case, the customer returned a revised PO with the changes indicated in the document. The Owner/President signed the document approving the changes when they were acceptable.




Page: 5 of 23

Several of the documents reviewed were being stored as records. The records were reviewed by the Owner/President and contained a digital signature indicating review. The reviews were timely based on the work completed date.

c. Are procedural controls adequate and procedure revision current?

YES or NO (describe in Findings/Deficiencies current section above)

d. Are procedural controls adequately implemented?

YES or NO (describe in Findings/Deficiencies current section above) or N/A


1.3 Verify that measures are established and implemented for control of items returned from the customer for repair/rework.

Checklist Interface:

Provide any related supplier nonconformance information to the audit team member evaluating checklist Section 11.

References:


ASME Section III

NQA-1 Requirement 15



(current)

N/A Kinetix Quality Service does not supply items and therefore performs no repair or rework activities.


(previous)


measures for control of items returned from the customer for repair/rework:

b. Describe implementation of the supplier’s measures (who, what, how) for return of items to be

repaired/reworked:






1.4 Verify that measures are established and implemented to ensure that final record packages, including Certificates of Compliance/Conformance, demonstrate that purchase order/contract technical and quality requirements were satisfied.

References:

Appendix B/ANSI N45.2 - (6/7) (17/18)

ASME Section III





Page: 6 of 23



(current)

Recommendation #1


(previous)

None

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to provide measures

to ensure that final record packages demonstrate that purchase order/contract technical and quality requirements were satisfied:

KQS NQA Manual Rev. 9, Section 5.0, Instructions, Procedures, and Drawings

KQS-701, Rev. 1, 12/1/2017 Order Entry and Contract Review

KQS-204, Rev. 1, 1/1/2020 Customer Training Classes

b. Describe implementation of the supplier’s measures (who, what, how) to ensure that final record

packages demonstrate that purchase order/contract technical and quality requirements were satisfied:

KQS-701, Rev. 1 Section 3.7.1 states that the Owner/President shall verify that the service being furnished complies with the procurement requirements. At the completion of the work, the Service Order is signed by the Owner/President constituting the evidence and record of his verification.

Six Service Orders were reviewed as part of this audit, five were closed and one is in progress. All six Service Orders were signed as Issued for Work. The five completed Service Orders were signed for Work Completed.

Customer orders where a PO is not issued does not receive a Service Order for the work.

Purchase Orders for two Service Orders were revised by the customer based on review by the Owner/President. In each case, the customer was contacted and modifications to the requirements were proposed. The changes were approved by the Owner/President in the method prescribed. Original and amended documents are included in the Service Order record.

KQS provides Lead Auditor Training services for the purpose of training potential NQA-1 Lead Auditors. The training is usually procured through the company website and purchase via credit card. Training is required by the requirements of ANSI N45.2.23 and NQA-1, Requirement 2 Section 303.

• The training includes knowledge and understanding of nuclear related codes, standards, regulations,

and regulatory guides; general structure of QA programs; auditing techniques, and planning for audits.

• KQS-204 Section 3.8.4 states that an examination score of 80% is required for a passing score.

• Training Records include Attendance Sheets, Course Objectives, and Completed Examination Cover

Sheets. Information regarding specific test questions and answer sheets are retained, however not as

a QA record for integrity of the examination. A review of the examinations is performed by the

Owner/President and the Office Manager.

• Class Records were reviewed for the classes conducted since last audit:

o Class #58 Leonardo DRS December 2019 Lead Auditor Training

o Class #03 Bettis APL NQA-1 Overview

Recommendation #01: KQS-204, Rev. 1 Section 3.8 and subparts neither specify that examinations are required, nor do they say are optional per course. Course #03 Bettis APL NQA-1 Overview did not have an examination to measure learning objectives. Revise this section to clarify certain courses do not require an examination and therefore the QA record is not applicable.








Page: 7 of 23

SECTION 1 – CONTRACT REVIEW (FIGURE 1)

CUSTOMER P.O. CONTRACT NUMBER

and DATE

ITEM/SERVICE DESCRIPTION and

PART NUMBER (as applicable)

SUPPLIER CONTROL DOCUMENTS (work orders, travelers, drawings, etc.)

and CORRECT TRANSLATION TO SUPPLIER

CONTROL DOCUMENTS (Yes/No)

CUSTOMER APPROVAL OF

P.O. / CONTRACT EXCEPTIONS

(Yes/No)

* 1.2 * 1.2 * 1.2 * 1.2

Graftel PO#202000218, dated 7/9/2020

Conduct internal audit of NQA-1 program, full scope in complying

with 10CFR50 Appendix B.

Yes, Service Order 2020-09, dated 7/30/2020

N/A

Jensen Hughes, Kinetix-2020-001, dated 1/31/2020

Provide two NQA-1 lead auditors to conduct a 3 day internal audit of Jensen Hughes’s Nuclear QA Program at their Wakefield, MA

office.


N/A

EPRI,PO #470000636NQA, dated 12/8/2019

KQS will provide services to modify Read Write and Read Only

versions of the EPRI Approved Vendors List and EPRI Approved

Projects List databases.


N/A

Rizzo International, PO#12670, Rev. 1, dated 6/24/2020

Consulting/Audit Services for an Independent Audit of QHSE Program (Quality Assurance

Program)


N/A

Structural Integrity Associates, PO# CWA-20-001.KQS, R1, dated

11/5/2020

Conduct a supplier audit of SERCO North America (NIAC#

25066) no later than 80 days after expiration of supplier on current SI

ASL. Mark-ups and inputs for changes to the ASL shall be provided for vendor/supplier

audited as appropriate. Contactor will contact SI vendor/supplier

directly to schedule audit dates.


N/A

Leonardo DRS. PO# 72CP014433, dated 8/22/2019

NQA Training as described in attached Kinetix Quote


N/A

* Refers to applicable question




Page: 8 of 23

SECTION 9 – DOCUMENT CONTROL/ADEQUACY



9.2 Verify that measures are established and implemented to control the preparation, review/approval, and issue of documents (i.e., procedures, instructions, drawings, work orders, etc.) including changes.

References:

Appendix B/ANSI N45.2 - (5, 6/6, 7)

ASME Section III

NQA-1 Requirement 6



(current)


(previous)


to control the preparation, review/approval, and issue of documents (i.e., procedures, instructions, drawings, work orders, etc.) including changes:

KQS Nuclear Quality Assurance Manual 6.0 Document Control Rev. 9 1/1/2020

KQS-501, Rev. 7 Controlled Documents, 12/1/2017

b. Describe implementation of the supplier’s measures (who, what, how) to control the preparation,

review/approval, and issue of documents (i.e., procedures, instructions, drawings, work orders, etc.) including changes:

KQS-501 provides the instructions for preparation, review, approval and distribution of KQS controlled documents. Controlled documents are KQS Procedures, KQS Nuclear QA Manual and work instructions. Section 3.1.1 states a Table of Contents be maintained for all current KQS procedures. A copy of this Table of Contents was found on the Kinetix Quality Services website: www.kinetixquality.com/procedures. The latest revisions of procedures, KQS Nuclear QA Manual, and Work Instructions are kept at this location as well.

KQS controlled documents are to be reviewed for adequacy and approval for release by authorized personnel. Section 3.4.2 states that all procedures and the QA Manual will be reviewed by the QA Manager. Section 3.4.2 states that Work Instructions be reviewed by the Office Manager. Controlled documents can be created by any KQS Team member. The preparer signs the “Prepared by” line on the first page of the document. Procedures and the QA Manual are to be reviewed by the QA Manager. Controlled documents are approved by the Owner/President and not reviewed and approved by the same person. Revisions to procedures uses the same approval as the original.

Reviewed the Table of Contents and procedures. The procedures are reviewed and signed by the QA Manager. The procedures are current except for KQS-701 which lists the date of Revision 1 as 12/01/2017. The procedure was approved on 11/27/2017, However the effective date listed on the first page of the procedure is 09/15/2017. This was brought to the attention of the Owner/President and corrected during the audit. No further action is required.

WI-1 Lead Auditor Annual Review was reviewed by the Office Manager.








Page: 9 of 23

\

SECTION 10 – ORGANIZATION



10.2 Verify that adequate measures are established and implemented for management, direction, and execution of the Quality Assurance Program.

References:

Appendix B/ANSI N45.2 - (1-3)

ASME Section III

NQA-1 Requirement 1



(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) which provide the measures for

management, direction, and execution of the Quality Assurance Program:

KQS Nuclear Quality Assurance Manual Statement of QA Policy, Rev. 9 1/1/2020

KQS Nuclear Quality Assurance Manual 1.0 Organization Rev. 9 1/1/2020

KQS-101, Rev. 8, Organization, 1/1/2020

b. Describe implementation of the supplier’s measures for management, direction, and execution of the

Quality Assurance Program (who, what, how) including:

• Designation of an individual/organization responsible for defining/measuring the overall effectiveness of the QA Program;

• Adequacy of the authority, independence and organizational freedom of personnel performing verification activities;

• Regular supplier management review of the status and effectiveness of the Quality Assurance Program including corrective actions for areas found to be unsatisfactory as a result of the review.

KQS-101 Organization describes the KQS Organization and functional responsibilities.

• KQS NQA Manual Section 1.5 states the Owner/President is responsible for establishing the overall

expectations for effective implementation of the quality assurance program and obtaining the desired end result.

The KQS NQA Manual Statement of QA Policy outlines the purpose of the QA Program and assigns the QA

Manager the overall responsibility and authority to organize, approve and maintain the NQA program through

direct access to all levels of Company management and personnel.

• KQS-101 Section 3.2.1 states that the QA Manager is responsible for assuring the overall adequacy and

effectiveness of the QA program and is delegated sufficient organizational freedom; responsibility and authority

to perform quality activities. Individuals and organizations performing quality assurance functions have direct

access to responsible levels of management and have the freedom to identify quality assurance problems.

• The QA Manager is responsible for QA Program Assessments biennially on even calendar years. This

assessment is documented in a letter to the Owner/President by the end of the calendar year. The assessment

is made from collected data from: Corrective Action Reports, Internal and External Audit Reports, Procedure

Revisions, Feedback on QA Program and Procedures from Lead Auditor Training Class Practical Exercises, and

Previous QA Program Assessments. Since the last internal audit there has been no further reports generated.








Page: 10 of 23

SECTION 11 – NONCONFORMING ITEMS/PART 21


11.1 Within the assessment/summary section of each checklist question, record the procedures/instructions/drawings including the revision/date used to verify implementation

11.2 Verify that measures are established and implemented to control items which do not conform to requirements:

References:


ASME Section III




(current)

N/A KQS does not procure, process, test, operate or inspect safety-related items for the services provided.


(previous)


measures for control of items which do not conform to requirements:

b. Describe implementation of the supplier’s measures (who, what, how) for control of items which do not

conform to requirements:






11.3 Verify that measures are established and implemented to disposition items which do not conform to requirements:

Note:

Customer approval of use-as-is and repair dispositions is necessary when required by customer purchase order.

• Procedures or instructions for repair and rework must be provided.

• Repaired and reworked items must be re-inspected.

If the supplier uses a Material Review Committee or similar organization, review a sample of the meeting minutes of this organization to verify follow through on any commitments from the meeting pertaining to significant conditions adverse to quality.

References:


ASME Section III






Page: 11 of 23


(current)

N/A KQS does not procure, process, test, operate or inspect safety-related items for the services provided.


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to disposition items

which do not conform to requirements:

b. Describe implementation of the supplier’s measures (who, what, how) to disposition items which do not

conform to requirements including:

• Documented disposition by authorized personnel.

• Justification for repair and use-as-is dispositions including customer notification when required.

• Identification of procedures/instructions for rework.

• Re-inspection of repaired/reworked items.

• Evaluation for 10CFR 21 reportability.






11.4 Verify that measures are established and implemented to address posting, evaluation, notification, and reporting requirements of 10CFR21.

References:

10CFR21.3, 10CFR21.6, 10CFR21.21, 10CFR21.41



(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to address posting,

evaluation, notification, and reporting requirements of 10CFR21:

KQS Nuclear QA Manual Section 15.0 Control of Nonconforming Items Rev. 9 1/1/2020

KQS Nuclear QA Manual Section 16.0 Corrective Action Rev. 9 1/1/2020

KQS-1602, Rev. 4 10CFR21 Reporting

b. Describe implementation of the supplier’s measures (who, what, how) to address posting, evaluation,

notification, and reporting requirements of 10CFR21 including:

• Posted documents per 10CFR21.6 (a) or 10CFR21.6 (b).

• Evaluation/Notification timeframes per 10CFR21.21 (a), (b) and (d).

• Reporting content per 10CFR21.21 (d).

• Copies of 10CFR Part 21, KQS-1602, Rev. 4 and Section 206 of the Energy Reorganization Act of 1974

are posted in the Kinetix Quality Services office. The postings were verified by photograph. Electronic

documents of the postings are posted at www.kinetixquality.com/Part21. The copy of 10 CFR Part 21 is

dated 9/9/2019, however the most recent revision of this regulation is dated September 15, 2020. This

http://www.kinetixquality.com/Part21




Page: 12 of 23

was brought to the attention of the Owner/President and was resolved during the audit. No further

action is necessary.

• KQS-1602 Rev. 4 Section 3.5.2 states the QA Manager is responsible for evaluating the conditions on

the Corrective Action Report within 60 days of discovery to determine if deviations could create a

substantial safety hazard. Section 3.5.3 continues that if the impact on performance cannot be

determined within five working days of discovery the customer should be notified in writing by the

Owner/President. Section 3.6.1 states that if the defect or noncompliance is reportable under 10 CFR

21, the President/Owner shall be notified.

• KQS-1602 Rev. 4 Section 3.6.1 states the President/Owner is responsible for notifying the NRC within 2

days of the determination. A follow-up certified letter is due to the NRC within 30 days of notification.

The procedure lists items to be included in the letter in accordance with 10 CFR 21.21(d)(4).

• Comment #1 from 2019 KQS Internal Audit: KQS-2019-01-BFC was addressed and KQS-1602 was

revised to reflect the recommendation.

• KQS issued no Part 21 notifications or reports since the previous audit.








Page: 13 of 23

SECTION 12 – INTERNAL AUDITS



12.2 Verify that measures are established and implemented to ensure a comprehensive system of planned and periodic internal audits.


Identify any corrective actions resulting from the audits to the audit team member evaluating Checklist Section 13.

Identify the Auditors to the audit team member evaluating Checklist Section 14.

References:


ASME Section III




(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to ensure a

comprehensive system of planned and periodic internal audits:

KQS NQA Manual Rev. 9 1/1/2020 Section 18, Audits

KQS-1801 Rev. 8 1/1/2020 Internal Audits

KQS-1010, Rev. 8 1/1/2020 Organization

KQS Internal Audit Schedule

b. Describe implementation of the supplier’s measures (who, what, how) to ensure a comprehensive

system of planned and periodic internal audits including:

• Planning/scheduling

• Auditor independence

• Adequate objective evidence to support the conclusions

• Management review of audit results

• Evaluation of corrective action effectiveness from previous audits

• KQS-1801 Section 3.1.1 states that Lead Auditors are responsible for planning, performing, reporting

results and audit follow-up. Section 3.2.1 states the Owner/President develops the annual audit

schedule for all planned internal audits. Internal audits are based on the status and importance of

activities to ensure adequacy and conformance to the quality program. Section 3.3 discusses the Audit

Planning instructions including the Audit Plan. Audits are conducted using a checklist or by a

procedure. Internal audits are to be conducted annually.

• KQS-1801 Section 3.2.3 states that the Owner/President shall assign personnel to conduct audits that

are qualified in accordance with KQS-203. Audit team personnel are assigned to activities with which

the audit team member has not been involved.

• KQS-1801 Section 3.4.1 states the Lead Auditor shall assure that sufficient objective evidence is

recorded on audit checklists or documented and traceable when other methods are used in lieu of

checklists.

• KQS-1801 Section 3.6.3 requires that management review of audit issues be documented in the QA

Records System. The QA Manager is responsible for preparing an Assessment of the QA Program on

a biennial basis. This assessment is documented in a letter to the Owner/President and kept as a QA

Record.




Page: 14 of 23

• KQS-1801 Section 3.3.1 states that issues from previous audits be evaluated in the next audit. Section

3.5.1 lists that a summary of follow-up, closeout and effectiveness of previous corrective action be

included in the Audit Report. The evaluation of Corrective Action is documented in the QA Program

Assessments issued in even calendar years.






12.3 Assess the overall effectiveness of the internal audit process by review of previous internal audits and comparison of the results/issues identified in these audits with those identified by this NUPIC audit.

References:


ASME Section III




(current)


(previous)

ASSESSMENT/SUMMARY: a. Describe the comparison of previous supplier internal audit results/issues with those identified by this

NUPIC audit:

Internal Audit KQS-2019-01-BFC was reviewed as part of this audit. The audit had one finding and two comments. CAR #2019-01 was generated in response to the finding on October 1, 2019. The CAR was closed on 1/8/2020. No recurring issues were observed. Management reviewed the audit issues from this audit on 4/14/2020.

b. Has the internal audit process been effective, overall?





Page: 15 of 23

SECTION 12 – INTERNAL AUDITS (FIGURE 12)

AUDIT SCOPE and

DATE(s)

AUDITOR(s)

NUMBER OF DEFICIENCIES and

STATUS (OPEN / CLOSED)

CORRECTIVE ACTION VERIFICATION

METHOD (document review

follow-up audit, surveillance, etc.)

* 12.2 * 12.2 * 12.2 * 12.2

KQS-2019-01-BFC/Full Scope 10CFR50, App. B; 10CFR21:

NQA-1:2008

Bradley F. Cellier, Lead Auditor

Richard K. Martin, Lead Auditor In Training

1 Finding – closed Document Review





Page: 16 of 23

SECTION 13 – CORRECTIVE ACTION



13.2 Verify that measures are established and implemented to assure that conditions adverse to quality are promptly identified and corrected.

Note:

The supplier’s program should define “significant” conditions adverse to quality.

If the supplier uses a Corrective Action Review Board or similar organization, review a sample of the meeting minutes of this organization to verify follow through on any commitments from the meeting pertaining to significant conditions adverse to quality.

References:


ASME Section III




(current)


(previous)


to assure that conditions adverse to quality are promptly identified and corrected:

KQS NQA Manual Rev. 9, 1/1/2020; Section 16 Corrective Action

KQS-1601, Rev. 5 Corrective Action 3/31/15

b. Describe implementation of the supplier’s measures (who, what, how) to assure that conditions adverse

to quality are promptly identified and corrected:

KQS-1601, Rev. 5 Corrective Action specifies the method by which adverse conditions are identified, documented, evaluated and resolved. Figure 2 is used to determine Significant Conditions Adverse to Quality (SCAQ). A significant condition adverse to quality is defined in the procedure as one that if uncorrected could have a serious effect on safety or operability. The figure lists examples of SCAQ.

Corrective Action Reports are to be written whenever one of the conditions are identified: Quality System process breakdowns, inadequate or deficient procedures, deviations from defined quality program and/or procedure requirements, deficiencies discovered during internal and outside audits. The CAR form example is found in the procedure.

All personnel working under the quality program are responsible for promptly reporting conditions adverse to quality. The President/Owner is responsible for assuring adverse conditions are investigated and measures are taken to preclude recurrence.

A CAR is generated by the person who identified the condition adverse to quality. The CAR Number is obtained from the QA Manager. After obtaining the CAR form and number, the initiation portion of the form is completed by the person who identified the condition. The CAR is reviewed by the QA Manager to determine if Stop Work action is required or if it is a SCAQ. If the condition is a SCAQ, the QA Manager is responsible for reviewing for 10CFR21 applicability. The CAR is issued to the appropriate person by the QA Manager.

The assigned individual reviews the CAR and evaluates to determine actions to correct the condition. If it is a SCAQ, determination of root cause and actions to preclude are determined. Figure 3 offers guidance for Root Cause Determination using the 5 Whys method. The CAR form is returned to the QA Manager within 30 days unless otherwise specified.

The CAR Response Review is done by the QA Manager to determine if the corrective/preventative actions appear sufficient. If the response is unacceptable, it is returned to the assigned person. If acceptable, the QA Manager indicates acceptance on the form.




Page: 17 of 23

Following CAR Response Acceptance, the QA Manager verifies the actions with objective evidence. If the condition is corrected, the CAR is closed by the QA Manager. If the actions taken were insufficient, the process begins again.

CARs are kept as QA Records.

CARs from audits of KQS are reviewed and concurred by the Lead Auditor (internal) or the customer (customer).






13.3 Verify that deficiencies identified/reported by customers, to the supplier, (e.g., receipt inspection rejections, source verification rejections, return material authorizations, site nonconformances, etc.) are adequately evaluated and entered into the supplier’s nonconformance or corrective action program, as applicable.

References:


ASME Section III




(current)


(previous)


to assure that deficiencies identified/reported by customers, to the supplier, are entered into the supplier’s nonconformance or corrective action program, as applicable, and adequately evaluated:

KQS NQA Manual Rev. 9, 1/1/2020; Section 16 Corrective Action

KQS-1601, Rev. 5 Corrective Action 3/31/15

b. Describe implementation of the supplier’s measures (who, what, how) to assure that deficiencies

identified/reported by customers, to the supplier, are adequately evaluated and entered into the supplier’s nonconformance or corrective action program, as applicable:

No customer CARs were generated since the last internal audit. The procedure for CARs generated by customer audits are the same as described in Section 13.2 b of this audit checklist.






13.4 Verify the overall effectiveness of the corrective action process.





Page: 18 of 23

Adequacy of corrective actions taken, as a result of the issues identified during the last NUPIC audit, will be provided by audit team members assigned to the checklist sections which identified the previous issues.

References:


ASME Section III




(current)


(previous)

ASSESSMENT/SUMMARY: a. Describe the adequacy of the corrective action process in preventing recurrence of previously identified

issues:

Reviewed CAR #2019-01. During the 2018 KQS Internal Audit one of the Audit Team members, a Lead Auditor in Training and a Technical Specialist did not fill out form KQS 203.2. This was evaluated to not having a negative impact on the audit. Procedure KQS-1801 was revised to clarify who is to receive training/qualification. The 2020 Graftel Internal Audit was conducted under the KQS NQA program. Both auditors were qualified using KQS 203.1. The corrective action was adequately resolved.

b. Has the corrective action process been effective, overall?





Page: 19 of 23

SECTION 14 – TRAINING/CERTIFICATION



14.2 Verify that measures are established and implemented to ensure quality program indoctrination and training of personnel who perform activities affecting quality.

References:


ASME Section III

NQA-1 Requirement 2



(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to ensure quality

program indoctrination and training of personnel who perform activities affecting quality:

KQS NQA Manual, Rev. 9 1/1/2020, Section 2.0 QA Program

KQS-201, Rev. 8 1/1/2020, KQS Personnel Training

KQS-203, Rev. 6 12/1/2017 Qualification/Certification of Audit Personnel

KQS-1801, Rev. 8, 1/1/2020 Internal Audits

b. Describe implementation of the supplier’s measures (who, what, how) to ensure quality program

indoctrination and training of personnel who perform activities affecting quality:

KQS-201 Rev. 8 describes the instructions for the control and documentation for indoctrination and training for KQS personnel. Indoctrination and training is conducted by qualified personnel. Indoctrination is provided by KQS for newly assigned personnel. The indoctrination is to familiarize personnel with the quality assurance requirements. The implementation of the quality assurance program is mandatory and enforced. Re-indoctrination is scheduled when there are revised or new codes/standards resulting in major changes to the KQS QA Manual or Procedures.

Continuing and new training is determined by the Office Manager. Coordination of training and documentation is done by the Office Manager.

Personnel completing training sign the Attestation Form (KQS-201.1). The forms are returned and reviewed by the Office Manager.

KQS-203, Rev. 6 Qualification/Certification of Audit Personnel provides instructions necessary to implement the requirements for qualification and certification of KQS audit personnel. Section 3.3 instructs how Lead Auditors are qualified initially and the process is documented on Form KQS-203.1. The procedure meets NQA-1-2008 Requirement 2 for qualification of Lead Auditors. Auditors who are qualified by other organizations are acceptable to use by KQS along with completing Form KQS-203.1.









Page: 20 of 23

14.3 Verify that inspection/test personnel, auditors, calibration, repair personnel and similar specialists (i.e., ASME Code design personnel to ASME Section III) are qualified and have certifications on file.

Note:

Special process personnel Qualification / Certification is addressed in Checklist Section 7.

References:

Appendix B/ANSI N45.2 - (2, 9, 10, 11, 18/2, 10, 11, 12, 19)

ASME Section III

NQA-1 Requirement 2



(current)


(previous)

2019 Internal Audit Finding #1

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to ensure that

inspection/test personnel, auditors, calibration, repair personnel and similar specialists are qualified and have certifications on file:

KQS NQA Manual, Rev. 9 1/1/2020, Section 2.0 QA Program

KQS-201, Rev. 8 1/1/2020, KQS Personnel Training

KQS-203, Rev. 6 12/1/2017 Qualification/Certification of Audit Personnel

KQS-1801, Rev. 8, 1/1/2020 Internal Audits

b. Describe implementation of the supplier’s measures (who, what, how) to ensure that inspection/test

personnel, auditors, calibration, repair personnel and similar specialists are qualified and have certifications on file:

KQS-201, Rev. 8 KQS Personnel Training describes the instructions for the control and documentation of indoctrination and training for KQS personnel. The training is documented on Form KQS-201.1, KQS Attestation Statement. The Office Manager reviews each Attestation and is kept as a QA Record. Personnel who prepare, review or approve procedures or the QA Manual are considered trained in the applicable portions of the NQA Program.

KQS-203, Rev. 6 Qualification/Certification of Audit Personnel provides instructions necessary to implement the requirements for qualification and certification of KQS audit personnel.

Lead Auditors are certified using Form KQS-203.1 using the 10 point scoring system described in Figure 2.

The Lead Auditor is responsible for ensuring the proper training and orientation is provided for the auditors used on their audit. This training/orientation is documented on the Auditor/Technical Specialist Indoctrination form found in Figure 3 of the procedure (Form KQS-203.2)

The Audit Team reviewed the Attestation Forms (KQS 201.1), Record of Lead Auditor Qualifications (KQS 203.1), and Auditor/Technical Specialist Indoctrination Form (KQS 203.2) as applicable to the following personnel:

Bradley F. Cellier, Lead Auditor

Richard J. Martin, Auditor

Kyle G. Robinson, QA Manager

The Audit Team reviewed KQS 2019 Internal Audit (KQS-2019-01-BFC) Finding #1. CAR #19-01 was issued and closed based on the finding and is adequately resolved.








Page: 21 of 23

SECTION 14 – TRAINING/CERTIFICATION (FIGURE 14)

NAME and

JOB TITLE

INDOCTRINATION and TRAINING COMPLETED

(Yes / No)

QUALIFICATION / CERTIFICATION TYPE and

LEVEL

* 14.2, 14.3 * 14.2, 14.3 * 14.2, 14.3

Bradley F. Cellier Lead Auditor

Yes, Expires 9/1/2021 NQA-1 Lead Auditor

Richard J. Martin

Audit Team Member

Yes, Completed 9/15/2019 Technical Specialist, Lead Auditor in Training

Kyle G. Robinson Yes, Reviewer of Documents; No attestation required

QA Manager





Page: 22 of 23

SECTION 15 – FIELD SERVICES



15.2 Verify that measures are established and implemented to control field services.


Query the audit team members regarding applicability of field services to their assigned checklist sections.

References:

Appendix B/ANSI N45.2 - 2/2

ASME Section III



(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to control field

services:

KQS NQA Manual, Rev. 9 1/1/2020, Section 2.0, Organization

KQS-203, Rev. 6 12/1/2017. Qualification/Certification of Audit Personnel

KQS-204, Rev. 1 1/1/2020, Customer Training Classes

KQS-1802, Rev. 4, 12/1/2016, Conduct of Customer Audits & Surveys

b. Describe implementation of the supplier’s measures (who, what, how) to control field services:

• Verify the controls for these services have been evaluated in the appropriate sections of the checklist and list the checklist sections determined to be applicable.

KQS provides qualified and trained personnel and customer training as field services. Personnel are trained and qualified under the KQS QA Program.

See Section 14 for discussion on training/qualification of Auditors and Technical Specialists.

KQS-204, Rev. 1 Customer Training Classes describes the methods to be used for scheduling, planning and conduction customer training classes. KQS uses the ADDIE model for the design basis of training courses. Training courses are scheduled by or for the President. Scheduling and notification, learning objectives, course materials and attendance, training feedback and examinations are discussed in the procedure.

KQS-1802, Rev. 4 Conduct of Customer Audits & Surveys describes the methods to be used by KQS for scheduling, planning, performing, reporting and follow-up audits and surveys. Lead Auditors performing NIAC Audits/surveys are required to have additional NIAC training. The procedure outlines the prescribed formats for Audit plan and reports.








Page: 23 of 23

SECTION 16 – RECORDS



16.2 Verify that adequate measures are established and implemented to ensure that all QA records not transferred to the

member are maintained in facilities that provide storage, retention requirements and protection against environmental

effects, damage and loss.


Query the audit team regarding the condition of any quality records which they have reviewed.

References:


10 CFR 21.51

10 CFR 52

ASME Section III




(current)


(previous)

ASSESSMENT/SUMMARY: a. List the Vendor Quality Manual reference and implementing procedure(s) established to ensure that all

QA records not transferred to the member are maintained in facilities that provide storage, retention requirements and protection against environmental effects, damage and loss:

KQS NQA Manual, Rev. 9 1/1/2020, Section 17.0 QA Records

KQS-1701, Rev. 8, 1/1/2020, Quality Assurance Records

b. Describe implementation of the supplier’s measures (who, what, how) to ensure that all QA records not

transferred to the member are maintained in facilities that provide storage, retention requirements and

protection against environmental effects, damage and loss including:

• Records are legible, identifiable, and retrievable.

• Records are retained and maintained per 10CFR21.51 including those for 10CFR52 licensed plants.

KQS-1701, Rev. 8 Quality Assurance Records provides instructions for the retention of records of Kinetix Quality Services activities. Documents generated during the performance of KQS activities are reviewed for: legible and capable of reproduction for duplicate storage, identifiable to an activity, authentication by the originator, completeness, free of entry corrections made by unacceptable methods.

Non-permanent records are retained for a minimum of ten (10) years.

Records storage was demonstrated via video conference on 11/12/2020. Record storage is on the President/Owner’s computer as well as a Microsoft SharePoint site. The retrievability of records was verified during the videoconference.

Record storage verification was performed during videoconference demonstration of the Service Order entry in Section 1 of this checklist.

The Audit Team reviewed Comment #2 from the 2019 KQS Internal Audit. The issue was adequately resolved.





Memorandum

Date: November 2, 2020

To: Chris Kulwicki

Owner/President Kinetix Quality Services

318 S Main Street

Albion, NY 14411

From: Mark Mohar, Lead Auditor in Training

Subject: NQA-1 Internal Audit of KQS

At your request, the audit of Kinetix Quality Services has been scheduled to commence on November 9,

2020.

Attached for your reference is the internal audit plan including the proposed audit schedule. The audit

team for this audit will be Frank McGee and Mark Mohar.

The primary focus of this audit will be an evaluation of the adequacy of controls of the Kinetix Quality

Services for providing quality services including customer audits and surveys, and customer training for

lead auditor candidates in accordance with the Kinetix Quality Services Nuclear Quality Assurance

Manual.

The Audit Entrance Meeting will be held at your facility on Monday, November 9, 2020 at 9:00 AM EST,

per teleconference. Due to the COVID-19 pandemic, the audit will commence remotely by review of

electronic records and video conferencing. An on-site visit is not expected to be necessary for this audit.

The Audit Exit Meeting will be held once all of the audit activities have been completed.

Thank you in advance for your cooperation and consideration in scheduling this activity. If you have any

questions or inputs, please contact me at your convenience at (612) 791-1927 or

[email protected]

Attachments: Audit Plan

mailto:[email protected]

Kinetix Quality Services Internal Audit Plan

Date: 11/2/2020

Audit Identification Number:

KQS-2020-07-FDM

Audit Team: Frank McGee Audit Team Leader/Lead Auditor

Mark Mohar Audit Team Member – Lead Auditor in Training

Audit Type: Internal – NQA-1 Compliance

Audited Organization Kinetix Quality Services

318 S Main Street

Albion, NY 14411

Audit Contact Chris Kulwicki, Owner/President

Audit Date(s):

November 9, 2020 through Completion Date

Audit Scope: The audit scope will evaluate the following elements as applicable

to the KQS Quality Assurance program: Order Entry/Contract

Review, Document Control Adequacy, Organization/Program,

Nonconforming Items/Part 21, Internal Audits, Corrective Action,

Training/Certification, Field Services, Corrective actions or

comments identified in previous audits, and Records.

Audit Bases/Standards: ASME NQA-1, 2008 as amended by the 2009 addenda

(Applicable Elements)

10CFR50, Appendix B (Applicable Elements)

KQS Quality Assurance Manual and KQS Procedures

KQS Internal Audit 2019-01-BFC

Audit Checklist: Nuclear Procurement Issues Committee (NUPIC) Audit Checklist,

Rev. 21

Audit Schedule: Entrance Meeting: November 9, 2020 9:00am EST

Exit Meeting: November 13, 2020, time TBD (anticipated)

Reviewed and Approved: ____________________________________________

Lead Auditor

11/18/2020