Eiar dow final report

EIAR & Dow Leadership in Action Final Report

August 13-15 2014

Trip Overview

Lab Audit Findings

Prioritization & Multigenerational Plan

John DiMuro, Matt Miller, David Stenhouse, Leo Hu, & LeAnn Bruns

October 7, 2014

Table of Contents

Trip Overview ................................................................................................................................................ 3

Agenda ...................................................................................................................................................... 3

Attendees .................................................................................................................................................. 5

Project Overview ....................................................................................................................................... 6

Audit Tool Overview ................................................................................................................................. 6

Change Management Overview ............................................................................................................. 18

Establishment of an Interdisciplinary Executive Team ........................................................................... 19

Summary of Audit Findings ..................................................................................................................... 20

Lab Audit Details ......................................................................................................................................... 23

Prioritization and Multi-Generational Plan ................................................................................................. 30

Multi-Generation Plan ............................................................................................................................ 31

Key concepts for the Interdisciplinary Executive team to consider ....................................................... 32

Executing and Implementing Change – Making it “stick” ....................................................................... 32

More Resources ...................................................................................................................................... 33

Trip Overview

Agenda: August 13-15, 2015

Time table Program Responsible When

Day-1 (9:00-9:45)

Welcoming of DOW team to EIAR

EIAR team (Solomon A.)

13 August 2014

(9:45 -10:00) Keynote speech by DDG of EIAR

(Dr. Aduja W.)

(10:00 - 10:15)

Short description on Ethiopia (the country)

EIAR team (Gelila A.)

(10:15 - 10:30)

EIAR profile

(Derese T.)

(10:30 - 11:00)

Coffee break

(11:00 - 12:00) Short brief about DOW DOW team

(12:00 - 01:00) Description on the state of EIAR Laboratories (Part 1)

EIAR team (Solomon A.)

(01:00 - 02:00) Lunch

EIAR Cafeteria

(02:00 - 02:30)

AQRL statues Description on (in the process of accreditation

EIAR team (Gelila A.)

(02:30 - 03:30) Practical visit to the AQRL:[all formats, working procedures and working instructions will be reviewed]

AQRL- team

(03:30 - 04:00) Coffee break

(04:00 -05:00)

General discussion and feedback

DOW and EIAR team

Day -2 (09:00-10:00)

Safety Presentation

DOW team

14 August, 2014

(10:00 - 11:00)

Dow team to use Global Analytical Lab tool to train EIAR - AQRL

DOW team

(11:00 - 11:30)

Coffee break

(11:30 - 01:00) Dow team to use Global Analytical Lab tool to train EIAR - AQRL

(01:00 - 02:00) Lunch

(02:00 - 03:30)

Dow team to use Global Analytical Lab tool to audit EIAR - AQRL

DOW team

(03:30 - 04:00)

Coffee break

(04:00 - 05:00) Dow team to use Global Analytical Lab tool to audit EIAR AQRL

Day -3 (09:00 - 11:00)

Debrief/results review and DOW experience sharing in managing Laboratories

DOW team August 15, 2014 8:30-11:30

(11:00 - 11:30) Coffee break

(11:30 -01:00) Presentation from Dow on Prioritization/Multigenerational Plan /Management of Change

DOW team and EIAR team

(01:00 - 02:00) Lunch

(02:00 - 4:00)

Seating milestones/ schedule that EIAR will follow in implementing DOWs experience

DOW and EIAR team

(04:00 - 05:00)

Coffee break & closing of training

Attendees: EIAR participants:

1. Dr. Fentahun Mengistu, director general of EIAR 2. Dr. Adugna Wakijira, deputy director general of EIAR 3. Solomon Abate, director for Agricultural and Nutritional Research Laboratories 4. Gelila Asaminew, lab head of Agricultural Quality Research Laboratory @ HQ. EIAR team

member in the project 5. Bilatu Agza, researcher in AQRL. EIAR team member in the project 6. Quality management Representatives of Research Centers (listed below) 7. Laboratory heads of Research centers (listed below)

Addis Ababa Mr. Belatu Agza

Mrs. Gelila Asamenew Mr. Dinka Mulugeta Mr. Samuel Mesfin

Mr. Hailu Reta Mr. Yibrah Amare

Mr. Aytenew Abitew

Holleta Agricultural Research Center Mr. Dereje Fikadu Mr. Zerihun Assefa

Wondo Genet Agricultural Research Center

Mr. Beriso Measo Mr. Beemnet Mengesha

Melkassa Agricultural Research Center

Mr. Kebede Dinkecha

Debre Zeit Agricultural Research Center Mr.MohamedYimam

Mr. Melese Menaleshewa

Jimma Agricultural Research Center Mr. Tesfu Kebede

Werer Agriclutural Research Center

Mr. Mulate Zerihun

Dow Chemical attendees:

1. John DiMuro 2. Leo Hu 3. Matt Miller 4. David Stenhouse 5. LeAnn Bruns

Project Overview: Through discussions with EIAR leadership, it was determined that the key project deliverable was a description of what a “world class” laboratory management program consisted of, and recommendations on how EIAR could establish such a program. EIAR leadership also wished to receive guidance on how disparate labs could be unified through common work processes and procedures. The project was therefore designed with two major elements to achieve this vision. The core of the project was an Audit Tool, which not only can be used to perform a gap assessment on individual labs, but which itself describes the management systems necessary to achieve world-class laboratory management practices. It highlights actions and programs that, if implemented consistently throughout the EIAR analytical labs, would set EIAR on the path to world-class laboratory management. The second element of this project was recommendations on how to use the tool as a platform for change and lab unification. Execution of EIAR’s strategy is critical, especially considering the multiple labs spread over a wide geographical area. This report highlights some key concepts discussed during the project, and describes how the Audit Tool can be used as a means for the execution of both lab improvements and EIAR’s lab unification strategy.

Audit Tool Overview: A key deliverable for the Dow/EIAR partnership was a gap and performance assessment of their labs from an analytical quality perspective. After consultation with company experts, discussion with the Analytical Technology Center uncovered a tool used within Dow to understand gaps within Analytical labs. This tool is named the “Global Analytical Performance Scorecard.” The tool is designed to evaluate laboratories with the purpose of setting expectations and measuring performance against ideal analytical practices. The audits occur across ten different dimensions, consisting of additional individual elements, and are listed below: Enablers: these are broad organizational items that promote people/client/instrument satisfaction and cohesiveness

Processes: items around how work is agreed to, planned, and executed Calibration: instrument calibration and standards Preventive Maintenance: keeping instruments working and tracking repairs Training: ensuring people are qualified and have knowledge they need Analytical Instrument Qualification: ensuring instruments can work as needed Statistical Quality Control: tracking performance of equipment Standard Operating Procedures: uniform execution of tests and processes Unplanned Events Management: planning for the unknown before it happens EH&S: making sure people are working safely Best Practices: items to ensure high performing teams and tests Each element has audit criteria that provide guidance as to what level of performance is needed to meet a rating of 0, 3, 6 or 9. It’s also possible to adjust the weighting of each individual element so pre-determined emphases may be leveraged. There was also a list of questions developed for each dimension to facilitate obtaining data to judge each element. Paired with this are individual pieces of data that auditors should look for in order to judge each element. The tables below are the questions and audit evidence for each dimension.

Table 1: Enablers dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

<100% of the direct report

workgroup members have an

Employee Development Plan (EDP)

in place.

Everyone has an EDP Same as 3.

There's a review process in place

for all the EDPs to ensure they are

still relevent

Same as 6.

Share the succession plan for

current leadership/expertise-

requiring roles

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A Need to share the data and the

rating scale so that the 70% is

understood. Questions need to be

challenging enough to be an arbiter

of performance.

Same as 3 - but 85% Same as 6 - but 100%

Element

Score 0 3 6 9

Evidence I am

Looking For

List of lab equipment and list of

MET showing that <50% is being

met

Same as 0, but 50-75% Same as 3, but >75% Same as 6, but 100% and there's

management-approved variances

that can be shown for non-

conformances.

1) Can you show me an employee development plan for at least 3 different employees in 3 different roles?

2) How do you track the review of EDPs? How do you hand these off to new leaders?

3) What's the process for getting client satisfaction survey data?

4) Can you share some results?

5) How often do you collect employee satisfaction data - what drove that decision?

6) How do you use the data to improve your service/performance?

7) What's the process/procedure for determining MET? Can we see the list?

8) Can you share your MET standard documents (procedures)?

9) How do you reconcile different equipment at different sites for the same task?

10) Can we verify that a MET procedure corresponds to the procedure/equipment in the lab? [Pick 4 METs/4 procedures]

Employee Development Plans (EDP's)

Client Satisfaction

Most Effective Technology (MET) standards Implemented

Table 2: Processes dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A There's some discussion, but no

formal document, about testing

plans for customers

Written agreement about testing

exists and contains commitments

for turnaround and priorities of

samples

Same as 6, and the service

agreement is reviewed annually

and is signed-off on by lab and

customer leadership.

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A There's some understand of what a

non-routing request would be and

that it needs to be handled

differently than routine work.

There's a documented process for

non-routing work and this process

contains verbiage around assessing

value of these requests so they can

be prioritized against normal work.

Same as 6, and there's historical

data kept on these requests.

There's also a clear record of

improvement decisions based on

these data.

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A A sample plan with customers Same as 3, but there is some review

of the document every 3 years at a

minimum.

Same as 6, but there is a process for

having a review of the agreement

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A There's a documented sample data

storage process. Samples can be

logged into a system.

Same as 3, and there's evidence of

improvement projects based upon

reviewing this system

Same as 6, and there's a formal

continuous improvement strategy

and the sample data are

electronically mined

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A Samples are logged in and data are

storged, no productivity

measurements

Same as 3, and there's a

productivity analysis

Same as 6, and there's a record of

actions taken via the productivity

analysis

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A There's a 3-5 year capital plan and

there's proof of annual review

Same as 3, and there's a readily

available instrument inventory that

can be shared.

Same as 6, and there's leveraging of

the capital plan into the purchasing

process to get discounts

Element

Score 0 3 6 9

Evidence I am

Looking For

N/A Documented process for

communicating/managing change

Same as 3, but there's an electronic

tool for communicating and

managing change

Same as 6, but there's a history of

auditing the process for

improvement opportunties

1) Can you share some service-level agreements with various other organizations, if they exist?

2) Can you share the process for handling non-routine requests to ensure that good data are being provided?

3) What is the sample plan for the various projects which require more routine testing?

4) How do you store data about samples and data - can you show this system? Can you show some results? What system do you have to track

efficiency?

5) Can you share your inventory and future capital purchase plans?

6) How do you communicate changes in procedures or updates to MET to the organization?

Management of Change (MOC) Process

Sample and Data Management Process

Sample and Work Load Management

Lab Inventory & Capital Plan

Service Level Agreements (SLA)

Non-Routine Request Process

Sample Plan

Table 3: Calibration dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No schedule Documentation on calibration for

all equipment

Proof of fully followed procedures

on calibrations for all equipment

Same as 6, but there's an automatic

schedule for calibrations that

prompts the tests. SQC data is used

to help determine calibration

frequency.

Element

Score 0 3 6 9

Evidence I am

Looking For

No documentation There's detailed instructions on

calibration procedures on <50% of

equipment

Same as 3, but 100% Same as 6, but the procedures also

have guidance on what to do if

there's failures of calibration

Element

Score 0 3 6 9

Evidence I am

Looking For

Required external calibrations are

not performed

Vendor does appropriate

calibrations, but there's no proof or

certification of the vendors

The vendor provides

documentation on traceable

standards and documentation on

what alterations were made to the

instruments

Same as 6, but there's additional

review by a lab SME and there's an

assessment of previous data quality

if there's poor calibration

Element

Score 0 3 6 9

Evidence I am

Looking For

Test materials are expired or audit

data says expired materials were

used

All test materials have expiration

dates that are in the future or

analytical data proving they are OK

if expired.

Same as 3, but there's procedures

for testing standards upon arrival

Same as 6, and all the standards are

NIST (or similar) traceable to be

certified/primary references.

Element

Score 0 3 6 9

Evidence I am

Looking For

No records Calibration data is electronically

stored

Same as 3, including "as found" and

"as left" data along with the raw

calibration data exist.

Same as 6, and the

current/appropriately historic data

are obtainable via a LIMS system

6) Where do you keep your calibrations procedures? Can we see them?

5) Do you have a calibration procedure, and can see records for X instrument calibrations?

2) What kind of materials are used as standards for your calibrations - where do you obtain these from?

3) Can you show me your calibration records for select equipment?

4) Can you show me data on what happens when an instrument fails calibration checks?

7) Can you show me records on vendor calibrations? Is there a sign-off for a EIAR lab SME?

8) Can we see your standards and reagents? What do the expiration dates look like - check this in person?

Standards and Reagents

Calibration Records/Reports

Calibration and Calibration Verification Schedules

Calibration and Calibration Verification Instructions (Internal calibrations)

Calibration by External Vendors

Table 4: Preventive Maintenance dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No documentation There's an informal strategy with

limited documentation

There's a formal maintenance

strategy that's documented with a

priority for certain critical

instruments

Same as 6, and there's an FMEA/risk

analysis for critical components.

There's a formal review to ensure

the strategy is implemented.

Element

Score 0 3 6 9

Evidence I am

Looking For

No procedures Some PM procedure

documentation

All PM procedures are documented Same as 6, and there's details in the

procedures about how to document

the PM

Element

Score 0 3 6 9

Evidence I am

Looking For

No schedule There's a PM schedule for >33% of

instruments

There's a PM schedule for all

instruments

There's a PM schedule with

automatic triggers and a record

proving full PM compliance

Element

Score 0 3 6 9

Evidence I am

Looking For

No spare parts list All quality critical devices have a

spare parts list

Same as 3, and there's enough

inventory to prevent a PM on

critical equipment

Same as 6, and there's a strategic

evaluation of what spare parts

need to be on hand for unexpected

breakdown of the most critical

equipment

Element

Score 0 3 6 9

Evidence I am

Looking For

No records Paper records (logbook) of PM

activities

Electronic records of PM activities Same as 6, and the historic data is

retrievable for understanding

trends

1) Can you show the maintenance records for your equipment?

Do you have a procedure for determining maintenance strategy? Does it review risks of not performing the tests?

Do you have PM procedures? Can we see them?

2) How do you determine the PM schedule for your equipment? Can you share the schedule?

Do you have records of the PMs actually being performed? Can we see them?

3) Do you keep a formal inventory of spare parts?

4) Do you have a determination of what equipment or parts are critical to quality and what is not? What is that and can we see it for your critical

equipment?

Equipment Spare Parts Inventory

Maintenance Records/Reports

Maintenance Strategy

Preventive Maintenance (PM) Procedures

Preventive Maintenance (PM) Schedule

Table 5: Training dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

Training plan exists for the Lab

Analyst (need to work out

equivalent role)

Training Records Exists for all Lab

Analysts

Training plan exists for the Lab

Analyst and Lab Technologist (need

to work out equivalent roles)


Analysts and Technologists.

List of All Lab and Quality personnel

with their job title and roles

available.

Training Plan exists for all

personnel and there is appropriate

training modules matching their

roles and responsibilities.


and Quality personnel

Same as score for 6 +

Every training module/procedure

has a completed competency

assessment.

Element

Score 0 3 6 9

Evidence I am

Looking For

No assessment/qualification

process evident

Qualification process is

documented in their work process

procedures.

There is an

assessment/qualification for each

training module/procedure


Assessment sheet must include:

Name of Trainee, training module

title, date of assessment, Assessor

name, Questions to prove trainee

understands the training received.

Assessor sign off that the trainee

completed the practical task

sucessfully (only required for

training on practical tasks e.g.

titration, chromotography etc)

Final sign off by trainee + assessor

with date.


Work process includes rules on re-

assessment/requalification

Element

Score 0 3 6 9

Evidence I am

Looking For

MET training completed by <50% of

the work group.

MET training completed by 50% to

75% of the work group

MET training completed by >75% of

the work group.

MET training completed by 100% of

the work group.

Plan exists with dates for all

personnel to get refresher training

Element

Score 0 3 6 9

Evidence I am

Looking For

No Training Records exist Traning Records exist

Training Record has trainee name,

trainer's name, title of training, and

date when the trainee completed

training.


Training Records have

- Questons answered.

- Assessor has marked the answers

and indicated a pass or failing mark.

- Assessor sign off that the trainee

completed the practical task

sucessfully (only required for

training on practical tasks e.g.

titration, chromotography etc)

- Final sign off by trainee + assessor

with date.


Training records show the date all

personnel have to complete

requalification. This time period

meets work process (MET)

requirements.

Training records show

requalification completed by

requalification date

1) Can you show me a list of all the Laboratory and Quality personnel with their job title, roles and responsibilities?

2) Can you show me the training plan required for each Job title?

3) Can you show me the Training record (record of training completion and date + Date of required Requalification) for the Laboratory and Quality

personnel?

4) Do you have an assessment process as part of each of your training modules/procedures?

5) Does the assessment process involve a practical assessment of the Task?

6) Can you show me in your work process (MET) procedures the section on qualification and requalification?

7) From the list of training modules/procedures check that there is an assessment file/sheet for each training module and procedure

8) Pick 3 training modules/procedures and review assessment sheets.

9) Can you show me where the you keep the training assessment records (Records of each individuals sign off for each training

module/procedure)?

10) Look at the Overall Training record and pick 3 records shown as completed and find the completed assessment forms. Review Assessment

against required standard

Training Records

Employee Training Program

Qualification Process

Most Effective Technology (MET) Training

Table 6: AIQ dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No AIQ program Documentation on DQ/IQ/OQ that

has parameters for proper

operation

There's PQ data showing stability

(e.g. check standards run every

week)

Same as 6, but even higher level

plan around qualifying instruments

that are currently not qualified.

Risk-based mechanism to manage

qualifications

Element

Score 0 3 6 9

Evidence I am

Looking For

AIQ process doesn't exist or is not

used

33% of quality-critical instruments

have used AIQ

67% of quality critical instruments

have used AIQ

100% AIQ for quality-critical devices

and evidence that this is standard

as part of the start-up process

Element

Score 0 3 6 9

Evidence I am

Looking For

Hard copy of initial qualification

records for 0-33% of quality-critical

instruments

Same as 0, but >33% Same as 3, but >67% 100% of instruments have a hard

copy of the initial qualitifcation

records, and records demonstrate

that the instrument has consistent

performance

1) Explain how new equipment is acquired and integrated into the lab.

2) Do you collect Design Qualification (DQ), Installation Qualification (IQ) and Operation Qualification (OQ). data? How many instruments do you

do this on? Can you show us?

3) Do you collect PQ data? Can you show us?

4) Do you save the initial qualification data on hard copy? Can you show us?

Analytical Instrument Qualification (AIQ) Program

Implementation of Instrument Qualification Process

Instrument Qualification Records

Table 7: SQC dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No documented SQC procedure

exists

SQC procedure exists and contains

- Calibration requirements and

frequencies

- Check Standard requirements and

frequencies


SQC procedure contains

- How Control limits are calculated

- SQC rules in use to define a

measurement as out of control


SQC procedure contains

- Actions to take if a measurement

is out of control

Actions I would expect to see are:

- Investigation of Issue

- Action taken to resolve issue

- Machine taken out of use until

issue is resolved

- All analysis ran since machine was

last within control are rechecked to

determine if results are correct

Element

Score 0 3 6 9

Evidence I am

Looking For

No SQC Records Available SQC Data (Control charts with

anlysis points marked on a chart) is

available in hard copy

SQC data available in electronic

system

- Data entered into electronic

system

- Control charts generated showing

performance


Current and historical SQC data can

be accessed easily within an

electronic system.

SQC data includes control limits

with automatic indication if any

results are out of control according

to SQC rules.

Action log with records for every

incident where the analysis is out

Element

Score 0 3 6 9

Evidence I am

Looking For

Sample analysis is not recorded in

electronic system.

Or

Sample analysis recorded in

electronic system but no

automated valuation of the result

against the specification occurs

Sample analysis recorded in

electronic system.

Results are automatically valuated

against the specification.

Anything outside of the

specification is highlighted as out

of spec.


All instrument SQC charts exist

within statistical software (excel

with appropriate statistical rules

coded into the spreadsheets would

count)

SQC rules and limits defined in

software.

Evidence that action has been

taken when instrument is out of

control e.g. action log


Regular review meetings are taking

place looking at the trends from the

statistical data and actions -

evidence would be review meeting

minutes, action log from meeting

showing changes to process due to

evaluation of data (e.g. column

change frequency increased on

instrument due to higher levels of

failure at end of current life)

Charts exist looking at the

perfomance of the instruments.

Charts could show total

perfromance over time,

performance per analysis over time

etc.

Procedures define prevantative

maintenace requiremnts based on

SQC data

Action log shows instrument taken

down for preventative maintenace

based on SQC data.

Element

Score 0 3 6 9

Evidence I am

Looking For

No measurment Capability studies

carried out and documented

1/3 of all the measurements have a

capability study.

2/3 of all the measurements have a

capability study.

All the measurements have a

capability study.

Measurment Capability Review

plan is in place.

Evidence that Reviews are being

completed. Looking to see that plan

is being acted upon.

1) Can you provide me with a copy of the SQC procedure?

2) Can you provide a list of the analytical equipment and the anlaysis that is run on each piece of equipment?

3) Can you show me where the records of your SQC analysis are kept?

4) Can you show me where the records of your analysis are kept?

5) Can you show me where your measurement capability studies for each analysis are kept?

6) Can you show me your measurement capability review plan?

Measurement Capability

Statistical Quality Control (SQC) Procedure

SQC Records

Data Evaluation

Table 8: SOP dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

Less than 25% of the procedures

meet the following requirements

- Format of the Procedure Template

is matched in procedures reviewed.

- Document control section of the

procedure meets the standards set

out in the document control (MET)

procedure

- Management of change section of

the procedure is completed and

meets standards set out in

management of change procedure

- Procedure is in centralised

location available to all personnel

at every Lab

Between 25% to 50% of the

procedures meet requirements

Between 50% to 75% of the

procedures meet requirements

100% of the procedures meet

requirements

Element

Score 0 3 6 9

Evidence I am

Looking For

Less than 25% of procedures

include Personal Protective

Equipment (PPE), hazards and

sufficient details to identify and

eliminate potential hazards.

Between 25% to 50% of procedures





Between 50% to 75% of procedures





100% of procedures include

Personal Protective Equipment

(PPE), hazards and sufficient details

to identify and eliminate potential

hazards.

Element

Score 0 3 6 9

Evidence I am

Looking For

Less than 25% of procedures have:

1) clearly defined operating steps

sufficient for a non-routine user to

complete the task successfully;

2) supporting information such as

specification of apparatus and

reagents; and,

3) cross reference to any related

documents such as ASTM.







reagents; and,









reagents; and,



All procedures have:






reagents; and,



Element

Score 0 3 6 9

Evidence I am

Looking For

Procedure Use Policy (PUP)

specifies tasks requiring

procedures and documents

requirements for using routine and

non-routine procedures.

All required procedures exist as

defined by PUP. This includes

procedures for sampling,

performing analyses, calibration,

SQC and equipment operation.

Maintenance instructions may be in

the form of SOP's and/or vendor

manuals.

Data collection process verifies use

of non-routine procedures and

usage data is evaluated over time.

Data collection process verifies

usage of non-routine procedures

and verifies that procedures are

executed correctly.

1) Can you provide me with your document control (MET) procedure?

2) Can you provide me with your management of change procedure?

3) Can you provide me with your electronic Template for procedures?

4) Can you provide me with a list of all your procedures?

5) Can you show me where you store your procedures?

6) Who has access to the procedures?

7) Do you have a record of which procedures meet your MET requirements for a procedure?

8) Select 12 procedures and review them comparing to standards required

9) Take 4 procedures and have user run through the task (check that the steps in the procedure match what the user is doing. Also check that there

is enough information in the procedure to complete the task

Procedure Use Policy

Document Control

Hazard Elimination

Procedure Details

Table 9: UPE Management dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No Investigation process or

No documentation in evidence of

formal investigation process

Formal Investigation process

documented in procedure

Procedure must include

- Definition of what is a Quality

failure/incident.

- Triggers/Rules defining when an

investigation should take place.

Formal Log for all quality

failures/incidents exists

Log must include a

- Description of the failure

- Root cause(s) of the issue

identified

- Action Log

Same as 3 +

Quality failures/incident reports

are generated. These should

include

- Description of the failure

- Root cause(s) of the failure

- Description of what the lab has

done to prevent the failure from re-

occurring

Evidence that reports are shared

between Labs

- Shared location for reports to be

viewed

- Section in Regular Lab Meeting

Schedule to review Incident

Reports

Same as 6 +

Regular Management Review

Meeting exist for quality

failures/incidents.

Meeting Minutes show

- Discussion on any trends

(common failures) that exist that

may need a change in the

management system (process &

procedural issues)

- Action Log with evidence that

management system issues are

being addressed

Same Process exists for reporting

and managing quality near misses

as for quality failures/incidents. A

quality near miss is where you have

had a failure or incident that could

have caused a quality issue but

luckily did not e.g. Issue found

with HPLC column. Retesting of

retained samples with new column

found results to have changed but

are still within specification.

Element

Score 0 3 6 9

Evidence I am

Looking For

No CAPA procedure section or

individual procedure exists

CAPA procedure section or

individual procedure exists

Documentation for the

investigation show and identified

root cause for the issue

Same as 3 +

Corrective action log exists with

evidence that actions are being

completed.

Analyse the incidents/failures

looking for the same problem

occurring repeatedly. If root cause

was identified and fixed then you

should not have the same failure

keep repeating.

Same as 6 +

Regular Management Review

Meeting exist for quality

failures/incidents.

Meeting Minutes show

- Discussion on any trends

(common failures) that exist.

- Action Log with evidence that

action is taken if the fixes put in

place are not addressing the issue.

Element

Score 0 3 6 9

Evidence I am

Looking For

Corrective and Preventive Action

(CAPA) procedure not established

and implemented.

CAPA procedure exists to identify

potential sources of non-

conformities and to define

preventive actions.

Evidence shows that preventive

actions focus on systems rather

than single events.

Effectiveness of preventive actions

is evaluated by summarizing CAPA

data and monitoring for

management system failure trends.

Element

Score 0 3 6 9

Evidence I am

Looking For

Protocol not defined &

documented for handling out of

specification situations.

Specific guidelines are documented

for Retesting & Resampling after

obtaining out of specification

results. Guidelines include

documentation requirements.

Nonconforming test results are

routinely investigated and

documented. Calibration

standards, instrument performance

and sample validity are evaluated.

Evidence indicates that all

nonconforming results are

investigated & that corrective

actions are taken to prevent

reoccurrence. Appropriate

techniques are used (i.e.

notification of Quality Coordinator,

reverse traceability, Root Cause

Investigation (RCI), etc).

Element

Score 0 3 6 9

Evidence I am

Looking For

External nonconformance received

related to testing in your lab

>1 Major internal nonconformance

received related to testing in your

lab.

No more than 1 major internal

nonconformance received related

to testing in your lab.

No nonconformances received

related to testing in your lab

1) Do you have a procedure on how you manage quality failures/incidents and can you provide me with the procedure?

2) Does the procedure document what is a classed as a quality failure/incident?

3) Does the procedure define a process that is to be used to investigate quality failures/incidents

4) Does the investigation process have triggers to initiate an investigation and tools to use for the investigation (Root cause investigation)?

5) Do you have any reports (LER's) giving a summary of the issue, the root cause and preventative actions taken to prevent reoccurrence of the

issue?

6) How are theses reports distributed to analytical/quality personnel (allows shared learnings to help prevent similar incident accross all

locations)?

7) Do you hold any management review meetings looking at quality failures/incidents and their investigations? If yes, please provide me with the

minutes from the last 3 meetings?

8) Do you record Quality near Misses? If so, what is the process for collecting this information?

9) Check if there is a Corrective and Preventative Action section in the Manage Quality Failures/Incident procedure. If not, ask to be provided the

CAPA procedure.

10) Can you show me a summary report showing all the incidents/failures for the last year?

11) Pick 3 incidents/failures and ask for the investigation documents recorded during the analysis.

12) Can you provide me with the corrective action log for quality failures/incidents?

13) Can you show documentation around the non-conforming test results and the documentation of investigation?

Control of Nonconforming Test Results

Nonconformances Detected by Inspection

Investigation Process

Corrective Actions

Preventive Actions

Table 10: EH&S dimension questions and audit evidence

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

Either don't measure or Recordable

Rate > 0.40.

Recordable Rate not more than

0.40.


0.30.


0.20.

Element

Score 0 3 6 9

Evidence I am

Looking For

Group does not have or require

personal safety goals.

10-30% of empolyees have


31-60% of employees have


>61% of employees have personal

safety goals.

Element

Score 0 3 6 9

Evidence I am

Looking For

Near misses not recorded Near misses recorded Same as 3+

Goals set for employees regarding

near miss recording

Same as 6+

Data reviewed/analyzed to view

trends and take action.

Evidence: Review meeting,

meeting minutes, trends and

actions

Element

Score 0 3 6 9

Evidence I am

Looking For

LOPC's not identified. LOPC's identified. LOPC's identified and tracked.

Evidence: Tracking tool, updated

recently

LOPC's identified, tracked, and

systems in place to reduce LOPC's.

Evidence: Analysis of LOPC trends

and actions taken and results.

Element

Score 0 3 6 9

Evidence I am

Looking For

No minimum EH&S requirements

exist for the labs.

EH&S lab requirements have been

identified and documented

EH&S lab requirements have been

identified, the lab audited, gaps

identified, and a timeline in place

to close the gaps.

Full compliance with minimum

EH&S lab requirements.

Element

Score 0 3 6 9

Evidence I am

Looking For

Critical Instrument plan not

implemented or critcal instrumnets

not identified.

Critical Instruments identified, but

gaps not identified.

Critical Instruments identified, gaps

identified, and gap closure plans in

place.

Critical Instruments Identified, gaps

identified, and gaps closed with full

compliance, or Critical Instruments

plan is determined to be not

applicable to Lab entity.

Element

Score 0 3 6 9

Evidence I am

Looking For

Life Critical Standards not

implemented.

All Life Critical Standards identified

and Life Critical Standards utilized.

All Life Critical Standards

identified, Life Critical Standards

utilized, and Life Critical Standards

audited with passing results < 90%

of time.

All Life Critical Standards

identified, Life Critical Standards

utilized, and Life Critical Standards

audited with passing results 90% of

time.

1) What is you recordable injury rate? (Generally, a recordable injury or illness under OSHA is one that requires medical treatment beyond first

aid, as well as one that causes death, days away from work, restricted work or transfer to another job, or loss of consciousness. The number of

injuries need to be divided by the total number of work hours performed, and put in the format of [(#injuries)/(200,000 work hours)]

2) Does the group have personal safety goals? What percentage of lab personnel have individual goals that align to safety?

3) Do you record "near-misses" (injuries that have almost occurred, but did not)? Can you show me where they are recorded?

4) Are you analyzing "near-misses" for trends? Can we see an analysis?

5) Do you meet to review "near-misses" regularly? Can we see meeting minutes of the last meeting?

6) Do you identify leaks, breaks, spills, otherwise known as Loss of Primary Containment Incidents(LOPCs)?

7) Do you record LOPCs formally?

8) Do you have a system in place to analyze and reduce the number of LOPCs?

9) Have you defined formal lab EH&S Requirements?

10) Do you assess the labs against these EH&S requirements?

11) Do you track progress against these EH&S requirements?

12) Do you have EH&S Critical Instrumentation? (Fire alarms, leak detection, vapor or fume alarms)?

13) What monitoring do you have in place (inspection, maintenance) for these EH&S Critical Instruments?

14) Are there any gaps? If so, are there plans in place to close the gaps?

15) Do you have Life Critical Standards (how to perform hot work, open lines, do electrical work, perform elevated work, confined space entry)

16) Do you utilize these Life Critical Standards?

17) Do you audit against the Life Critical Standards? If so, how often does the audit pass? (>90% or <90%)

Life Critical Standards

Loss of Primary Containment (LOPC)

Lab EH&S Requirements

EHS Critical Instrumented Systems

Lab Recordable Injury Rate

Personal Safety Goals

Near Miss Program

Table 11: Best Practices dimension questions and audit evidence

Once the audit is complete, the scoring values on these elements are arrayed into “spider plots” for each individual dimension (see below in Figure 1 for the “enablers” spider plot). The averaged scores for each dimension are also arrayed into a spider plot for a rapid determination of gaps as a function of different bulk areas (Figure 2).

Figure 1: Enablers dimension spider plot showing a score of “3” across all elements

Questions

Element

Score 0 3 6 9

Evidence I am

Looking For

No documented job or role

descriptions in place.

Documented job descriptions in

place for >50% of the work group.


place for >75% of the work group.


place for all personnel that define

roles and key job responsibilities.

Job descriptions are reviewed on a

regular basis.

Element

Score 0 3 6 9

Evidence I am

Looking For

Little or no method validation

exists for analyses.

Method validation exists and is

sufficient for 1/3 of the analyses.

Method validation exists and is

sufficient for 2/3 of the analyses.

Existing method validation

sufficient for the analyses. New

methods are validated as per the

EIAR process/procedure

1) Can you show us your method validation procedure? How many methods have gone through this process?

2) Can you show your job roles and descriptions? Do you review these?

Role and Job Descriptions

Method Validation

Figure 2: Dimension spider plot showing a score of “3” across all dimensions

These dimensional audit areas are analyzed by the lab leadership to determine a multi-year and multi-generational plan that will get the lab closer to an idealized final result. The leadership determines what focus areas and specific remedies need to be executed and when. This tool can be used among different labs or re-administered to the same group after a year or more to see if there’s been quantitative progress.

Change Management Overview: It was understood, through conversations with EIAR leadership during the project scoping

process, that “Lab Unification” was a key medium-term goal. Successful integration of

procedures and processes requires skillful change management, with consideration for the

technical and organizational structure of EIAR.

This project was designed to illuminate both technical and non-technical areas of opportunity.

During the process of addressing these opportunities, or said another way “closing the gaps”,

can be very helpful to standardize lab operating procedures and organizational culture.

Organizational change and more specifically organizational culture change is not something

that is achievable rapidly without significant disruption to operations. It will take a concerted

and prioritized effort by lab leadership over many months. This team recommends using the

audit as a tool as a guide to structure the change program that will need to be established.

Open and transparent communication about the lab unification effort, as well as dialogue with

key stakeholders from various labs will be critical for successful implementation. Different labs,

due to the uniqueness of the individual researchers and staff members, may require different

approaches. As with any large organization, sub-groups and sub-cultures surely exist within

EIAR, and the approach for specific labs may need to be modified accordingly. However, it is

recommended that the core of the change program be an understood vision of the future,

common goals, and clear description of the ideal state that is defined by EIAR leadership,

through consultation with key organizational stakeholders. The audit tool is designed to

facilitate the discussion of what a world-class lab management program consists of. It is role of

EIAR leadership, however, to refine these criteria through the vision and mission of EIAR to

communicate a vision for the future that is best understood by EIAR staff. The concept of

engaged employees committed to continuous self-improvement is foundational to the success

of this program.

Establishment of an Interdisciplinary Executive Team:

It is our recommendation that EIAR form an Executive Team comprised of representatives from

various disciplines to consider the following in addition to the audit tool descriptors of ideal lab

operating conditions:

Consider how EIAR’s Vision and Mission can be a constructive frame through which to

prioritize lab improvements.

Implementation timeline: Is this a 5-year project? A 3-year project?

Consider the resources available to the change effort. How much change can the

organization tolerate? How can you continue to remain effective while also improving?

Consider data collection: Data will be collected (the lab audit is one source of data) that

will identify areas of opportunity for EIAR. What other data are necessary? (employee

surveys, discussions, feedback collection)

Summary of Audit Findings: What follows are two separate prioritizations for action. First, priority is measured by element. This element-by-element prioritization provides a detailed view of which elements should receive the highest priority. More often, however, it is useful to consider which dimensions have the highest priority overall. Often, a thoughtful project plan will consider both aspects in order to generate a comprehensive multi-generational plan. These priority orders are the assessment generated by the Dow team, but EIAR may wish to consider how best to prioritize based upon EIAR-specific needs and capabilities. The Audit findings for each element are listed below along with the prioritization score that highlights the elements in order of most benefit for least amount of effort. The Benefit and effort scoring were accomplished using a scoring system of 1, 3 and 9 (1 being Low, 3 being medium and 9 being high benefit and effort). The priority is worked out by the calculation below:

Dimension Element

Element Score

for EIAR Lab

Benefit of Improving to Next Level(s)

Effort to Implement

Priority to Improve

EH&S Goals and Perf. Tracking

0 9 1 9

EH&S Environmental Excellence

0 9 1 9

EH&S Lab EH&S Requirements

0 9 1 9

Calibration Calibration/Verify Instructions

6 9 1 9

Calibration Calibration Records/Reports

0 9 1 9

Preventive Maintenance


0 9 1 9

SQC SQC Records 0 9 1 9

SOP Hazard Elimination 0 9 1 9

Enablers MET Implementation 0 9 3 3

Processes MOC Process 3 9 3 3

Training MET Training 0 9 3 3

SQC SQC Procedure 0 9 3 3


PM Schedule 0 3 1 3

SQC Data Evaluation 0 9 9 1

Enablers Client Satisfaction 0 3 3 1

Calibration Calibration/Check Schedule

6 3 3 1


PM Records 3 3 3 1

Training Qualification Process 3 3 3 1

Training Training Records 3 3 3 1

AIQ Qualification Program

3 3 3 1

Unplanned Events Management

Investigation Process 3 3 3 1


Preventive Actions 3 3 3 1

Processes Service Level Agreements

0 1 1 1

Processes Sample Plan 6 1 1 1


PM Procedures 6 1 1 1

Processes Sample & Data Management

3 3 9 0.333333333

Processes Sample and Work Load

3 3 9 0.333333333


Nonconforming Results

6 3 9 0.333333333

Processes Non-Routine Request Process

3 1 3 0.333333333

Processes Lab Inventory & Capital Plan

6 1 3 0.333333333

Calibration Vendor Calibrations 6 1 3 0.333333333

AIQ Implementation of AIQ Program

0 1 3 0.333333333

AIQ Qualification Records 6 1 3 0.333333333

SQC Measurement Capability

6 1 3 0.333333333


Corrective Actions 6 1 3 0.333333333


Nonconformances by Inspection

3 1 3 0.333333333

Enablers People Success 0 1 9 0.111111111

Calibration Standards and Reagents

6 1 9 0.111111111


Parts Inventory 9 1 9 0.111111111

Training Employee Training Program

9 1 9 0.111111111

SOP Document Control 9 1 9 0.111111111

SOP Procedure Details 9 1 9 0.111111111

SOP Procedure Use 0 1 9 0.111111111

Lab Audit Details

When determining what to work on first it is usually best to complete the tasks that require the least effort with the biggest benefit (A common US/UK expression for these items is “Low hanging fruit”). You should also look at the items that have a high benefit but require more significant levels of effort as these may be very important to for your organization. In some cases these items may need to be started earlier than their priority suggests because they are very important and are only lower due to the time and effort. The low hanging fruit to work on first are (general prioritization by dimension):

Dimension Element Actions Required

EH&S Goals and Perf. Tracking

EH&S Goals should be established

EH&S Environmental Excellence

EIAR should consult with other local agencies to establish environmental plans that are meaningful and achievable.

EH&S Lab EH&S Requirements

Safety: It is the responsibility of EIAR to establish a safety culture. Some priority areas to address include, but are not limited to the following standard safety measures:

Personal Protective Equipment (hand and eye protection)

Operational fume hoods

Safety eye-wash stations and safety showers

Proper storage of laboratory chemicals and reagents

SQC SQC Records

Create a Check Standard(s) for each calibration with known quantities of everything the machine is required to measure. Run Check standard at a set frequency and record data on an excel sheet. Plot this data in graphs in excel. Once sufficient information is gathered then Develop Control Limits (+/- 3 sigma) and plot these on the graph.


SOP Hazard Elimination

Each Lab procedure is to have a Hazards section added with all the potential Hazards associated with the task being carried out in the procedure. Each Lab procedure will outline the Personal Protective Equipment (PPE) required to carry out the task being carried out in the procedure.

Calibration Calibration/Verify Instructions

Instructions added to each calibration procedure on what to do if the calibration fails. These instructions should include:

- Where the failure should be documented and How the failure should be documented

- Whom (May be a role rather than a person e.g. Lab Manager) the failure should be communicated.

- The information that should be communicated - What action to take to confirm that analysis

completed since last calibration is correct. - Under what conditions the instrument should

be taken out of service (e.g. how many attempts at re-calibration)

Calibration Calibration Records/Reports

Calibration record created electronically (could use excel) for all calibrations carried out. Records to include

- Date and Time calibration is carried - Analyst who carried out the calibration - Equipment the calibration was carried out on - Analytical results for calibration are documented - Result of Calibration are documented (e.g. Pass

or Fail)



A maintenance strategy should be established that is achievable and value-oriented.

The Elements that need to be worked on first even though they require more effort are:



Enablers MET Implementation

The Implementation of a standardised work process (MET) across all labs will take quite a long time to fully design and implement. This step is critical to ensuring that you function as one organisation and that you can rely on getting the same result for the analysis no matter which laboratory the sample is tested. First a set of MET requirements/procedures need to be written by the senior leadership on the requirements/work process they want all the labs to follow. This should be a high level document aligning with your strategy in each of these areas. I would expect to see MET requirements/procedures for most of the sections in the Audit

- EHS - Management of Change Process - Sample and Data Management Process

o Sample/plan and None routine analysis o Results recording and analysis

- Calibration and Statistical Quality Control Process

- Preventative Maintenance Process - Instrument Qualification Process - Unplanned Events Management - Standard Operating Procedures - Training Process


SQC SQC Procedure

Create SQC procedure for Lab. Lab procedure must comply with EIAR’s MET requirements for statistical quality control. The procedure should contain

- Frequency of testing for each check standard - Frequency of calibration for each analysis (I

advise starting with a set frequency for calibration and use the analytical data from the check standards over time to determine if the frequency needs to be extended or shortened. Eventually moving to full statistical control to determine when to calibrate the machine)

- Procedure to contain statistical rules determining when the analysis is out of control (you may want to start with a simple rule +/- 3 sigma and add in more advanced rules as you have the ability. Statistical software or more advanced excel sheets with code required to run more advanced rules)

- Actions to take when the analysis is out of control. This should include

o when to re-calibrate o when to take the machine down for

further analysis of the problem o when to remove the machine from

service o The analysis that needs to be re-tested.

As you are unsure when the machine went out of control since the last check, all analysis since that time may be incorrect. Once machine has been fixed then the suspect analysis needs to be re-checked to confirm results are correct.

SQC Data Evaluation

An electronic Laboratory Information Management System is required to record and evaluate results. Below are some open source (free) and proprietary source LIMS systems.

Free and open-source LIMS software

LIMS software

Language base

License Other info Website

LabKey Server

Java Apache

Software License

primarily a web-based data management platform

http://www.labkey.org

MISO Java GPLv3

Web-based information management system for next-generation sequencing (NGS) experiments, developed at The Genome Analysis Centre

www.tgac.ac.uk

Bika LIMS

Python - Zope - Plone (linux

based)

GNU AGPLv3

Web-based Lab Information Laboratory Management System for clinical, water, ambiental, chemical, microbiological and others analytical services.

http://www.bikalabs.com/

Proprietary LIMS software

LIMS System Website

Accelrys LIMS from Accelrys www.accelrys.com

ApolloLIMS from Common Cents Systems, Inc. www.apollolims.com

Biotracker from Ocimum Bio Solutions www.ocimumbio.com

Biotracker Lite from Ocimum Bio Solutions www.ocimumbio.com

CaliberLIMS from Caliber Technologies Pvt. Ltd. www.caliberindia.com

CCLAS from Ventyx, an ABB company, formerly Mincom (company)

www.ventyx.com

Clarity LIMS from GenoLogics Life Sciences www.genologics.com

CloudLIMS from CloudLIMS www.cloudlims.com

Darwin from Thermo Fisher Scientific www.thermofisher.com

ELab from LabLynx www.lablynx.com

Element LIMS from Promium www.promium.com

Exemplar Biomarker Discovery from Sapio Sciences www.sapiosciences.com

Exemplar Dx LIMS from Sapio Sciences www.sapiosciences.com

Exemplar Research LIMS from Sapio Sciences www.sapiosciences.com

Galileo from Thermo Fisher Scientific www.thermofisher.com

LABbase from Analytik Jena www.analytik-jena.com

LABVANTAGE from LABVANTAGE Solutions www.labvantage.com

Labware from LabWare www.labware.com/en

LABWORKS from PerkinElmer www.perkinelmer.com

Labway-LIMS from Ambidata Digital Innovation Solutions & Consulting

www.ambidata.pt

http://www.labkey.org/

http://www.tgac.ac.uk/

http://www.bikalabs.com/

http://www.accelrys.com/

http://www.apollolims.com/

http://www.ocimumbio.com/

http://www.ocimumbio.com/

http://www.caliberindia.com/

http://www.ventyx.com/

http://www.genologics.com/

http://www.cloudlims.com/

http://www.thermofisher.com/

http://www.lablynx.com/

http://www.promium.com/

http://www.sapiosciences.com/




http://www.analytik-jena.com/

http://www.labvantage.com/

http://www.labware.com/en

http://www.perkinelmer.com/

http://www.ambidata.pt/

LDMS from Frontier Science and Technology Research Foundation

www.fstrf.org

Matrix Gemini from Autoscribe www.autoscribeinformatics.com

Nautilus from Thermo Fisher Scientific www.thermofisher.com

ProlabQ from Open-Co

www.openco.it/index.php?option=com_k2&view=item&id=9:prolabq-lims&Itemid=39&lang=en

readyLIMS from Analytik Jena www.analytik-jena.com

Result Point from Accelerated Technology Laboratories, Inc www.atlab.com

SampleManager from Thermo Fisher Scientific www.thermofisher.com

Sample Master from Accelerated Technology Laboratories, Inc www.atlab.com

SIMATIC IT Unilab from Siemens www.Siemens.com

STARLIMS from STARLIMS Corporation www.starlims.com

TITAN from Accelerated Technology Laboratories, Inc www.atlab.com

Watson from Thermo Fisher Scientific www.thermofisher.com

webLIMS from LabLynx www.lablynx.com

WinLIMS from QSI Corporation N www.qsius.com

NuGenesis from Waters Corporation www.Waters.com

Secondary Elements to work on


Processes MOC Process

Develop management of change process and procedure. This procedure will outline the different triggering criteria that would require the initiation of MOC as well as standardize what type of communication needs to occur within the organization. For example, changing a procedure requires lab leadership approval and distribution to all the lab personnel.

Training MET Training Train all personnel in the MET requirements/procedures

Enablers MET Implementation

Each Lab to review its procedures to ensure that they all comply with the MET requirements/procedures. Individual Lab procedures updated to conform to the new MET requirements/procedures.

http://www.fstrf.org/

http://www.autoscribeinformatics.com/

http://www.autoscribeinformatics.com/


http://www.openco.it/index.php?option=com_k2&view=item&id=9:prolabq-lims&Itemid=39&lang=en




http://www.analytik-jena.com/

http://www.atlab.com/



http://www.siemens.com/

http://www.starlims.com/



http://www.lablynx.com/

http://www.qsius.com/

http://www.waters.com/



PM Schedule

Create preventative maintenance schedule for all instruments. A place to start is manufacturer’s recommendations as a function of use. Secondly, the lab can use the SQC data that will ideally be acquired from the recommendations above. These data will provide information about how often the equipment needs to be preventively maintained in order to provide the required performance. These data can be leveraged among multiple labs.

SQC SQC Records

Use statistical software or advanced excel sheets to record your SQC data having the SQC rules built in so that if the result is outside of your rules then it is easily identified when the result is entered. Create Action log for each control chart. This action log should include

- Date and Time of Analysis - Analyst who carried out the analysis - Issue with analysis e.g. control outside of upper

control limit. - Action taken e.g. equipment re-calibrated and

checked. Samples since last check re-analysed and all results are correct.

Prioritization and Multi-Generational Plan

The Audit Tool is designed to highlight gaps, but not all gaps can be addressed simultaneously, nor should they be addressed with the same level of emphasis. It is therefore critical to consider both the priority management gives to certain sub-elements, as well as when the issues will be addressed. These two elements should be managed by the executive leadership team, and key concepts that were reviewed during the project are described below. For system improvement, there is no inherently right or wrong answer for what to work on first. The lab audit results may show some obvious high-priority areas to improve, but there may be a large “middle ground” of neither high nor low priority items that need to be addressed. In this case, EIAR needs to consider the benefits that would be gained from each item and the resources needed to pursue them. Step 1: Develop List of “Improvement Projects”: The audit findings will highlight the gaps to the “ideal state”, but projects will be necessary to address these gaps. These projects will have different benefits, and will be viewed by different stakeholders differently. As such, stakeholder management is a critical step to successful change. Step 2: Determine Benefit by Stakeholder: Different stakeholders in EIAR will have different priorities and will be benefited by change differently. The executive team should be the clearinghouse for high-level strategic concepts and set the direction, taking into account EIAR’s larger goals. For agreement on path forward, it may be helpful to consider benefits by stakeholder group or by element of EIAR’s mission/vision. One way to generate an organization-wide benefit ranking is to use a Pugh Matrix, with 1, 3, and 9 ranking scale. Each stakeholder should determine the benefits they or their group would receive. The SUM of the benefits should be then used to prioritize. Example Pugh Matrix:

Stake-holder 1 Stake-holder 2 Stake-holder 3 SUM

Project 1 1 3 3 7 (lowest priority)

Project 2 1 9 3 13

Project 3 3 1 9 13

Project 4

3 9 9 21 (highest priority)

Project 5 9 1 1 11

Step 3: Determine “must-do” improvements that will proceed regardless of effort required.

Health and Safety projects fall into this category.

Step 4: Plot Improvement Projects as follows:

Multi-Generation Plan:

Multi-Generation Plan (MGP): The improvements, once plotted, can be used to develop a Multi-Generation Plan (MGP). The MGP is a guide for what to work on first, next and last. It should be re-evaluated periodically as you progress, to ensure that priorities have not shifted and that resources needed for projects are re-evaluated. Logically, projects that yield the highest benefit and the lowest required effort (denoted in the shaded area 1, above) should receive attention first. Also important to note are projects that yield very high benefit, but may take a considerable amount of time. These projects may need to begin earlier (receive higher priority) than the MGP might suggest due to the high magnitude of the benefit, or the strategic direction of the Executive Team.

Key concepts for the Interdisciplinary Executive team to consider:

Link the changes to the strategic plan. Ensure that you can explain why you are initiating

the change, and be ready to talk about it.

Determine success criteria: What does success look like?

Allocate resources effectively. Change requires effort above and beyond core job

responsibilities.

Develop training on the change, and allow people the time to ask questions and learn.

Communicate to the organization the strategy and the changes that support it.

Buy-in is critical. One way to increase buy-in is to have the change designed by

empowered sub-teams comprised of those impacted by the change. For instance: a

change that will affect all labs should not be comprised of team members from only 1 or 2

labs.

Give sub-teams a goal (the improvement), the definition of success, resources, and a

timeline. Make the team responsible for refining the improvement and implementing it.

Guide the sub-teams on how to communicate, which stakeholders to involve, and how

often to report back.

Develop a rewards structure for success. Recognize the teams and their results

appropriately.

Be patient – change takes time!

Executing and Implementing Change – Making it “stick”:

Once the improvement has been defined, and is ready for implementation, communication and

training become critical to success. Everyone who is impacted by a change should be informed

of it, and given the opportunity to ask questions or state their opinion, even if they do not have

decision-making authority. Structural changes may be necessary (procedure change,

equipment change) and should be well documented and completely thought through. Follow-

up should occur periodically to ensure that the change has occurred (another audit, for

instance) and improvement should be measured and celebrated.

As stated initially, this should be an ongoing effort, and one which should not have a definitive

end-date. All organizations can improve, and the excellent organization is constantly looking

for ways to improve itself. Evaluate the improvements. What has gone well? What has not

gone well? What has the executive team learned? What has the organization learned?

More Resources:

There is considerable management literature on managing change, improving organizations and

creating learning organizations. This is intended to be a high-level framework for change

management, but is not comprehensive. The executive team may find specific challenges that

are not addressed here, and may find solutions in management journals and books on the

subject. A book that may provide helpful approaches and was discussed during our in-person

visit is:

Stavros, J. M., & Hinrichs, G. (2011). The Thin Book Of® SOAR: Building Strengths-

Based Strategy. Thin Book Publishing.