A Roadmap towards comparable, standards based and reliable Certification of EHRs Dr. Jos Devlies, EuroRec, Belgium
Dec 23, 2015
A Roadmap towards comparable, standards based and reliable Certification of EHRs
Dr. Jos Devlies, EuroRec, Belgium
WoHIT 2012 - Copenhagen - EuroRec Session
2
IssuesWhy certification?
Software Quality Standards
Main tracks for quality labelling and certification
Quality Labelling and Certification Standards
State of the Art in Europe
Recommendations
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
3
Thematic
Network 2009-2012
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
4
The Quality Challenge
eHealth and more specifically Electronic Health Record systems have an enormous potential to improve quality, accessibility and efficiency of care, provided they are:
fit for purpose
reliable, trustworthy and of sufficient quality;
sharable and interoperable;
used appropriately.
Patients are too important to just suppose that EHR systems and other eHealth applications are trustworthy.
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
5
The Quality IssueMyers et al*. show that adverse events related to the use of EHR systems are mainly resulting from:• missing or incorrect data;
• data displayed for the wrong patient;
• chaos during system downtime;
• system unavailable for use.
“Bad Health Informatics can kill”: an overview of reported incidents in healthcare where a health information system was the cause or a significant factor: http://iig.umit.at/efmi/badinformatics.htm
*Myers DB, Jones SL, Sittig DF, Review of Reported Clinical Information System Adverse Events in US Food and Drug Administration Databases, Applied Clinical Informatics 2011; 2: 63–74
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
6
Software Quality Standards
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
7
Are there quality standards for the EHR ?
What kind of standards do we mean when addressing EHR Quality in Quality Labelling?
• Standards on data modelling.
• Standards on programming methodology.
• Standards about documenting the EHR application.
• Content related standards..
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
8
Software Quality ISO 25010
“The quality of a system is the degree to which a system satisfies the stated and implied needs of its various stakeholders, and thus provide value”.
Distinguishes:
Quality in use
Product quality
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
9
Quality in use
Effectiveness
Effectiveness
Efficiency
Efficiency
Satisfaction
Usefulness
Trust
Pleasure
Comfort
Freedom from risk
Economic risk mitigation
Health and safety risk mitigation
Environmental risk mitigation
Context coverage
Context completeness
Flexibility
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
10
System/Software Product Quality (1/2)
Functional Suitability
Functional completeness
Functional correctness
Functional appropriateness
Performance efficiency
Time behaviour
Resource utilisation
Capacity
Compatibility
Co-existence
Interoperability
Usability
Appropriateness recognisability
Learnability
Operability
User error protection
User interface aesthetics
Accessibility
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
11
System/Software Product Quality (2/2)
Reliability
Maturity
Availability
Fault tolerance
Recoverability
Security
Confidentiality
Integrity
Non-repudiation
Accountability
Authenticity
Maintainability
Modularity
Reusability
Analysability
Modifiability
Testability
Portability
Adaptability
Installability
Replaceability
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
12
Focus of Quality Labelling & Certification
Functional Suitability
Functional completeness
Functional correctness
Functional appropriateness
Performance efficiency
Time behaviour
Resource utilisation
Capacity
Compatibility
Co-existence
Interoperability
Usability
Appropriateness recognisability
Learnability
Operability
User error protection
User interface aesthetics
Accessibility
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
13
Quality Labelling and Certification
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
14
How to assess quality?
Quality labelling and certification through professional third party assessment offers best chances for a comparable and reliable quality documentation of those systems.
Addressing different domains:Functionality (incl. ability to produce ‘messages’)
Data exchange (part of interoperability)
Administrative and Billing aspects
Use validation and Measurements
Software Product as such
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
15
Tracks, Methods & Approaches
Two Tracks : “Authority driven” versus “Market Driven”
Public initiative / Supplier initiative
For the market driven approach: by an independent organisation or by an industrial organisation
Two Methods: third party assessment versus self-assessment
Two main Approaches: system “Functionality” versus “Interoperability” testing
National or even Regional Certification versus Cross-Border Quality Labelling
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
16
Procedures and kind of attestation
Certification procedure Attestation granted
Third party assessment by a CAB being a public authority or an organisation granted power by a public authority either by law or by regulation.
Certificate
Third party assessment by a CAB on requirements issued by an organisation not empowered by law or by regulation.
Quality label
Self-assessment with an external audit. Conformity assessment is done by the supplier and documented to a third party, being a public entity, a professional organisation or an industry federation.
No “attestation” but a Quality Mark on the
product is allowed
Self-assessment by vendor who performed testing on his own products and affirms that they conform to a given set of requirements.
Declaration of quality
most
su
itab
le
pro
ced
ure
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
17
The Quality Assessment Process
Certification body
CAB
InitiatorIAF
Product
Supplier Authority
Accreditation body
Certificate
Label
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
18
5 Good Reasons for National EHR Certification
• Assure compliance to national rules and standards.
• Increase quality of the products through coherent and pre-tested functionality.
• Leverage exchange of health (care) related data and interoperability of systems.
• Improve patient safety in care.
• Have a reliable data source for secondary use.
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
19
Cross-border Quality Labelling?
There is no “authority driven” cross-border certification.
The three “private” initiatives are indeed border-independent:
EuroRec: independent, focus on EHR systems (functional and exchange as function)
I.H.E.: industry driven, focus on testing the exchange and the technical interoperability
Continua Health Alliance: industry driven, focus on devices content portability
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
20
EHR Quality Labelling and Certification in Europe
(based on EHR-QTN projects)
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
21
“National” Clinical Certification•Existing “national” certification
• Foreseen within 1-2 years
• Considered
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
22
Quality Labelling and Certification
Standards
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
23
The Quality Assessment Process
Certification body
CAB
InitiatorIAF
Product
Supplier Authority
Accreditation body
Certificate
Label
ISO/IEC 17020 =>
<= ISO/IEC 17011
ISO/IEC 17025 =>
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
24
Applicable standards
Stakeholder Applicable ISO/IEC StandardAccreditation Body ISO/IEC 17011:2004
ISO 9000:2000ISO/IEC 17000:2004VIM:1993
Certification Body ISO/IEC Guide 65Conformity Assessment Body ISO/IEC 17025:2005
[1] General requirements for accreditation bodies accrediting conformity assessment bodies (ISO/IEC 17011:2004)- Quality management systems – Fundamentals and vocabulary (9000:2000)[2] Conformity assessment – Vocabulary and general principles (ISO/IEC 17000:200)[3] International vocabulary of basic and general terms in metrology (VIM:1993)[4] General requirements for bodies operating product certification systems (ISO/IEC Guide 65)[5] General requirements for the competence of testing and calibration laboratories (ISO/IEC 17025:2005)
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
25
Main standard requirements
•Discretion and Confidentiality
•Impartiality
•Openness
•Distinct roles involved organisations
Independence
•Initial Documentation
•Rules of Evaluation
•Testing Documentation
Documentation of the process
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
26
Main standard requirements
•Involvement of all stakeholders•Distinguish generic and domain specific•Consider national / regional variants
Content to be validated /
tested
•Precise unambiguously the version of the SW•Limit the validity to intended user group(s)•Limit validity to region or country (if applic.)
Limitations of Certificate or
Label
•Pay attention to effective use to realise full added value
Effective Use
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
27
Recommendationsby the EHR-QTN Consortium
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
28
Prerequisite
“ If quality labelling and certification of EHR systems is to become generalised, then it needs endorsement at the highest competent levels e.g. by the EU Commission, the responsible Member States Ministries, the Healthcare Providers Organisations and the specialised industry.”
See the Belgrade Declaration on www.eurorec.org
8 May 2012
29
Recommendations (summarised)
1. Create and harmonise the legal and regulatory framework stimulating national and regional authorities to enforce the use of quality labelled and certified applications.
2. Certification bodies should be accredited and compliant to international standards, more precisely ISO 17020.
3. Strengthen the European scale pioneering initiatives (EuroRec / I.H.E.) in order to keep certification on the agenda and invest in maintenance and expansion of the actual descriptive statement and profiles.
4. Address the issue of personnel shortage in health informatics in general and more specifically in health informatics quality assessment.
8 May 2012
WoHIT 2012 - Copenhagen - EuroRec Session
30
Recommendations (summarised)
5. Third party assessment is the most suitable procedure for quality labelling and certification in the still immature market of the EHR systems.
6. The incentivised model seems the most promising, surely for self-employed healthcare professionals.
7. Promote equivalence of certificates across Europe by validating at European level both the functional descriptive statements of EuroRec and the IHE profiles.
8. Consider the possibility to create a cross-border “Register of Quality Labelled or Certified Clinical Software”, offering information about the products and documentation about the certification process.
8 May 2012WoHIT 2012 - Copenhagen - EuroRec Session