MAST: a model for HTA-based assessment of telemedicine applications
ANNA KOTZEVA, on behalf of the MAST Working groupCatalan Agency for Health Information, Assessment and Quality, Spain
HTAi Annual meeting, June 2012, Bilbao
Contents
1. MAST as a multidisciplinary assessment model Background Development process Aim and structure of the model
2. Empirical test of MAST in the RENEWING HEALTH project Project overview Experiences and results so far
3. Conclusions
Background
Telemedicine is considered as a possible solution to thechallenges of healthcare systems
Critique of existing evidence for telemedicineHailey et al. 2002: Lack of clinical outcomesWhitten et al. 2002: Economic studies do not meet
standardsHersh et al. 2006: Few studies are well designed
Lack of high quality evidence on the effectiveness oftelemedicine as a main barrier for its wider implementation
MethoTelemed project (2009)Aiming to provide a structured framework for assessing the
effectiveness and contribution to quality of care of telemedicine applications
Based on users’ and stakeholders’ needs
Background
Promotion of assessment of telemedicine-based services
Development of MAST
Comprehensive process
Scientific evidence
• Systematic literature review
Stakeholder & users’ opinion
• 2 Workshops (20+18 participants)
Overview of other models
• EUnetHTA Core Model• Donabedian’s model for
quality of care• Med Res Council on
Complex interventions• Wootton et al.• Ohinmaa et al.• ......
MAST (Manual & Toolkit)
MAST can be used when the assessment aims to describe effectiveness and contribution of telemedicine to quality of care and to produce basis for decision making
Assessment defined as:
A multidisciplinary process that summarizes and evaluates information about the clinical, economic, organizational and socio-ethical issues related to the use of telemedicine, in a systematic, unbiased and robust manner.
Aim
*based on EUnetHTA Core Model
How does MAST ensure a multidisciplinary
assessment?
The framework suggests 3 stages of assessment:
I. Preceding considerations
II. Multidisciplinary Assessment
III. Transferability of results
Structure and elements of MAST
This stage is about “setting the context”
Various aspects have to be taken into consideration: Maturity of the technology Selection of appropriate comparator/s Level on which the assessment should be carried out – single
hospital, region, state Number of patients included in the assessment Existing legislation and reimbursement policy for telemedicine-
based services
I. Preceding considerations
WHAT should be assessed?
1. Health problem and characteristics of the application
2. Clinical effectiveness 3. Safety 4. Patient perspectives5. Economic aspects 6. Organisational aspects
7. Socio-cultural, ethical and legal aspects
II. Multidisciplinary assessment
Descriptive
Descriptive
Evaluation by outcome(systematic review OR empirical study)
DO
MA
INS
HOW it should be assessed?
Study design - Aim for highest possible level of evidence
Outcome measures should be: • patient-important outcomes• based on scientific literature to enable comparison• validated instruments
II. Multidisciplinary assessment
Answers the question:
Can results be generalized to other settings?
This consideration has to be made within each of the domains!
Examples of limitations for direct transferability of results: Country-specific reimbursement conditions Healthcare price/hour Values and culture influence user preferences and hence, satisfaction
with the new service
III. Transferability assessment
?
Empirical test of MAST in the RENEWING HEALTH project
Project overview
Experiences and results so far Tools to support study design, reporting &
analysis
Training
Close collaboration and feedback from users
Empirical test of MAST in the RENEWING HEALTH project
Project overview
Experiences and results so far Tools to support study design, reporting &
analysis
Training
Close collaboration and feedback from users
Funded by the European CommunityICT Policy Support Program, Competitiveness and Innovation Framework Program
So far, the largest pragmatic RCT in the area of telemedicine
7158 patients
84 centers
21 pilots
10 clusters
3 diseases
1 observ.
9 RCTs
DMCOPDCVD
9 European regions
General objective:
To produce evidence and decision support for the EU health policies and the regional authorities regarding the future deployment of telemedicine services in those fields where these can lead to improved care and reduced cost.
RENEWING HEALTH Project is not envisioned to produce scientific evidence on clinical outcomes of telehealth, BUT
a broader knowledge on benefits of telemedicine services through a number of multidisciplinary outcomes
Empirical test of MAST in the RENEWING HEALTH project
Project overview
Experiences and results so far Tools to support study design, reporting &
analysis
Training
Close collaboration and feedback from users
Application of MAST in practice stimulates its further development and improvement
A number of tools were developed to support study design, data management, analysis
and reporting of results
1• Clinical study protocols
2• Minimum dataset of common outcomes
3• Clinical database and guidance on coding
and monitoring
4• Guide for analysis and reporting of results
Ensures quality and comparability of data
Open seminars Berlin, May 2010
Introduction to the framework, 80 participantswww.mast-model.info
Treviso, Feb 2012
Guidance on data analysis and reporting of results, 60 participants
Training
Dissemination Kidholm et al. A model for assessment of telemedicine applications -
MAST. Int J Tech Ass Health Care, 28:1, Jan 2012 Communications Website forthcoming!!!
Collaboration and users feedback
European projects
RENEWING HEALTH http://www.renewinghealth.eu/ (21 trials)
http://www.incasa-project.eu/news.php (5 trials)
INTEGRATED HOME CARE http://www.integratedhomecare.eu/
Recommended by:British Thoracic Society (statement on respiratory care)National Danish Strategy for Telemedicine
Collaboration and users feedback
European regions adopting MAST as a general framework for telemedicine assessment Norbotten (Sweden) Veneto (Italy) Basque country (Spain)
Development of a French versionMEETIC: Modèle pour l'Evaluation (Economique) de la Télémédecine Recherche Clinique Santé Publique, Paris
In conclusion…
Advantages of MAST Multidisciplinary and very comprehensive Based on scientific evidence, on criteria for quality and on
stakeholders’ needs Establishes a common standard for all stages of study development
(design, data collection, analysis and reporting)
Limitations of MAST Time consuming Only relevant in assessment of mature telemedicine applications Doesn’t give indications for prioritization when resources are limited More operational criteria are needed
MAST 2ON THE WAY TO
Empirical test of MAST is ongoing (>25 trials use it currently)
Based on the results a revision will be made
So far MAST has been a useful and comprehensive framework for study design, data collection and assessment, and ensures comparability of results for telemedicine studies in different cultural contexts
Based on the validation we hope MAST will be established as the first widely accepted methodology for the evaluation of complex telemedicine interventions
In conclusion…