Critical Reading of Medical Articles

Critical Reading of Medical Articles

Introduction to Primary Care

a course of the Center of Post Graduate Studies in FM

PO Box 27121 – Riyadh 11417Tel: 4912326 – Fax: 4970847

Aim: knowledge on the principles of critical reading (appraisal) of

medical articles.

Objectives: at the end of this presentation, the participants should be able;

• to assess the quality of the article to determine if the conclusions are valid

• to explain how to start reading

• to explain the results of an article and their relevance to your practice.

What is critical reading?

• The assessment of evidence by systematically reviewing its relevance, validity and results to specific situations.

- Chambers, R. (1998).

It is an essential skill for evidence-based medicine.

Why read medical articles critically?

1. Convert our information needs into answerable questions.

2. Track down, with maximum efficiency, the best evidence with which to answer these questions.

3. Appraise the evidence critically, assess its validity and usefulness (clinical applicability).

4.To implement the results of this appraisal into our clinical practice.

5.To evaluate our performance.

Published research is not always reliable orrelevant.

Difficulties with Critical Appraisal

• Can be time consuming initially

• Doesn’t provide an “easy” answer

• It could show a lack of good evidence in a particular topic

Key Steps to Effective Critical Appraisal

1. Are the results valid?

2. What are the results?

3. How will these results be relevant to

the patient?

Basic elements of clinical decision making

Steps to EBM

1. Formulate a clear, focused clinical question 2. PICO model3. Search the literature for the best external evidence4. Critically appraise the evidence for its validity and usefulness5. Implement the useful evidence in clinical practice6. Evaluate the results

Different types of question requiredifferent study designs

Atiology:what caused this illness?

Diagnosis:what does this test result mean?

Prognosis: what is likely to happen to this patient?

Harm:is having been exposed to this substance likely to do harm, and, if so, what?

Effectivenessis this treatment likely to help patient with this illness?

Qualitative:what are the outcomes that are most important to patients with condition?

Study designs for Question Types

DiagnosisProspective cohort study with good quality validation against “Gold Standard”

TherapyRandomized controlled clinical trial (RCT)

Etiology/

Harm

RCT, cohort or case-control study (probably retrospective)

PrognosisProspective cohort study

PICO Structure

Patients P

Intervention I

Comparison C

Outcome O

Clinical Scenario

What therapeutic agents can be used for rate control of atrial fibrillation (AF) in a patient with congestive heart failure (CHF)?

Starting Point

Break the question into the following components:

Population: Patients with atrial fibrillation AND congestive heart failure

Intervention: Rate control

Comparison: N/A

Outcome: Mortality, effectiveness of rate control

MeSH

• Medline Subject Headings (MeSH terms) are assigned to each article.

• Provide consistent way to retrieve articles• MeSH “a controlled vocabulary organised

in a tree structure”• index = like chapter headings in a text

book– Major Subject Headings (eg Leg)

• Subject Headings (eg knee)– Subheadings (eg cruciate ligament)

How can we do Critical Appraisal?

• Use common sense

• Use simple checklists

• Use different checklists depending on the different types of studies (i.e., RCTS, systematic reviews etc)

• Checklists help you focus on the important parts of the article

• Who were the participants of the study?

• How were they recruited?

• Was there bias in the recruiting methods?

• How was the data collected?

• What statistical tests were used?

• Where the data collection methods accurate?

Research Methodology

Critical appraisal questions

• What is the paper about?

• Why was the study done?

• What type of study was done?

• Was it primary research (experiment, RCT, cohort, case-control, cross-sectional, longitudinal, case report/series)?

Critical appraisal questions

• Was it secondary research (overview, systematic review, meta-analysis, decision analysis, guidelines development, economic analysis)?

• Was the design appropriate (for study on treatment, diagnosis, screening, prognosis, or causation)?

• Was the study ethical?

What makes studies reliable?

• First look for biases in the study.

• Bias can be defined as ‘the systematic

deviation of the results of a study from the

truth because of the way it has been

conducted, analysed or reported’.

Key sources of bias in clinical trials

Selection biasBiased allocation to comparison groups

Performance biasUnequal provision of care apart from treatment under evaluation

Detection biasBiased assessment of outcome

Attrition biasBiased occurrence and handling of deviations from protocol and loss to follow up

• A study which is sufficiently free from bias is said to have internal validity

• Different methods of collecting evidence are graded as to their relative levels of validity.

• To minimise bias, double-blind randomised controlled trials are the best for evidence.

A system for reading an original article

• First, “scan” the article: -A brief review of the length, headings, key and

concluding sentences of each section is worthwhile.

-How much time it will take to read the article carefully

-Provoke questions or the need for clarification which will direct your focus during a second detailed reading.

In general, original articles will comprise:

• A title

• An abstract or summary

• A methods (and materials) section

• Results

• Discussion

• References

A title

• A title will normally contain three elements :

-setting (types of patients and circumstances under which the study is performed)

- intervention (in prospective randomised controlled trials) or study factor

- outcome measure - E.g. Effect of carbon dioxide neumoperitoneum

[ intervention] on development of atlectasis- [ outcome] during anaesthesia [setting].

A)A) AIMAIM Research Question? Well defined? Worthwhile? Research Question? Well defined? Worthwhile? (Intro)(Intro) Appropriate? Tested properly by method?Appropriate? Tested properly by method?

B)B) BACKGROUND INFORMATIONBACKGROUND INFORMATION (Intro) (Intro)Author, Journal, Type of Study.Author, Journal, Type of Study.

Strengths/weaknessesStrengths/weaknesses

C)C) SETTINGSETTING Primary/secondary care? Type of practice? Primary/secondary care? Type of practice?

(Method)(Method) Strengths/weaknessesStrengths/weaknesses

C) SAMPLEC) SAMPLE Large enough? Comparable with normal Large enough? Comparable with normal (Method) (Method) population? Randomly selected? Biased? population? Randomly selected? Biased?

Inclusion/exclusion criteria?Inclusion/exclusion criteria?

C)C) CONTROLSCONTROLS Used or not? How were they selected? Used or not? How were they selected?

(Method)(Method) Matched for all relevant factors?Matched for all relevant factors?

Any confounding factors?Any confounding factors?

D)D) DESIGNDESIGN Appropriate? Retrospective or prospective? Appropriate? Retrospective or prospective? (Method) (Method) Quantitative/qualitative? Quantitative/qualitative? Observational/ Observational/ Experimental? Double Experimental? Double blind? Placebo controlled?blind? Placebo controlled?

D)D) DATADATA Were questionnaires piloted? Response Were questionnaires piloted? Response rate? (Method/results)rate? (Method/results) Outcome measures defined?Was Outcome measures defined?Was

measurement measurement validated/reliable? validated/reliable? Understandable clear data represented Understandable clear data represented

accurately? Are accurately? Are statistics appropriate?statistics appropriate?

E)E) ETHICALETHICAL Local Ethics Committee Approval? Local Ethics Committee Approval?

Pt consent? Pt consent?

E)E) EVALUATIONEVALUATION Summary of findings. Study strength and Summary of findings. Study strength and (Discussion)(Discussion) limitations. Comparison with existing limitations. Comparison with existing

literature. Answer research literature. Answer research question?question?

E)E) EFFECTIVEEFFECTIVE Are conclusions made in discussion Are conclusions made in discussion justified (Discussion)justified (Discussion) by results? Clinically by results? Clinically significant? Clinically significant? Clinically relevant? Clinically relevant? Clinically applicable?applicable?

StatsStats!..…!..…

• Means to compare samples of populationsMeans to compare samples of populations

• P values P values : : The probability that any particular out The probability that any particular out come would have arisen by chancecome would have arisen by chance

p < 0.05 = statistically significantp < 0.05 = statistically significant

• Confidence Intervals Confidence Intervals : Estimated range that is likely to : Estimated range that is likely to include the true valueinclude the true value

• Numbers Needed to Treat Numbers Needed to Treat : :

Estimated number of patients that need to be treated with a new Estimated number of patients that need to be treated with a new treatment for one additional patient to benefittreatment for one additional patient to benefit

Relative R

isk

Absolute

Risk D

iffere

nce

Absolute Benefit Increase

Numbers Needed to Treat (NNT)

Confidence Intervals

Relative Risk Reduction

P Value

Absolute Risk Reduction

Likelihood Ratio

Predictive Value of tests

Intention to treat

Sensitivity

Specificity

Odds Ratio

READER acronym to aid critical readingREADER acronym to aid critical readingMacAuley. BJGP 1994; 44,83-85MacAuley. BJGP 1994; 44,83-85

• Relevance Relevance - reader’s context- reader’s context

• Education Education - behaviour modification - behaviour modification

• ApplicabilityApplicability - readers own practice- readers own practice

• DiscriminationDiscrimination - validity, quality - validity, quality

• EvaluationEvaluation - scoring system- scoring system

• ReactionReaction - what to do with the paper..! - what to do with the paper..!