V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y Evidence Based Medicine: The Scientific Method and the Role Statistics Plays Al M Best, PhD Affiliate.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Evidence Based Medicine:Evidence Based Medicine:

The Scientific Method and The Scientific Method and the Role Statistics Playsthe Role Statistics Plays

Al M Best, PhDAl M Best, PhDAffiliate Professor of Biostatistics, School of Affiliate Professor of Biostatistics, School of

MedicineMedicine

Director of Faculty Research Development, Director of Faculty Research Development, School of DentistrySchool of Dentistry

School of DentistrySchool of Dentistry

[email protected]@VCU.edu

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

CODA CODA →→ EBD EBD →→ CAS CAS

CODACODA– 2-9 Competent to apply 2-9 Competent to apply critical thinking critical thinking and and

problem-solving skills in the comprehensive care of problem-solving skills in the comprehensive care of patients, scientific inquiry and research methodologypatients, scientific inquiry and research methodology

– 2-21 Competent to access, 2-21 Competent to access, critically appraisecritically appraise, apply, , apply, and communicate scientific and lay literature as it and communicate scientific and lay literature as it relates to providing evidence-based patient care relates to providing evidence-based patient care

Critical Appraisal SkillsCritical Appraisal Skills– Are the results of the study valid?Are the results of the study valid?– What are the results?What are the results?– Will the results help locally?Will the results help locally?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

OutlineOutline

Science: Using data to answer questionsScience: Using data to answer questions Example studyExample study

– Estimate prevalence in a populationEstimate prevalence in a population Sampling, measurement, randomnessSampling, measurement, randomness EstimationEstimation

– Compare two groupsCompare two groups Hypothesis testingHypothesis testing P-valueP-value

– Interpret the resultsInterpret the results Different outcomes?Different outcomes?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

ScienceScience

““a systematic enterprise a systematic enterprise that builds and organizes that builds and organizes knowledge in the form of knowledge in the form of testable explanations testable explanations and predictions about and predictions about the universe.”the universe.”

Wilson, Edward O. (1998). Wilson, Edward O. (1998). Consilience: The Unity of Knowledge. Consilience: The Unity of Knowledge. New York, NY: Vintage Books. pp. 49–New York, NY: Vintage Books. pp. 49–71. 71. ISBN 0-679-45077-7..

© Matt Groening, Futurama

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

A Scientist’s QuandaryA Scientist’s Quandary

Are the results of the study Are the results of the study validvalid??– Most experiments are highly local but have Most experiments are highly local but have

general aspirations.general aspirations.– How can findings How can findings generalizegeneralize to other people, to other people,

in other settings, with comparable in other settings, with comparable interventions, and other outcomes. interventions, and other outcomes.

How will you assess whether the paper’s findings How will you assess whether the paper’s findings will generalize to your situation? will generalize to your situation? – This is the question of This is the question of external validityexternal validity. .

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Solution?Solution?

Use a Use a processprocess where sample data where sample data does generalizedoes generalize

to the units, treatments, variables and settings to the units, treatments, variables and settings not directly observed.not directly observed.

Follow the process called Follow the process called StatisticalStatistical InferenceInference using the using the Scientific MethodScientific Method..

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Statistical InferenceStatistical Inference

Inference

Measurement

Population

Sampling

Sample

Data

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Example: David’s Example: David’s “Cardiac Acute Care Nurse “Cardiac Acute Care Nurse

Practitioner and 30-Day Practitioner and 30-Day Readmission.”Readmission.” ““The purpose of this study was to determine if the The purpose of this study was to determine if the

addition of a cardiac acute care NP (CACNP) to care addition of a cardiac acute care NP (CACNP) to care teams could improve utilization outcomes (ie, time teams could improve utilization outcomes (ie, time of discharge, length of stay, and readmission rates) of discharge, length of stay, and readmission rates) in patients admitted to a cardiovascular intensive in patients admitted to a cardiovascular intensive care unit (CCU).”care unit (CCU).”

From the objectives of David, et al. (Epub) From the objectives of David, et al. (Epub) J J Cardiovasc NursCardiovasc Nurs. . pubmed/24651684

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Classic Steps: Classic Steps: A Paper’s OrganizationA Paper’s Organization

1.1. What’s the question? (What’s the question? (IntroductionIntroduction))– Conceptualize the populationConceptualize the population– State the questionState the question

2.2. How will you answer How will you answer the question? (the question? (MethodsMethods))– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

4.4. What does it mean? (What does it mean? (DiscussionDiscussion))

3.3. Answer the question Answer the question ((ResultsResults))– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

David’s “Cardiac Acute Care Nurse David’s “Cardiac Acute Care Nurse Practitioner and 30-Day Practitioner and 30-Day

Readmission.”Readmission.”What is the PICO question?What is the PICO question?Who is the Who is the PPopulation of interest?opulation of interest?

– P = ?P = ?What is the What is the IIntervention being studied?ntervention being studied?

– I = ?I = ?The intervention is The intervention is CCompared to what?ompared to what?

– C = ?C = ?What is the What is the OOutcome of interest?utcome of interest?

– O = ?O = ?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

David’s “Cardiac Acute Care Nurse David’s “Cardiac Acute Care Nurse Practitioner and 30-Day Practitioner and 30-Day

Readmission.”Readmission.”What is the PICO question?What is the PICO question?Who is the Who is the PPopulation of interest?opulation of interest?

– P = patients admitted to a cardiovascular ICUP = patients admitted to a cardiovascular ICUWhat is the What is the IIntervention being studied?ntervention being studied?

– I = addition of cardiac acute care NPI = addition of cardiac acute care NPThe intervention is The intervention is CCompared to what?ompared to what?

– C = house-staff team without NPC = house-staff team without NPWhat is the What is the OOutcome of interest?utcome of interest?

– O = readmission rateO = readmission rate

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

David’s “Cardiac Acute Care Nurse David’s “Cardiac Acute Care Nurse Practitioner and 30-Day Practitioner and 30-Day

Readmission.”Readmission.” Main question: CompareMain question: Compare– Rehospitalization within 30 days to the Rehospitalization within 30 days to the

Emergency Department in patients Emergency Department in patients cared for cared for by the NPby the NPtoto

– Rehospitalization within 30 days to the Rehospitalization within 30 days to the Emergency Department in patients Emergency Department in patients not cared not cared for by the NPfor by the NP

Design a measurement systemDesign a measurement system– CCU team = staff+NP or staffCCU team = staff+NP or staff– ED return = yes or noED return = yes or no

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

EDreturn staff+NP staff Total %Yes small n Big N ? lowNo Big N small n ? ?Total 100? 100? 200? 100.0

CCU team

Looking AheadLooking Ahead

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Backing up: Backing up: State the State the questionquestion For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate..

The population has a The population has a parameterparameter—call it —call it ππ—we are trying to estimate this using data.—we are trying to estimate this using data.

Inference

Measurement

Population

Sampling

Sample

Data

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

The The populationpopulation has a has a parameterparameter—call it —call it ππ

Conceptualization: The populationConceptualization: The population True:nTrue:nstaff+NPstaff+NP = true count of everyone who = true count of everyone who

was cared for by a NPwas cared for by a NP True:nTrue:nrere = true count of everyone who was = true count of everyone who was

cared for by a NP cared for by a NP andand also readmitted also readmitted π π = true prevalence proportion of readmits = true prevalence proportion of readmits

in NP patients.in NP patients.

π π = True:n= True:nre re / True:n/ True:nstaff+NPstaff+NP

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Estimation of population Estimation of population parameter using parameter using sample sample

statisticstatistic Definition:Definition: A A statisticstatistic is a single descriptive is a single descriptive number computed from the data.number computed from the data.

Conceptualization: The sampleConceptualization: The sample nnstaff+NPstaff+NP = count in sample of NP patients = count in sample of NP patients

nnrere = count in sample of NP patients = count in sample of NP patients andand also readmitted also readmitted

p = p = estimatedestimated proportionproportion of readmits of readmitsin NP patients.in NP patients.

p = np = nre re / n/ nstaff+NPstaff+NP

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Backing up: Backing up: State the State the questionquestion In NP patients, In NP patients, estimate the 30da readmission estimate the 30da readmission

rate.rate.

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

π = True:nre / True:nstaff+NP

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Classic StepsClassic Steps

What’s the question? (Introduction)What’s the question? (Introduction)– Conceptualize the populationConceptualize the population– State the questionState the question

How will you answer How will you answer the question? (Methods)the question? (Methods)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

What does it mean? (Discussion)What does it mean? (Discussion)

Answer the question Answer the question (Results)(Results)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

EstimationEstimation

Analyze the dataAnalyze the data– Readmission of NP patientsReadmission of NP patients

– nnstaff+NPstaff+NP = count in sample cared for by = count in sample cared for by the NPthe NP

– nnrere = count in sample cared for by the NP and subsequently readmitted = count in sample cared for by the NP and subsequently readmitted p = p = estimatedestimated proportionproportion of readmits in NP patients. of readmits in NP patients.

p = np = nre re / n/ nstaff+NPstaff+NP

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

Segue: Segue: estimation errorestimation error For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate. . December, 2008December, 2008

Variability due to sampling:8 patients

Variability due to measurement:nre = 2nNP = 8Excluded = ?

2/8 = 0.25

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

Segue: Segue: estimation errorestimation error For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate.. January, 2009January, 2009

Variability due to sampling:4 new + 8 previous patientsVariability due to measurement:no new readmitsnre = 2nNP = 12

2/12 = 0.17

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

Segue: Segue: estimation errorestimation error For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate.. February, 2009February, 2009

Variability due to sampling:7 new + 12 previous patientsVariability due to measurement:1 new readmitnre = 3nNP = 19

3/19 = 0.16

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

Segue: Segue: estimation errorestimation error For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate.. March, 2009March, 2009

Variability due to sampling:2 new + 19 previous patientsVariability due to measurement:no new readmitsnre = 3nNP = 21

3/21 = 0.14

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Inference

Measurement

Population

Sampling

Sample nnssttaaffff++NNPP

Data pp == nnrree // nnssttaaffff++NNPP

Segue: Segue: estimation errorestimation error For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate.. September, 2010September, 2010

Variability due to sampling:6 new + 103 previous patientsVariability due to measurement:no new readmitsnre = 13nNP = 109

p= 13/109= 0.12

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Estimate of Estimate of p?p?

Analyze the dataAnalyze the data– ED return within 30 daysED return within 30 days

– From sample From sample p = 13/109 = 11.9%/100 = proportion 0.119p = 13/109 = 11.9%/100 = proportion 0.119

– Point estimate of p = 0.119Point estimate of p = 0.119

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Answer Answer the questionthe question For NP patients, For NP patients, estimate the 30day readmission rateestimate the 30day readmission rate. . The population has a The population has a parameterparameter—call it —call it ππ—we are trying to —we are trying to

estimate—using data.estimate—using data.

Inference

Measurement

Population

Sampling

Sample

Data

In the team with the CACNP had a 30-day emergency department return rate of 11.9%

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Classic StepsClassic Steps

What’s the question? (Introduction)What’s the question? (Introduction)– Conceptualize the populationConceptualize the population– State the questionState the question

How will you answer How will you answer the question? (Methods)the question? (Methods)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

What does it mean? (Discussion)What does it mean? (Discussion)

Answer the question Answer the question (Results)(Results)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Defining the research Defining the research question:question:

Testable consequence?Testable consequence? Conceptual progression from general to specificConceptual progression from general to specific General questionGeneral question

– Determine the impact on utilization outcomes Determine the impact on utilization outcomes of NPs on medical teams for cardiovascular of NPs on medical teams for cardiovascular intensive care patientsintensive care patients

Specific hypothesisSpecific hypothesis– Is the 30d readmission rate lower in NP Is the 30d readmission rate lower in NP

patients than in those not cared for by an NP?patients than in those not cared for by an NP? Testable consequenceTestable consequence

– Prediction of a relationshipPrediction of a relationship– Potentially refutable by dataPotentially refutable by data

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Defining the research Defining the research question: Predictionquestion: Prediction

Prediction: a statistical Prediction: a statistical relationship between relationship between intervention and outcomeintervention and outcome

– Readmission will be Readmission will be lower in NP patientslower in NP patientsthan in controlsthan in controls

How can we arrive at this conclusion? How can we arrive at this conclusion? Using a Using a refutablerefutable hypothesis hypothesis

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Defining the research Defining the research question: Formalizationquestion: Formalization

A refutable hypothesisA refutable hypothesis Statistical formalization:Statistical formalization: Ho: proportion readmit(staff+NP) Ho: proportion readmit(staff+NP) ==

proportion readmit(staff)proportion readmit(staff)– Which may be disproved beyond a reasonable Which may be disproved beyond a reasonable

doubt through falsification by data via doubt through falsification by data via statistical hypothesis testing, in favor of:statistical hypothesis testing, in favor of:

Ha: proportion readmit(staff+NP) Ha: proportion readmit(staff+NP) << proportion readmit(staff)proportion readmit(staff)

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Critical AppraisalCritical Appraisal

Is it a testable, research question?Is it a testable, research question? How did they try to rule out bias, confounding, How did they try to rule out bias, confounding,

chance?chance? How did they consider multiple outcome How did they consider multiple outcome

measures and multiple predictors?measures and multiple predictors? Did they disclose what was done with enough Did they disclose what was done with enough

detail so others may replicate?detail so others may replicate?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

How Science Advances How Science Advances Clinical KnowledgeClinical Knowledge

Science forms a question and brings data to bear Science forms a question and brings data to bear to answer the question. to answer the question.

Informally:Informally:– Frame a clinical research question.Frame a clinical research question.– State its testable consequences as either State its testable consequences as either

“just random variability” or “unusual “just random variability” or “unusual outcomes”.outcomes”.

– Compare the actual data with these two Compare the actual data with these two choices and decide which to believe.choices and decide which to believe.

– Discuss our present understanding.Discuss our present understanding.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Testing HypothesesTesting Hypotheses

Or, linking the four steps to Or, linking the four steps to thethe standard standard IMRD organization of a paper:IMRD organization of a paper:

1.1. What’s the question? (What’s the question? (IIntroduction)ntroduction)

2.2. How do you answer the question? How do you answer the question? ((MMethods)ethods)

3.3. Answer the question. (Answer the question. (RResults)esults)

4.4. What does it mean? (What does it mean? (DDiscussion)iscussion)

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

What’s the Question?What’s the Question?

““The purpose of this study was to determine if the The purpose of this study was to determine if the addition of a cardiac acute care NP to care team could addition of a cardiac acute care NP to care team could improve utilization outcomes (ie, time of discharge, improve utilization outcomes (ie, time of discharge, length of stay, and readmission rates …”length of stay, and readmission rates …”

From the Objective of David, et al. (2014) From the Objective of David, et al. (2014) J Cardiovasc J Cardiovasc NursNurs..

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

How do you How do you answer the question?answer the question?

OutlineOutline– Propose two states of naturePropose two states of nature– Use the rule of simplicityUse the rule of simplicity– Take into account that “noise happens”Take into account that “noise happens”– Use a test statistic to decide: Signal or Noise?Use a test statistic to decide: Signal or Noise?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Two states; Two hypothesesTwo states; Two hypotheses

We begin by conceiving the true state of nature We begin by conceiving the true state of nature as as being either: being either: – no difference or no difference or – a difference. a difference.

We We alwaysalways start by assuming that nothing is start by assuming that nothing is going ongoing on—that any apparent differences are —that any apparent differences are purely because of chance. Our preference, as purely because of chance. Our preference, as scientists, is to believe the simplest explanation scientists, is to believe the simplest explanation for a phenomenon.for a phenomenon.– Assume: no difference (AKA Assume: no difference (AKA null hypothesisnull hypothesis).).

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Rule of SimplicityRule of Simplicity When you have two competing theories which make When you have two competing theories which make

exactly the same predictions, the one that is simpler is exactly the same predictions, the one that is simpler is the better.the better.

The simplest explanation for some phenomenon is The simplest explanation for some phenomenon is more likely to be accurate than more complicated more likely to be accurate than more complicated explanations.explanations.

The explanation requiring the fewest assumptions is The explanation requiring the fewest assumptions is most likely to be correct.most likely to be correct.

AKA “Occam’s Razor”AKA “Occam’s Razor”

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Statistical World ViewStatistical World View

Thou shalt not interpret randomness.Thou shalt not interpret randomness. Chance happens. Noise exists.Chance happens. Noise exists.

Making an interpretation that goes beyond this Making an interpretation that goes beyond this requires justification.requires justification.

If random noise, measurement error, or chance If random noise, measurement error, or chance occurrence can account for variations occurrence can account for variations (differences) in the observations, then there is no (differences) in the observations, then there is no need to formulate a more complicated need to formulate a more complicated explanation.explanation.– We embody this We embody this preferencepreference in the statement of in the statement of

the null hypothesis.the null hypothesis.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Null HypothesisNull Hypothesis

We evaluate this proposition using statistical We evaluate this proposition using statistical techniques. techniques.

The null hypothesis is the statement that is tested. The null hypothesis is the statement that is tested.

It’s abbreviated H0:It’s abbreviated H0: A null-hypothesis is the simplest explanation of A null-hypothesis is the simplest explanation of

events:events: There is no difference. There is no change. There is no difference. There is no change. There is no improvement. Nothing unusual is There is no improvement. Nothing unusual is occurring.occurring.

A null-hypothesis is the statement A null-hypothesis is the statement we hope to we hope to contradict with data.contradict with data. That is, we usually hope to That is, we usually hope to reject the null hypothesis.reject the null hypothesis.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Assume: Nothing is going onAssume: Nothing is going on

Proportion readmitted within those cared for by Proportion readmitted within those cared for by the NP the NP is is equalequal to the to the proportion readmitted within those not cared for proportion readmitted within those not cared for by the NP.by the NP.

HO: HO: ππstaff+NPstaff+NP = = ππstaffstaff

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Two HypothesesTwo Hypotheses Proportion readmitted within those cared for by the Proportion readmitted within those cared for by the

NP NP is is equalequal to the proportion readmitted within those to the proportion readmitted within those not cared for by the NP. not cared for by the NP.

HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff

Can we reject the above, in favor of:Can we reject the above, in favor of: Proportion readmitted within those cared for by the Proportion readmitted within those cared for by the

NP NP is is different different than the proportion readmitted within than the proportion readmitted within those not cared for by the NP. those not cared for by the NP.

Ha: PHa: Pstaff+NPstaff+NP ≠ P ≠ Pstaffstaff So: Done with step 1: We’ve stated the question.Next: How will you answer the question?

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Proof By ContradictionProof By Contradiction Nature is either in one state or the other.Nature is either in one state or the other.

– We prefer to believe the simplest explanation.We prefer to believe the simplest explanation. Collect data from the real world.Collect data from the real world. Assess the likelihood of observing this data under Assess the likelihood of observing this data under

the null hypothesis.the null hypothesis. Choose to believe:Choose to believe:

– If the data is within what we would expect then If the data is within what we would expect then we retain our preference for the null-hypothesis we retain our preference for the null-hypothesis as the best explanation.as the best explanation.

– If the data is very unlikely, then we reject the null If the data is very unlikely, then we reject the null hypothesis in favor of its alternative.hypothesis in favor of its alternative.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

What if: HO is true?What if: HO is true? In staff+NP patients, estimate the proportion readmitted.In staff+NP patients, estimate the proportion readmitted. In staff patients, estimate the proportion readmitted.In staff patients, estimate the proportion readmitted.

Inference

Measurement

Population

Sampling

nnssttaaffff++NNPP nnssttaaffff

ppssttaaffff++NNPP == nnrree,,ssttaaffff++NNPP// nnssttaaffff++NNPP

ppssttaaffff == nnrree,,ssttaaffff // nnssttaaffff

π = πstaff+NP= πstaff

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

What if?What if?

Assess the likelihood of observing various Assess the likelihood of observing various data possibilities under the null hypothesis.data possibilities under the null hypothesis.

Assume this is true:Assume this is true:

– HO: PHO: Pstaff+NPstaff+NP == P Pstaffstaff

Then the sample estimate of PThen the sample estimate of Pstaff+NPstaff+NP will be “close to” the sample estimate of Pwill be “close to” the sample estimate of Pstaff.staff.

– By “close to” we mean that, because of By “close to” we mean that, because of sampling variability and measurement sampling variability and measurement error we expect them to be somewhat error we expect them to be somewhat different.different.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Contingency TableContingency Table

“Results: We included 185 patients in this study. …

EDreturn staff+NP staff Total %Yes 32 17.3No 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Contingency TableContingency Table

Assume this is true:Assume this is true:

– HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff = 0.173 = 0.173

EDreturn staff+NP staff Total %Yes ? ? 32 17.3No ? ? 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Contingency TableContingency Table

Assume this is true:Assume this is true:

– HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff = 0.173 = 0.173

– 17.3% of 109 = 18.917.3% of 109 = 18.9

EDreturn staff+NP staff Total %Yes ? 32 17.3No 153 82.7Total 109 76 185 100.0

CCU team

19

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Contingency TableContingency Table

Assume this is true:Assume this is true:

– HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff = 0.173 = 0.173

– 17.2% of 76 = 13.117.2% of 76 = 13.1

EDreturn staff+NP staff Total %Yes 19(17.4%) ? 32 17.3No 153 82.7Total 109 76 185 100.0

CCU team

13

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Contingency TableContingency Table

Assume this is true:Assume this is true:

– HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff = 0.173 = 0.173

– 109 – 19 = 90109 – 19 = 90– 76 – 13 = 6376 – 13 = 63

EDreturn staff+NP staff Total %Yes 19(17.4%) 13(17.1%) 32 17.3No 90(82.6%) 63(82.9%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Test StatisticTest Statistic

Test stat: Test stat: Difference in prevalence Difference in prevalence

– HO: HO: PHO: HO: Pstaff+NPstaff+NP = P = Pstaffstaff = 0.173 = 0.173

– Expected difference = 0%Expected difference = 0%

EDreturn staff+NP staff Total %Yes 19(17.4%) 13(17.1%) 32 17.3No 90(82.6%) 63(82.9%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Testing HypothesesTesting Hypotheses

Recall:Recall:– Frame a clinical research question.Frame a clinical research question.– State its testable consequences as either State its testable consequences as either

“just random variability” “just random variability” or “unusual outcomes”.or “unusual outcomes”.

– Compare the actual data with these two Compare the actual data with these two choices and decide which to believe.choices and decide which to believe.

– Discuss our present understanding.Discuss our present understanding.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

I presume the null-I presume the null-hypothesis is true, do the hypothesis is true, do the

data support this?data support this? Observed difference = 0.3%Observed difference = 0.3% P-value = 0.9549P-value = 0.9549

EDreturn staff+NP staff Total %Yes 19(17.4%) 13(17.1%) 32 17.3No 90(82.6%) 63(82.9%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

I presume the null-I presume the null-hypothesis is true, do the hypothesis is true, do the

data support this?data support this? Observed difference = 1.4%Observed difference = 1.4% P-value = 0.736P-value = 0.736

EDreturn staff+NP staff Total %Yes 18(17%) 14(18.4%) 32 17.3No 91(83%) 62(81.6%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

I presume the null-I presume the null-hypothesis is true, do the hypothesis is true, do the

data support this?data support this? Observed difference = 3.7%Observed difference = 3.7% P-value = 0.464P-value = 0.464

EDreturn staff+NP staff Total %Yes 17(16%) 15(19.7%) 32 17.3No 92(84%) 61(80.3%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

I presume the null-I presume the null-hypothesis is true, do the hypothesis is true, do the

data support this?data support this? Observed difference = 42%Observed difference = 42% P-value < 0.001P-value < 0.001

EDreturn staff+NP staff Total %Yes 0(0%) 32(42.1%) 32 17.3No 109(100%) 44(57.9%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Trade offsTrade offs

Alpha = Type I error = prob. of rejecting a true null hypothesisAlpha = Type I error = prob. of rejecting a true null hypothesis

Beta = Type II error = prob. of not finding a true differenceBeta = Type II error = prob. of not finding a true difference

  Conclusion

Truth

Do not reject null-hypothesis

(p-value > .05)

Reject null-hypothesis (p-value < .05)

Null-hypothesis (no difference)

correct Type I error

Alternative hypothesis (difference)

Type II error correct

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Significance LevelSignificance Level

The significance level is represented by the Greek symbol “alpha”, α.

It is the probability of rejecting a true null hypothesis.

The researcher chooses the risk of making this error: concluding that the null hypothesis is false when it really is true. – The most typical values are α = .05, .01,

or .10.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Universal Decision RuleUniversal Decision Rule

ChooseChoose to believe: to believe: HO: null-hypothesisHO: null-hypothesis

(For non-extreme values of the test (For non-extreme values of the test statistic)statistic)– Choose this if p-value ≥Choose this if p-value ≥ α (usually 0.05) (usually 0.05)

HA: alternative-hypothesisHA: alternative-hypothesis

(For extreme values of the test statistic)(For extreme values of the test statistic)– Choose this if Choose this if p-value <p-value < α (usually 0.05)(usually 0.05)

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

! Are we done yet? !! Are we done yet? !

What’s the question? (Introduction)What’s the question? (Introduction)– Conceptualize the populationConceptualize the population– State the questionState the question

How will you answer How will you answer the question? (Methods)the question? (Methods)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

What does it mean? (Discussion)What does it mean? (Discussion)

Answer the question Answer the question (Results)(Results)– The sampleThe sample– The measurementsThe measurements– Analysis techniqueAnalysis technique

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

I presume the null-I presume the null-hypothesis is true, do the hypothesis is true, do the

data support this?data support this? Observed difference = 13.1%Observed difference = 13.1% P-value = 0.0207P-value = 0.0207

EDreturn staff+NP staff Total %Yes 13(11.9%) 19(25%) 32 17.3No 96(88.1%) 57(75%) 153 82.7Total 109 76 185 100.0

CCU team

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Chi-square testChi-square test Expected dataExpected data

Observed data (chi-square, P-value = Observed data (chi-square, P-value = 0.0207)0.0207)

EDreturn staff+NP staff Total %Yes 19(17.4%) 13(17.1%) 32 17.3No 90(82.6%) 63(82.9%) 153 82.7Total 109 76 185 100.0

CCU team

EDreturn staff+NP staff Total %Yes 13(11.9%) 19(25%) 32 17.3No 96(88.1%) 57(75%) 153 82.7Total 109 76 185 100.0

CCU team Table 2:P = .011

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

State a ConclusionState a Conclusion Proportion readmitted within those cared for by the Proportion readmitted within those cared for by the

NP NP is is equalequal to the proportion readmitted within those to the proportion readmitted within those not cared for by the NP. not cared for by the NP.

HO: PHO: Pstaff+NPstaff+NP = P = Pstaffstaff

Can we reject the above, in favor of:Can we reject the above, in favor of: Proportion readmitted within those cared for by the Proportion readmitted within those cared for by the

NP NP is is different different than the proportion readmitted within than the proportion readmitted within those not cared for by the NP. those not cared for by the NP.

Ha: PHa: Pstaff+NPstaff+NP ≠ P ≠ Pstaffstaff Evidence: 11.9% readmit vs 25% readmit (P =.0207)

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

P-valueP-value

The p-value is the The p-value is the probability that the data probability that the data occurred by chanceoccurred by chance, assuming the null hypothesis , assuming the null hypothesis is true.is true.

The p-value is NOT the probability that the null-The p-value is NOT the probability that the null-hypothesis is true.hypothesis is true.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

The p-value is NOT The p-value is NOT the probability that the probability that the null-hypothesis the null-hypothesis

is true.is true.

(and 1−pvalue is NOT the probability that (and 1−pvalue is NOT the probability that the alternative hypothesis is true)the alternative hypothesis is true)

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

Trade offsTrade offs

Alpha = Type I error = prob. of rejecting a true null hypothesisAlpha = Type I error = prob. of rejecting a true null hypothesis

Beta = Type II error = prob. of not finding a true differenceBeta = Type II error = prob. of not finding a true difference

  Conclusion

Truth

Do not reject null-hypothesis

(p-value > .05)

Reject null-hypothesis (p-value < .05)

Null-hypothesis (no difference)

correct Type I error

Alternative hypothesis (difference)

Type II error correct

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

ActualityActuality

Alpha = Type I error = prob. of rejecting a true null hypothesisAlpha = Type I error = prob. of rejecting a true null hypothesis

Beta = Type II error = prob. of not finding a true differenceBeta = Type II error = prob. of not finding a true difference

  Results

Truth

Do not reject Do not reject null-hypothesis null-hypothesis

( (pp-value -value > .05)> .05)

Reject null-Reject null-hypothesis hypothesis ((pp-value < .05)-value < .05)

Null-hypothesis (no difference) Blind alleyBlind alley ??

Alternative hypothesis (difference) ?? Discovery!Discovery!

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

P-valueP-value

A modest reality:A modest reality:

The p-value is simply the probability that the data The p-value is simply the probability that the data occurred by chance.occurred by chance.

Big leap:Big leap:

A significant p-value is a license to A significant p-value is a license to make up a story.make up a story.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

DiscussionDiscussion

““The addition of a CACNP to a multidisciplinary inpatient The addition of a CACNP to a multidisciplinary inpatient medical team caring for myocardial infarction and heart medical team caring for myocardial infarction and heart failure patients was associated with lower 30-day emergency failure patients was associated with lower 30-day emergency department readmission and 30-day hospital readmission department readmission and 30-day hospital readmission rates.”rates.”

ConclusionConclusion ““It is recommended that CACNPs be considered for patient It is recommended that CACNPs be considered for patient

teaching, care coordination, and multidisciplinary integration teaching, care coordination, and multidisciplinary integration to reduce costly rehospitalizations of patients with heart to reduce costly rehospitalizations of patients with heart failure and myocardial infarction.”failure and myocardial infarction.”

**See page 443, the last Discussion paragraph.See page 443, the last Discussion paragraph.

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

ReviewReview

To assess validity, ask:To assess validity, ask:– What’s the question?What’s the question?– Where did the data come from? (Sampling and Where did the data come from? (Sampling and

measurement.)measurement.)– What would you expect if “nothing is going What would you expect if “nothing is going

on”?on”?– Is the observed data different than that?Is the observed data different than that?

And consider that other factors could account for And consider that other factors could account for the observed differencethe observed difference– Bias, confounding, multiplicity, chanceBias, confounding, multiplicity, chance

V I R G I N A C O M M O N W E A L T H U N I V E R S I T Y

www.phdcomics.com/comics/archive.php?comicid=905

Questions?Questions?