H860 Reading Difficulties Week 7 Reading Interventions: How Do They Weigh Up?

H860 Reading Difficulties

Week 7

Reading Interventions: How Do They Weigh

Up?

Today’s session

1. ‘Urban’ presentation2. Evaluating reading programs: intro3. Break4. Your turn to evaluate

Today’s feature presentation…

The Issue

We want to know that the reading instruction we give is maximally effective

Before you came to HGSE, when deciding what reading program was perhaps a much simpler task, what would shape your decision?

But wait, there are a few hurdles here…

• Hurdle number 1

We are human…

Cognitive Heuristics

• The availability heuristic: judging the likelihood of something on how easy it is to think of an example

• The anchoring and adjustment heuristic: tendency to form an initial impression of something and then insufficiently adjust that on the basis of new information

• Also: practice in a technique leads to success with specific students

Reinforces teacher’s belief in effectiveness of technique

But risky! Are these students representative of everyone with reading difficulties?

• We can’t avoid being human, but we can be aware of these thought biases……and we can try and increase the pool of information from which our decisions can be based…

But wait, there are a few hurdles here…

• Hurdle number 2

We could try turning to the research-base – but how do we know what to believe?

Research

• Not all research is fully ‘scientific’

• Scientific Inquiry = rigorous reasoning supported by interplay of methods, theories and findings = hard work

• But Stanovich and Stanovich can help us!

The difficulty of doing good research is again confounded by…

• Coalition for Evidence-Based Policy

“There are a vast array of educational interventions that claim to improve educational outcomes and to be supported by evidence……introduced with great fanfares as being able to produce dramatic gains……yielding little in the way of positive and long lasting changes” - The What Works Clearinghouse

N.B…

• fad  (f æ d)n.A fashion that is taken up with great enthusiasm for a brief period of time

Enthusiasm is not the problem here, but the lack of a strong scientific research base can lead to brevity and hence, a ‘fad’.

N.B…

• Also think:

Is controversy about effectiveness of program per se, or the bravado of the claims?

e.g. Fastforward, Reading Recovery…

Strong scientific research

1. Published in peer-reviewed journal2. Replicated?3. Is there a consensus?

(n.b. due to differences in methods, populations etc. there will almost always be conflicting evidence)

We clearly access to strong scientific research, both personally, and also because…

NCLB

• No Child Left Behind Act, Reading First, 2001

• Requires methods that are used in the classroom to teach children how to read to be based upon valid scientific findings

• Based on findings from the National Reading Panel: meta-analysis of scientific studies on reading that met certain criteria

What if it’s not mentioned by NRP…?

Randomized Control Trials

Groups

• Intervention (s) • Active control (to control for placebo)• Passive control (no intervention)

Randomized Control Trials

Design

• Randomized assignment of participants into different groups

• Groups are equal in reading before intervention

• Compare the groups after intervention (1)• Cross-over design (2)

Randomized Control Trials

1.

2.

Early/bad RCTs

• Translating a medically-oriented design to education requires care…

The case of speech therapy

Aphasia therapy

Lincoln et al (1984)• 191 patients untreated or treated by

therapist• 16% dropped out• 2 hrs per week treatment over 24 weeks

(but 3/4 had less than 18hrs total)• no difference in outcome for treated and

untreated

• Can argue about methods• Can argue about the population patients

were drawn from• Can argue about type and amount of

treatment• Can argue about whether treatment

effects were measured in the right way

• Particular treatments will be effective for particular clients

Aphasia therapy

But things are better now!

Or at least have the potential to be…

• Whatworksclearinghouse – some elegant examples of RCTs

• Refinement of quasi-experimental designs:

Equating design = participants are not randomly assigned to intervention and control groups, but the groups are equated/appropriate statistical controls are put in place

Single-case designs = repeated measurement of a single individual in different conditions or phases over time

RCT or Quasi-experiment+

Effective in one or more setting=

Strong evidence!

Your turn…

In groups…

• What is the source?• Who are the participants/how many?• What is the setting?• Is it an RCT or quasi-experiment?• What comparison groups are used?• Who is carrying out the instruction?• What are the outcome measures – are they

appropriate? • How effective is the intervention?• Is there a long-term follow-up?

• Overall rating out of 10

• http://www.badscience.net/?p=320

Take-away

• Scientific research is a process• It will not provide us with all the answers,

right now

But

• It gives us more information than we have ever had before

• If you understand it, you can do it in your classroom!

Break

Have a good spring break!