Design for Learning: Improving Patient Safety

Design for Learning:Improving Patient Safety

Lou Fogg, PhDProject Statistician

February 1, 2012Phase II Site Coordinator Meeting

Who Am I?

Lou Fogg, Ace Statistician I was trained as a mathematical

psychologist I have been working in Nursing research

since 1987 I enjoy working with ‘messy’ data

What is our Design?

The study we are conducting is called a ‘randomized cluster design’

This means that we collect data from a relatively large number of organizations, and we assign them at random to either the intervention group, or a control group

Stochastic Matching

In addition, there were a set of three variables (such as the number of new nursing hires expected each year) that we wanted to keep roughly equivalent across the two treatment groups

In order to achieve this initial balance, we used a technique called ‘Stochastic Matching’

Stochastic Matching-2

Stochastic matching is done by generating many different random assignments for these organizations, and then selecting the subset of these randomization schemes that leave the key variables roughly equivalent between our two groups

The fear is that if we let these variables vary freely, there will be a large difference among the groups

This is called a failure of randomization

Failure of randomization

Failure of randomization is a serious problem, and is especially serious in randomized cluster designs because the overall sample size (of organizations) is fairly small, and randomization failures occur more often in small samples

So, this makes it more likely that we will be able to make valid comparisons between the two groups

Missing Data

Missing data is a serious problem in nursing research

While there are algorithms for ‘imputing’ (statistician-talk for ‘making up’) missing values, the confidence in your results drops the more missing data we are forced to make up…Oops!, I mean impute…

Missing data

So while I know that it can be very difficult to collect complete data for a study like this, it is very important for our ability to understand how well the intervention actually works

And the less able we are to make this determination, the less useful these data will be for understanding 1) if the intervention works, and 2) what we can do to make it work better

Finally

Some of you will be assigned to the control group and some to the intervention group today.

But you are all precious to a statistician, we need as much of both groups’ data as we can possibly get, in order to obtain valid results.

So, don’t think that because you are in the control group, that we value your contribution any less. It ain’t so…

Working with Messy Data

I like to work with ‘messy’ data, but I don’t court disaster

It is more difficult to draw useful inferences from data that is too ‘messy’

So, I am asking for all of you to help me out and make this the ‘neatest’ (least messy) study that we can

Questions?

Questions? Comments? Suggestions?

And Thank You for being a nice audience

And I’ll leave you with one of my favorite Chicago pictures—the Chicago lighthouse