Department of Statistics 1 December 2006 SSP Core Facility What Researchers Need to Know about the Statistical Consulting Process or Working with the Statistician.

1 December 2006 SSP Core Facility

Department of Statistics

What Researchers Need to Know about the

Statistical Consulting Processor

Working with the Statistician as a

Research Colleague

Walt StroupProfessor & Chair, UNL Department of Statistics



Outline for Talk

I. Role of a Statistician

II. Statistical View of Research Process From planning to finished product

III. Essential features of an effective partnership What is expected of you; what you should expect

IV. What can go wrong & what to do about it



Cool Quote – Source Unknown

Writers tell stories with words

Painters tell stories with pictures

Musicians tell stories with music

Statisticians tell stories with numbers

HoweverThere must be a story to tellor the words, pictures, music,numbers --- and sophisticated methodology ---are “...full of sound and fury, signifying nothing...”



So...Statisticians:

The object of consulting is

The object of consultingconsulting collaborationisThe Sciencenotthe math nor the statistical techniques



Cool Quote – Source Unknown

There are three subjects everybody thinks they can be an expert in with no formal training:−Law

−Medicine

−Statistics

NOT !!!



Role of Statistics in Research

Another cool quote, source also unknown: “Think of ’Statistics’ as ‘the rules of evidence

for scientific inquiry’” Webster: “Statistics is a branch of

mathematics...”

Better: Oxford English Dictionary – “Statistics is the science of data collection, analysis, and reporting for the purposes of characterizing a population, testing a theory, or making a decision



Some Influences

My Major Professor: R.L. Anderson “This is the worst data set I ever saw! Why didn’t they talk to me first? Unbelievable!”

GeraldHahn

90% of the value of statistics is in what happens before any data are collectedWhen people see me after the fact I often feel thatthe only thing I can do is write the death certificate

“Don’t argue – take data”

“You need to learn to think of yourself not as a statistical consultant but as a research consultant”



E. O. Wilson

not the exact quote, but true to spirit:

What sets good research apart is

usually not one’s cleverness in finding

the answer but one’s skill in•asking the right question, and•phrasing it carefully and well

to which I would add:“...and collecting the right data...”

Theme?Planning!





Steps in Research Planning Protocol Description

What is the Research Question?

Relevant Evidenceto answer question(response variables

How will evidence beused to answer(hypothesis test?)

How will evidencebe collected – design?(clinical trial model? sample survey? )

How much evidence?(sample size?)

Collect data

Data analysis ConclusionsNew questions

Answer

?



When is this Planning Applicable?

Grant writing

Preparing a dissertation proposal Journal Publication Informing Policy Decisions



late Senator Daniel Patrick Moynihan

we in this nation face a choice

we can inform public policywithsound research

or

we can continue to make policy on the basis of data-free ideology



Another of “Walt’s mentors”: Dal Kratzer

defined statistics as “experimentation science” Protocol description form

Intended for pharmaceutical research & development

With word-smithing and tweaking, readily applicable as a template for research planning



Protocol Description FormWhat are the objectives – these must be translated into specific questions that can be answered with data Key: operating definitions

What is the relevant evidence? Distinguish between essential variables and auxilliary variables

What are the measurement units? This implied defining the target population & how you intends to sample / observe it

What is the treatment design?

Experiment design?

Power analysis – how many observations do I need

Murphy’s Law – what can go wrong? what will I do?

How will I analyze the data? How does this relate to objectives?

Plan for reporting? Target audience? Intended purpose



Working with Statistician

Haste makes waste Build a relationship Plan to educate statistician about your research

−Two way street

−You’re not a statistical expert; the statistician is not an expert in your discipline

“De-jargonization”



What you don’t want



Types of statistical investigations

“Data mining”−characterization, “fishing expedition”

− looking for patterns that “might” be there

− latent class, factor analysis, other multivariate methods

Associations−“cross-tabs”, log-linear models, path analysis

−often based on survey or retrospective data

Treatment or factor effect−based on experiment design or quasi-design

−“clinical trial” model; closest thing to cause & effect



Essential Parts of the Process - I

What is the research question?− It is easy to be vague

− It is easy to be vague when you think you aren’t

−Operating definitions are essential

Is “Math in the Middle” effective?−what do you mean by “effective”?

−compared to what?

−once “effective” has an operating definition, can you measure it? how?

− if you can’t, you don’t have an operating definition yet



Essential Parts of the Process - II Power First

−what is the population?−what elements represent the population?−how do you (ethically) observe the elements?

Then−what are the relevant sources of variation among the

elements?−what is the magnitude of variation assoc with these

sources?−how big an effect is considered important?



What power analysis is NOT



Essential Parts of the Process - III

Power Should be a dialog At its best

−mutual learning

−dress rehearsal for data collection and analysis

INCLUDE: what can go wrong & what is my plan?−missing data happen

−missing data are inevitable



Essential Parts of the Process - IV

Stay away from jargon and “discipline-speak”−applies to you−applies to statistician

Ask questions−don’t let “stat-speak” go by− it’s not your ignorance, it’s the statistician’s language skills

Answer questions− if a good statistician asks you a question, it’s because she is

curious & trying to get herself in the best position to be helpful

−questions are not a hidden agenda to find fault or embarrass you



Essential Parts of the Process - V

If possible, face-to-face meetings are better Haste makes waste

− last minute, rush-job consulting invites trouble

Another “mentor quote”−“we don’t have time to do it right, but we have plenty of

time to do it over”

EGO



Analysis

Vastly easier in well-designed study Vastly better if statistician has been involved all

along Parsimony

− if two analyses show essentially same results and plausibly satisfy assumptions, the simpler analysis is better

−90% rule

However− just because it is traditional does not make it right

−computer has had huge impact on statistical methods



Analysis & Reporting Part of statistician’s job is to help write report

−material & methods: statistical design analysis

− result and discussion help explain and interpret

Statistical science itself is a work in progress−your project may motivate statistical research

−methods may be unfamiliar to reviewers

−statistician may need to educate referees / editors



Etiquette & Ethics Authorship

−would research / grant have been possible without statistician’s efforts?

−NO means a co-authorship should be offered

Funding−external funding is lifeblood of research university

−statistician’s dilemma in academia:

−“do I close my door to colleagues in allied disciplines to write my own grants, or do I participate in interdisciplinary teams?”

− for latter, co-PI grant arrangements with real $$$ essential



More Etiquette & Ethics Special Note on Graduate (Dissertation) Research

Common Unfortunate Situation: Graduate Student as Ping-Pong Ball

When statistician advises graduate student on design and/or analysis (ideally, “and” – ed.) student, student’s major professor, and statistician should be on same page

Ideal – statistician on PhD committee But at least – meet face to face periodically



Final Thought

Most Statistics departments are seriously understaffed – UNL no exception

Not a complaint – just a reality Conflicting demands

− teaching

−research

−consulting

Applied statistics need interdisciplinary work – just as quality research needs engaged statistics

BUT – you may have to be a nag

that’s all

thanks

Any Questions?

Department of Statistics 1 December 2006 SSP Core Facility What Researchers Need to Know about the Statistical Consulting Process or Working with the Statistician.

Documents

ssp core facility slide

value of statistics

ssp core facility steps

ssp core facility outline

research consultant

law medicine statistics

decision slide

research question