SampleCon 2013: Chuck Miller Explains Routing

Once upon a time…

A Brief History of Routers,

Our World Today, and

Implications for

the Future

How it Began (and Why)

Sampling Theory & Managing Bias

Next Steps for Router R-on-R

Takeaways for the Future

Rivers need Routers. Routers don’t need Rivers.

Scalable

Valid

Replicable

Changing Market Forces and The Quest for Validity

Sampling procedures should be employed to produce research having the greatest validity, through the reduction of error.

Validity and Quality are not binary.

There is a continuum.

Sampling 101

• Coverage Error – The result of all units in a defined population not having a known non-zero

probability of inclusion in the sample drawn to represent the population.

• Sampling Error – The result of surveying a sample of the population rather than the entire

population. Inherent by virtue of the scientific method. You know the rules…

• Nonresponse Error – The result of not getting some people to respond to the survey who, had they

done so, would have provided different answers than those who did respond.

Managing Bias

• Coverage Error – Source broadly

– Identify and balance characteristics

– Stabilize your sampling frame

• Sampling Error – Establish business rules, then draw samples consistently

– Analyze reasonable bases

• Nonresponse Error – Understand your audience, and survey taking modes

– Provide a reasonable survey instrument

– Provide a reasonable (yet nominal) incentive

Managing Bias

• Coverage Error – Source broadly

– Identify and balance characteristics

– Stabilize your sampling frame

• Sampling Error – Establish business rules, then draw samples consistently

– Analyze reasonable bases

• Nonresponse Error – Understand your audience, and survey taking modes

– Provide a reasonable survey instrument

– Provide a reasonable (yet nominal) incentive

Takeaways

• The evolution of routers is a great example of adaptation to a changing world

• With that evolution comes appropriate critical evaluation – giving ourselves a grade

• Our challenge is to use this guidance to refine our methods and create totally new tech-driven approaches

The Challenge for Researchers

Aware that there is no perfect research and that our environment is ever-changing, we understand our challenge:

• Maintain perspective, with a foundation in research fundamentals

• Use CREATIVITY and CRITICAL EVALUATION to develop our next generation solutions And execution wins.

Questions for the Day

Problems present opportunities… what do we tackle next?

Which areas are ripe for new innovations?

Change necessitates evolution… what’s next?

Thank You

Chuck Miller [email protected]