Survey Perspective American Association for Public Opinion Research (AAPOR) Institute for Employment Research (IAB) Joint Program in Survey Methodology.

Survey PerspectiveAmerican Association for Public Opinion Research (AAPOR)

Institute for Employment Research (IAB)

Joint Program in Survey Methodology (JPSM)

[email protected]

Big Change in Survey Environment - Administrative data largely unused - New emerging data - Surveys are expensive and take time …

BUT - Big Data / Admin data often variable poor - Linkage necessary

Challenge to Inference

RespondersSample

Consenters

Non-Consenters

Non-Responders

Sample Frame/Admin Data

Linked

Non-Linked

Selectivity of Consent

Age, race/ethnicity, gender, education, marital status, wealth, earnings, health status, health insurance, employment Sala et al., 2014; Sala et al., 2012; Bates and Pascale, 2006; Jenkins et al., 2006; Banks et al., 2005; Dunn et al., 2003; Young et al., 2001; Woolf et al, 2000; Olson, 1999; Pullen et al., 1992

Item missing data, interviewer characteristicsSala et al., 2012 ; Jenkins et al., 2006

Wording and placement of consent request Sala et al., 2014; Sakshaug and Kreuter, 2014; Sakshaug et al., 2013

The information you have provided so far would be a lot more valuable to us if we could link it to public voting records. Do we have your permission to link your answers to your voting record?

The information you have provided so far would be a lot less valuable to us if we could link it to public voting records. Do we have your permission to link your answers to your voting record?

The information you have provided so far would be a lot less valuable to us if we could not link it to public voting records. Do we have your permission to link your answers to your voting record?

12% more “yes” answers

Other A-B tests on consent

o Front 95% vs. back 86%o Variation by interviewer: 75 – 99%o Across studies in US: 24% - 85%

Question for the discussion

Can we make a reasonable claim that current consent requirements are not working?- Because hard understand- Because hard to know what use in future- Because consent easily influenced by wording/framing

Can we shift responsibility?

Responsible data USER

Consent Bias – 1 of many errors

Total Survey Error

- Successful use in ‘small data’

- Good starting point

- Development needed

Always in light of

- Research question

- Hypothesis

Groves et al. 2004

13

UKDA, Essex will be next

Data Access I