Top Banner
Are the Hard-to-Cover Also Less Likely to Respond? AAPOR 2015 Stephanie Eckman Frauke Kreuter
18

Are the Hard to Cover Also Less Likely to Respond?

Feb 13, 2017

Download

Data & Analytics

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Are the Hard to Cover Also Less Likely to Respond?

Are the Hard-to-Cover Also Less Likely to Respond?

AAPOR 2015

Stephanie EckmanFrauke Kreuter

Page 2: Are the Hard to Cover Also Less Likely to Respond?

2

Motivation

Reducing undercoverage means:‐ Phone: calling cell numbers ‐ Face-to-face: including homeless, institutionalized‐ Web: providing tablet, internet access

Costly

Are the additional people included disproportionately nonresponders?

Page 3: Are the Hard to Cover Also Less Likely to Respond?

3

Examples of Trade-Off

RDD + mobile phone surveys (AAPOR Cell Phone Task Force Report, 2010)

‐ Lower RR among mobile only HHs

LISS online panel (Leenheer & Scherpenzeel, 2013)

Screener experiment (Tourangeau, Kreuter & Eckman, 2012)

Condition Coverage Rate Recruit. RateInternet not provided 90% 84%Internet provided ~ 100% 79%

Condition Coverage Rate Response RateDirect: “Is anyone there 35-55?” 32% 86%Roster: Age of all adults in HH 45% 72%

Page 4: Are the Hard to Cover Also Less Likely to Respond?

4

Simulation Setup

Coverage propensity CPi‐ Propensity for case to be included on frame‐ Determined by X variables: ‐ comfort with technology, attachment to HH

Response propensity RPi‐ Propensity for case to respond ‐ Determined by Z variables‐ at-home patterns, privacy concerns, topic interest

Page 5: Are the Hard to Cover Also Less Likely to Respond?

5

Path Diagram

X

Z

Y

CP

RP

Page 6: Are the Hard to Cover Also Less Likely to Respond?

6

Path Diagram

X

Z

Y

CP

RP

ρ

βX

βZ

γX

γZ

Page 7: Are the Hard to Cover Also Less Likely to Respond?

7

Simulation Setup

Y related to X, Z

Survey A:

Survey B:‐ CP increased by κ ‐ RP unchanged

Page 8: Are the Hard to Cover Also Less Likely to Respond?

8

Simulation Setup

Y related to X, Z

Survey A:

Survey B:‐ CP increased by κ ‐ RP unchanged

Parameter Values0.2, 0.6, 1-0.2, -0.6, -1

0.2, 2

0.2, 2

0.2, 2, -0.2, -2

0.2, 2, -0.2, -2κ 1, 4, 10

Page 9: Are the Hard to Cover Also Less Likely to Respond?

9

Outcome Measures

When we increase coverage (A →B), what happens?

Coverage rate, RR‐ Under what conditions do we see trade-off?

Nonresponse Undercoverage Bias Bias

‐ Under what conditions will bias decrease (in absolute value)?

Page 10: Are the Hard to Cover Also Less Likely to Respond?

10

Results: Rates

cov(X,Z) > 0; strong relationship between X, CP & Z, RP

Page 11: Are the Hard to Cover Also Less Likely to Respond?

Results: When Does Absolute Bias Decrease?

X

Z

Y

CP

RP

ρ

βX

βZ

γX

γZ

11

+

Page 12: Are the Hard to Cover Also Less Likely to Respond?

Results: When Does Absolute Bias Decrease?

X

Z

Y

CP

RP

ρ

βX

βZ

γX

γZ

12

+

Page 13: Are the Hard to Cover Also Less Likely to Respond?

Results: Bias

X

Z

Y

CP

RP

ρ

βX

βZ

γX

γZ

13

+

Page 14: Are the Hard to Cover Also Less Likely to Respond?
Page 15: Are the Hard to Cover Also Less Likely to Respond?

www.iab.de

[email protected]

Website: stepheckman.com

Thanks – comments & ideas welcome

Page 16: Are the Hard to Cover Also Less Likely to Respond?

16

Results: Rates

Page 17: Are the Hard to Cover Also Less Likely to Respond?

17

Results: Rates

Page 18: Are the Hard to Cover Also Less Likely to Respond?

18

Coverage, Response Rates for 2 Surveys

CaseLandline Only Landline + Mobile

Covered Response Propensity Covered Response

Propensity

1 Yes 0.48 Yes 0.482 Yes 0.60 Yes 0.603 No 0.15 Yes 0.154 No 0.36 Yes 0.365 No 0.26 Yes 0.26

Coverage Rate 40% 100%Response Rate 54% 37%