1

FY10 ACS Methods Panel Update

Jennifer Guarino TancretoChief, ACS Data Collection Methods Staff

Decennial Statistical Studies Division

Presentation to COPAFSMarch 5, 2010

2

ACS Methods Panel

Vehicle for testing changes to ACS

• Improving current questions/adding new questions

• Testing new methods to improve cost-efficiency

3

FY 2010 ACS Methods Panel

• Results of two tests from 2009

• Conduct one test

• Plan for two future tests

4

Tests conducted in FY09

5

Additional Mailing to Mail Nonrespondents

• Goal: improve mail response

• Postcard or another questionnaire package – Mailed ~2 weeks after replacement questionnaire

• Tested in March 2009– 1/3 got third questionnaire– 1/3 got postcard– 1/3 did not get an additional mailing

6

Current ACS Data Collection Methods

MailTelephone Follow-Up

NR NRPersonal

Visit Follow-Up

No

Phone #? Yes Subsample

Pre-noticeLetter

Initial Questionnaire

Reminder Postcard

Replacement Questionnaire

NR

7

ACS Data Collection with an Additional Mailing

MailTelephone Follow-Up

NR NRPersonal

Visit Follow-Up

No

Phone #?Yes Subsample

AdditionalMailing

NR

88

9

Additional Mailing to Mail Nonrespondents

Response Rate by Treatment

21.3% 22.3%

15.2%

0%

10%

20%

30%

Control Postcard Questionnaire

Treatment

Res

pons

e R

ate

10

Multi-lingual Brochure Test

• Goal: assist non-English speaking households in responding to ACS

• Mail brochure in five languages: English, Spanish, Chinese, Russian, Korean– Dedicated toll-free number for each language

• Tested in April through July 2009 – 1/3 got brochure with pre-notice– 1/3 got brochure with initial questionnaire– 1/3 did not get brochure

1111

12

13

Multi-lingual Brochure Test

• Results imminent

• Study impact on response, specifically

– Response from households that report speaking a language in the brochure

14

FY10 Testing

15

2010 ACS Content Test

• Similar in design to 2006 ACS content test

• OMB Interagency Committee for the ACS provided structure/process for identifying new and revised content

16

2010 ACS Content Test - New Questions

1. Computer Ownership and Internet Access (FCC)

2. Parental Place of Birth (Census)

17

2010 ACS Content Test - Revised Questions

1. Food Stamps (USDA)– Change program name to SNAP

2. Veteran Identification and Period of Service (VA)

3. Public Assistance Income (HHS)

4. (CATI/CAPI only) Wages/Salary/etc. and Interest/Dividends/etc. income (HHS)

18

2010 ACS Content Test – Process

• Subcommittees determine preliminary questions

• Cognitive testing

• Expert Panel

• Field Test

19

2010 ACS Content Test

• Split-panel design

– Control: current questions + version 1 of new questions– Test: revised questions + version 2 of new questions

• Initial sample of 35,000 housing units per panel

– Nonresponse follow-up by telephone, pers. visit– Behavior coding using recordings (CARI)– Content follow-up reinterview

20

2010 ACS Content Test

• Field test in September – November 2010

• Final changes in ACS production in January 2013

21

Plans for Future Testing

2222

2011 ACS Internet Test

• Evaluate feasibility of Internet option for ACS and PRCS

– English and Spanish

• Experimentally test ways of offering the Internet option to maximize Internet response

23

2011 ACS Internet Test

• Treatments:

– Prominent Offer (Choice) – Not Prominent Offer– Push Internet– Push Internet (modified mail schedule)

• Control (production): No Internet offer

24

25

26

27

2011 ACS Internet Test

• Two strata: Targeted and Not Targeted

• Fully factorial design of 10 experimental panels

• Main evaluation measure – response rates

28

2011 ACS Internet Test

• Designed to present stimuli consistent with other ACS data collection modes to minimize potential mode effects

But at the same time…

• Take advantage of the technology to improve data quality to extent possible

29

30

Open-ended question

31

Mark one

32

Check boxes

33

2011 ACS Internet Test

• Content reinterview (by phone) of INT respondents

• Follow-up interview about materials and Internet accessibility

• Test planned for March 2011

• Results expected late 2011

34

ACS Content Reinterview Survey

• Study and monitor response error for ACS questions

– Help identify items for content testing– Provide concrete estimates of response error

• Exploring use of Latent Class analysis

• Timing TBD

35

Contact

Jennifer Guarino [email protected]

301-763-4250