1 FY10 ACS Methods Panel Update Jennifer Guarino Tancreto Chief, ACS Data Collection Methods Staff Decennial Statistical Studies Division Presentation to COPAFS March 5, 2010
Mar 31, 2015
1
FY10 ACS Methods Panel Update
Jennifer Guarino TancretoChief, ACS Data Collection Methods Staff
Decennial Statistical Studies Division
Presentation to COPAFSMarch 5, 2010
2
ACS Methods Panel
Vehicle for testing changes to ACS
• Improving current questions/adding new questions
• Testing new methods to improve cost-efficiency
3
FY 2010 ACS Methods Panel
• Results of two tests from 2009
• Conduct one test
• Plan for two future tests
4
Tests conducted in FY09
5
Additional Mailing to Mail Nonrespondents
• Goal: improve mail response
• Postcard or another questionnaire package – Mailed ~2 weeks after replacement questionnaire
• Tested in March 2009– 1/3 got third questionnaire– 1/3 got postcard– 1/3 did not get an additional mailing
6
Current ACS Data Collection Methods
MailTelephone Follow-Up
NR NRPersonal
Visit Follow-Up
No
Phone #? Yes Subsample
Pre-noticeLetter
Initial Questionnaire
Reminder Postcard
Replacement Questionnaire
NR
7
ACS Data Collection with an Additional Mailing
MailTelephone Follow-Up
NR NRPersonal
Visit Follow-Up
No
Phone #?Yes Subsample
AdditionalMailing
NR
88
9
Additional Mailing to Mail Nonrespondents
Response Rate by Treatment
21.3% 22.3%
15.2%
0%
10%
20%
30%
Control Postcard Questionnaire
Treatment
Res
pons
e R
ate
10
Multi-lingual Brochure Test
• Goal: assist non-English speaking households in responding to ACS
• Mail brochure in five languages: English, Spanish, Chinese, Russian, Korean– Dedicated toll-free number for each language
• Tested in April through July 2009 – 1/3 got brochure with pre-notice– 1/3 got brochure with initial questionnaire– 1/3 did not get brochure
1111
12
13
Multi-lingual Brochure Test
• Results imminent
• Study impact on response, specifically
– Response from households that report speaking a language in the brochure
14
FY10 Testing
15
2010 ACS Content Test
• Similar in design to 2006 ACS content test
• OMB Interagency Committee for the ACS provided structure/process for identifying new and revised content
16
2010 ACS Content Test - New Questions
1. Computer Ownership and Internet Access (FCC)
2. Parental Place of Birth (Census)
17
2010 ACS Content Test - Revised Questions
1. Food Stamps (USDA)– Change program name to SNAP
2. Veteran Identification and Period of Service (VA)
3. Public Assistance Income (HHS)
4. (CATI/CAPI only) Wages/Salary/etc. and Interest/Dividends/etc. income (HHS)
18
2010 ACS Content Test – Process
• Subcommittees determine preliminary questions
• Cognitive testing
• Expert Panel
• Field Test
19
2010 ACS Content Test
• Split-panel design
– Control: current questions + version 1 of new questions– Test: revised questions + version 2 of new questions
• Initial sample of 35,000 housing units per panel
– Nonresponse follow-up by telephone, pers. visit– Behavior coding using recordings (CARI)– Content follow-up reinterview
20
2010 ACS Content Test
• Field test in September – November 2010
• Final changes in ACS production in January 2013
21
Plans for Future Testing
2222
2011 ACS Internet Test
• Evaluate feasibility of Internet option for ACS and PRCS
– English and Spanish
• Experimentally test ways of offering the Internet option to maximize Internet response
23
2011 ACS Internet Test
• Treatments:
– Prominent Offer (Choice) – Not Prominent Offer– Push Internet– Push Internet (modified mail schedule)
• Control (production): No Internet offer
24
25
26
27
2011 ACS Internet Test
• Two strata: Targeted and Not Targeted
• Fully factorial design of 10 experimental panels
• Main evaluation measure – response rates
28
2011 ACS Internet Test
• Designed to present stimuli consistent with other ACS data collection modes to minimize potential mode effects
But at the same time…
• Take advantage of the technology to improve data quality to extent possible
29
30
Open-ended question
31
Mark one
32
Check boxes
33
2011 ACS Internet Test
• Content reinterview (by phone) of INT respondents
• Follow-up interview about materials and Internet accessibility
• Test planned for March 2011
• Results expected late 2011
34
ACS Content Reinterview Survey
• Study and monitor response error for ACS questions
– Help identify items for content testing– Provide concrete estimates of response error
• Exploring use of Latent Class analysis
• Timing TBD