Assessing the quality impacts Assessing the quality impacts of posting Census of posting Census Questionnaires Questionnaires Garnett Compton Garnett Compton Q2008 Conference, 8-11 July Q2008 Conference, 8-11 July 2008 2008
Feb 25, 2016
Assessing the quality impacts of Assessing the quality impacts of posting Census Questionnairesposting Census Questionnaires
Garnett ComptonGarnett ComptonQ2008 Conference, 8-11 July 2008Q2008 Conference, 8-11 July 2008
1. Setting the scene – Post-out and the 2007 Census Test
2. Differences in response
3. Quality of the address register
4. Costs
5. Other quality and operational impacts
6. Conclusions and questions
Overview
Why consider Post-out?
• To reduce serious risks experienced in 2001, in particular the failure to recruit a large number of enumerators.
• To provide savings to invest in improving response.
• Because of the limited success of making contact at delivery.
Household response rates by delivery method and ETC
ETC Hand delivery
Post-out Difference P value
Total 53.4% 50.6% 2.8% <1%
1 66.9% 63.4% 3.6% 1%
2 55.7% 51.2% 4.5% 4%
3 47.8% 44.7% 3.1% 9%
4 36.8% 37.0% -0.2% 54%
5 33.8% 29.3% 4.5% 1%
Success rates at follow-up by delivery method by ETC
ETC Hand delivery
Post-out
1 37.1% 35.4%2 27.0% 26.9%3 23.6% 22.2%4 16.5% 17.6%5 14.9% 13.6%
Overall 26.0% 25.8%
Address register coverage
• 680 (1.3%) new addresses found during enumeration in hand delivery areas
– Nearly 70% of new addresses were sub-premise addresses – suggest existed at time of AC.
– About 20% found already existing/available latest update.
– About 1/6 found in hand delivery areas during follow-up
Estimated Cost Savings
Estimated cost savings between 100% post-out and 100% hand delivery
Initial return* rate difference
(%’age points)
Estimated savings
5 £28m - £35m
6 £25m – £33m
10 £6m - £21m
15 -£18m - £1m
* At the start of follow-up – 23 May
Quality and operational impacts
• No difference in under/over count between two methods
• No difference in number failing 2 of 4 rule• No large difference in age/sex distributions
between delivery methods• 50% more calls to the contact centre in post-
out areas• CTES:
– No difference in views on “junk” mail– Small difference recognised as “official” mail
Conclusions• Post-out has an impact on return rates, but not
success at follow-up. Post-out requires more follow-up to obtain same overall response rate.
• Differences in return rates are not affected by the hard to count characteristics of an area (i.e. the ETC).
• No significant differences in response quality• A post-out methodology will allow savings to invest in
targeted follow-up and community liaison.• The levels of AR undercoverage will be small with
minimal, but manageable, impact on the overall quality.
• Some operational impacts but manageable through design and development
Thank you
Any Questions?