-
Backstage of Population Census in Italy:
fieldwork and paradata to manage data
quality collection
The role of Istat in Tuscany
Bianca Maria Martelli, Linda Porciani, Alessandro Valentini
5th Conference of European Survey Research Association
Ljubljana, Slovenia, July 15-19, 2013
-
Summary
� 15th Population and Housing Census in Italy
� Territorial organization
� Managing data quality in the case of Tuscany
�Theoretical approach
� Application� Application
• Field work
• Paradata indicators
• Post census ad hoc survey
� Discussion
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
-
Main innovations of Census
� From a totally door-to-door to assisted census (by LAU2
Population Register)
� Use of other administrative registers to reduce
under-coverage
(fiscal registers, ‘permits-to-stay’ , etc…)
� Mail-out of census forms� Mail-out of census forms
� Multi-way modalities to answer the questionnaires (paper
and
web based)
� Sample strategies in order to reduce response burden
(short
and long forms)
� Supervision of the process via on line system SGR (Survey
Recording System) SGR
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
1
-
NUTS0 - Italy NUTS2 - Region
- ISTAT
- Statistical Office
of the Italian
Ministry of
Interior
Regional Offices
of Census (URCs)
Territorial organization
Responsibility to manage the data collection process
in its own territory
Responsibility to manage the data collection process
in its own territory
NUTS3 - Provinces LAU2 - Municipalities
Statistical Offices
of Prefectures
(UPCs)
Municipal Offices
of census (UCCs)
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
1
-
Territorial organization: a focus on Tuscany
Tuscany:
Regional Office of
Istat URC
10 Provinces UPCs 287 Municipalities
(212 UCCs)
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
2
-
Managing Data Quality (DQ) in the case of Tuscany
Top-down approach
Researcher selects
DQ attributes
According Wang and Strong (1996), the methodological
framework adopted is the following
Field workIntuitive
Top-down approach
How data quality
weakens during
data collection process
Bottom up approach
Investigation on
actors’ opinions
about DQ attributes
Paradata tool
Ad hoc survey
on census
operators
Theoretical
Empirical
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
3
-
Definition of a matrix quality measures/actions developed by
URC
Intuitive approach: field work
Quality measures
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
4
-
Field-work actions
Training
Scheduled: between July and October 2011[Census day: October,
9th 2011]
Subjects: UPCs, UCCs, Enumerators
(around 5,000 persons)
SupportingIn-distance: by phone (91%) and by e-mail (9%)
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
5
SupportingIn-distance: by phone (91%) and by e-mail (9%)
In-presence: additional training sessions
MonitoringInspections: checks to UCCs by URC staff
Interventions: on critical cases
-
Response rate by nationalities
Field-work actions: an intervention
The “elusive population” of foreigners in Prato[15% of the total
population in 2011]
1. Official actions promoted by local
institutions (meeting URC, UCC, head of local community);
2. mass media communication (press conference, internet pages by
chineselanguage)
1. Official actions promoted by local
institutions (meeting URC, UCC, head of local community);
2. mass media communication (press conference, internet pages by
chineselanguage)
1. Informal actions (direct mailing from UCC)
1. Informal actions (direct mailing from UCC)
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
6
-
Theoretical approach: paradata tool
1. Dashboard indicators
Real time indicators -> to realize an
immediate set of interventions to
solve general or local criticisms
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
7
2. Roadmap indicators
Ex post indicators -> to improve DQ
in incoming surveys
(reengineering of the process)
-
Paradata tool 1.Dashboard indicators
Indicator Description Target
value (%)
QUES.LAC Percentage ratio between number of questionnaires
filled in and number of questionnaires in the population registry
lists (LAC) at 31.12.2010
≥ 97 ≤ 103
QUES.CA Percentage ratio between number of questionnaires
processed and number of questionnaires enabled to the comparison
(CA) between census and population registers.
100
Dashboard indicators consist of 6 indicators derived from SGR
data
population registers.
DIARIO.LIFA Percentage ratio between processed rows and total
rows of potential undercoverage derived from administrative
archives (LIFA)
100
DIARIO.NETTO Percentage ratio between rows processed and total
number of rows excluding (NETTO) rows of potential undercover
derived from LIFA
100
CFR01.EDI Percentage ratio between number of surveyed building
(EDI) and number of surveyed buildings in 2001 census
≥ 100 ≤ 150
CFR01.ABI Percentage ratio between number of surveyed unoccupied
dwellings (ABI) and number of surveyed unoccupied dwellings in 2001
census
≥ 85 ≤ 115
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
8
-
Indicator DIARIO.LIFA
(tested undercoverage)
An application to LAU2
0 – 20,000 inhabitants
Paradata tool 1.Dashboard indicators
Problem
Indicator CFR01.ABI (unoccupied dwellings)(tested
undercoverage)
Problem
Localized criticism
Action
Targeted intervention on specific LAU2
Problem
Widespread criticism due to lacking information
Action
Additional training phase to all network census actors
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
9
-
Paradata tool 2. Roadmap indicators
97.9
94.9
97.6
95.095
96
97
98
99
100
%
Italy
Tuscany
33.431.7
22.624.5
31.7 31.4
20
25
30
35
40
%
Italy
Tuscany
Standardization process and rate of postal
delivery (%). Italy and TuscanyQuestionnaires by channel of
answer (%).
Italy and Tuscany
90
91
92
93
94
95
Standardization
process
Rate of postal
delivery
%
12.3 12.4
0
5
10
15
20
Web Municipal data
collection
centers
Post offices Enumerators
NO need of specific territorial intervention Need of deeper
analysis
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
10
-
∑=
+=4
1 i
o (WR)logit ii xββ
Paradata tool 2. Roadmap indicators
Where:
WR = Quota of web respondent (by Municipality)
X1 = Number of members per households
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
11
X1 = Number of members per households
[effect of class >2.29; ref. category ≤2.29]
X2 = Quota of foreigners
[effect of class >0.0745; ref. category ≤0.0745]
X3 = Ageing index
[effect of class >200.3; ref. category ≤200.3]
X4 = Demographic dimension of Municipality
[effect of class > 5,573 ; ref. category ≤5,573]
-
Paradata tool 2. Roadmap indicators
+- 0
Demographic
dimension
Quota of
foreignersAgeing indexHousehold
dimension
Results:
Main insights:
1. Use of web is linked to demographic characteristics of
population;
2. To increase the quota of web users it is necessary to support
specific
subsets of population: ‘traditional families’ and ‘old
population’;
3. A particular care has to be dedicated to small LAU2, with
more traditional
habits
dimensionforeignersdimension
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
12
-
Empirical approach: ad hoc survey on operators
At the end of Census operations an ad hoc survey (Ivalcens,
‘inquiry to evaluate the census’) was submitted to qualified census
operators
The survey was realized via a web technique. The rate of
response was 94.6% for national level (100% in Tuscany)
Main results of Ivalcens
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
13
Main results of Ivalcens
� Scarce interest for the internet channel (one of the lowest
evaluation in Italy)� Criticisms about data collection of
unoccupied dwellings (problem already emerged during the field work
and paradata analysis);� Low appreciation of mail-out (despite the
good quality of road-map indicators)� Limited appreciation of the
(institutional) context
-
Discussion
The key factor for DQ should be represented by a complete
integration between the central organism of planning and the
peripheral offices devoted to supervise the application of rules
(URCs in our case).
From our experience, the set of instruments introduced is a From
our experience, the set of instruments introduced is a valid tool
to:� easily point out the criticalities through simple indicators�
prevent bad practices in data collection and help best ones
� monitor the field work and, finally� assess data and process
quality
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
14
-
Thanks for your attention!
Bianca Maria Martelli, [email protected]
Linda Porciani, [email protected]
Alessandro Valentini, [email protected]
Backstage of Population Census in Italy: fieldwork and paradata
to manage data quality collection. The role of Istat in Tuscany
B. M. Martelli, L. Porciani, A. Valentini – Ljubljana, Slovenia,
July 15-19, 2013
15