www.ipums.org/internation al 1 IPUMS-Eurasia, 2003-2007: IPUMS-Eurasia, 2003-2007: Preserving Eurasian census Preserving Eurasian census microdata, making them useful, microdata, making them useful, and promoting their use and promoting their use * * * * * * Robert McCaa, Steven Ruggles, Matthew Sobek, Robert McCaa, Steven Ruggles, Matthew Sobek, Deborah Levison and Miriam King Deborah Levison and Miriam King University of Minnesota Population Center University of Minnesota Population Center [email protected][email protected]
29
Embed
Www.ipums.org/international1 IPUMS-Eurasia, 2003-2007: Preserving Eurasian census microdata, making them useful, and promoting their use * * * Robert McCaa,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.ipums.org/international 1
IPUMS-Eurasia, 2003-2007: IPUMS-Eurasia, 2003-2007: Preserving Eurasian census Preserving Eurasian census
microdata, making them useful, microdata, making them useful, and promoting their useand promoting their use
* * ** * *Robert McCaa, Steven Ruggles, Matthew Sobek, Robert McCaa, Steven Ruggles, Matthew Sobek,
Deborah Levison and Miriam KingDeborah Levison and Miriam KingUniversity of Minnesota Population CenterUniversity of Minnesota Population Center
10% OK10% OK» License fee: $$$ proportional to sample densityLicense fee: $$$ proportional to sample density» Division of tasks (provisional): equitableDivision of tasks (provisional): equitable» Calendar (provisional): begin in 2003Calendar (provisional): begin in 2003» 1989 sample: OK? Or will a new one be drawn?1989 sample: OK? Or will a new one be drawn?» 1979 and 1970: do any microdata tapes still 1979 and 1970: do any microdata tapes still
exist?exist?
www.ipums.org/international 3
……official statistics that meet the official statistics that meet the
test of practical utility are to be test of practical utility are to be
compiled and made available on an compiled and made available on an
impartial basis by official impartial basis by official
statistical agencies to honor statistical agencies to honor
citizens’ entitlement to public citizens’ entitlement to public
information.information.
-- UN Statistical Commission, 1994-- UN Statistical Commission, 1994Widespread Internet Technology Widespread Internet Technology
diffusion isdiffusion is
“a pre-requisite for the development “a pre-requisite for the development
of civil society based on free access of civil society based on free access
to information through the global to information through the global
Internet.“Internet.“
--President Putin, March 6, 2001 --President Putin, March 6, 2001
» » Easy-to-use web-Easy-to-use web-interfaceinterface» » Highest scientific Highest scientific standardsstandards» » Proven, powerful Proven, powerful integrationintegration» » A quantum leap in usageA quantum leap in usage
Imagine a new statistical Imagine a new statistical productproduct: : scientifically anonymized, scientifically anonymized, integrated census microdata integrated census microdata samples made up of samples made up of unidentifiable individuals...unidentifiable individuals...
» » 1998: 1 country signed1998: 1 country signed» » 1999: 3 countries1999: 3 countries» » 2000: 9 2000: 9 » » 2001: 15 2001: 15 » » 2002: 32; first release, 6 2002: 32; first release, 6 countriescountries
www.ipums.org/international 5
BBeeffoorree
EEuurrooppee??
IIPPUUMMSS--EEUURRAASSIIAA
Eurasia Phase: Eurasia Phase: 2003-20072003-2007Advantages of a Eurasia-phase, Advantages of a Eurasia-phase, beforebefore Europe Europe• Statistical coherence of Statistical coherence of 1989/2000 censuses1989/2000 censuses• Readily organizable Readily organizable • 12 countries, not 4012 countries, not 40• One linguistic standard: One linguistic standard: RussianRussian
Progress on negotiating Progress on negotiating agreementsagreements• Technical OKsTechnical OKs: Belarus, : Belarus, Moldova RepublicMoldova Republic• Negotiating:Negotiating: Armenia, Armenia, Azerbaijan Azerbaijan Republic, Georgia, Republic, Georgia, Kazahkstan, Kyrghz Kazahkstan, Kyrghz Republic, Russia, Tajikistan, Republic, Russia, Tajikistan, Turkmenistan, Ukraine, Turkmenistan, Ukraine, UzbekistanUzbekistan• Not participating: none, as yet.Not participating: none, as yet.
www.ipums.org/international 6
BBEENNEEFFIITTSS
II
PP
UU
MM
SSii
» » Researchers, world-wide:Researchers, world-wide:free, high quality datafree, high quality dataharmonized, harmonized,
comprehensivecomprehensive» » National Statistics National Statistics Institutes: Institutes:
ratioratio payment for license payment for license
fees, expertisefees, expertise » » People:People:
who we arewho we arewhat the future may what the future may
bringbringhow policies might how policies might
improveimprove
www.ipums.org/international 7
IPUMS-International, a global IPUMS-International, a global collaboratory of National collaboratory of National
Statistical/Research Institutes:Statistical/Research Institutes: » 1. 1. InventoriesInventories the world’s census microdata the world’s census microdata» 2. 2. PreservesPreserves endangered microdata and endangered microdata and
documentationdocumentation* * ** * *
» 3. 3. IntegratesIntegrates datasets of selected countries using datasets of selected countries using UNSD, Eurostat and other standardsUNSD, Eurostat and other standards
» 4. 4. AnonymizesAnonymizes census microdata to preserve census microdata to preserve statistical confidentiality, using highest statistical confidentiality, using highest standardsstandards
» 5. 5. DisseminatesDisseminates customized extracts free of customized extracts free of charge (with complete copies on CDs to all charge (with complete copies on CDs to all partners)partners)
» Scope: Scope: Latin AmericanLatin American census microdata, 1960-present
» Work Plan (Work Plan ( funded by National Institutes of funded by National Institutes of Health)Health)» 2222222 2 n l ns n r m nts w t oig ice i g ag ee e i h fficil n sa age cie
» 2002: Obtain funding from U.S. NIH2002: Obtain funding from U.S. NIH» 2003: Develop/translate microdata & metadata 2003: Develop/translate microdata & metadata » 2004: Country expert teams design national 2004: Country expert teams design national
integrationsintegrations» 2005: MPC/expert teams design regional integration2005: MPC/expert teams design regional integration» 2006: MPC integrates microdata and metadata2006: MPC integrates microdata and metadata» 2007: MPC disseminates to bona fide researchers who 2007: MPC disseminates to bona fide researchers who
sign non-disclosure license. sign non-disclosure license. National census/research institutes via CDs/web. National census/research institutes via CDs/web.
» 1. 1. RespectRespect absolute anonymity and confidentiality absolute anonymity and confidentiality» 2. 2. PreservePreserve all original data, except adjustments all original data, except adjustments
to insure privacy (top codes, blurrings, masking, to insure privacy (top codes, blurrings, masking, re-ordering, etc.)re-ordering, etc.)
» 3. 3. HarmonizeHarmonize codes using international codes using international standardsstandardsoccupation: ISCO, HISCO (detailed, general)occupation: ISCO, HISCO (detailed, general)education: ISCED education: ISCED “ “ “ “family: IPUMS, etc. family: IPUMS, etc. “ “ “ “
» 4. 4. EnhanceEnhance with constructed variables with constructed variables
Place of usual residence x x x x x x x x x x x x x x x x x x x x xPlace of birth x x x x x x x x x x x x x x x x x x x . .Duration of residence x x . . . . . . . . x x x . . x x x x . .Place of previous residence x x . . . . . . . . . x x . . . . . . . .Place of residence at a specified date in the past . . x x x x x x x x x . . x x x x x x x x
Household and family structureRelationship to head of household/householder x x x x x x x x x x x x x x x x x x x x x
Demographic and socialSex x x x x x x x x x x x x x x x x x x x x xAge x x x x x x x x x x x x x x x x x x x x xMarital Status x x x x x x x x x x x x x x x x x x x x xCitizenship . . . . x x x x x x x x . . . . x x x . .Religion . . . . . . . . . . x x x x x . . . . . xLanguage . . . . . . . . . . . . x x x . . x x . .National and/or ethnic group . . . x . . . . . x x x . . x x x x x x x
Fertility and mortalityChildren ever born . x x x . . . . . x x x x x x x x x x x xChildren living . x x x . . . . . x x . . . x . . . . x xDate of birth of last child born alive . x . x . . . . . x x . . . x . . . . x xDeaths in the past 12 months . . . . . . . . . . . . . . . . . . . . xMaternal or paternal orphanhood . . . . . . . . . x x . . . . . . . . . .Age, date or duration of first marriage . . . . . . . . . . . . . . . x x x . . .
EducationLiteracy x x x x . . . . . x . x x x x . . . . x xSchool attendance . x x x . . . . . x x . x x x x x x x x xEducational attainment x x x x x x x x x x x x x x x x x x x x xField of education and educational qualification . . . . x x . . . . . . . . x . . . . x x
EconomicsActivity status x x x x x x x x x x x . x x x x x x x x xTime worked x . x . . . . . . . . x x x x x x x x . .Occupation x x . x x x x x x . x x x x x x x x x xIndustry x x . x x x x x x . . x x x x x x x x x xStatus in employment x x x x x x x x x x . x x x x x x x x . .Income . . . . . . . . . . . . x x x x x x x . .Institutional sector of employment . . . . x x x x x . . . . . . . . . . . xPlace of work . . . . x x x x x . . . . . x x x x x . .
International migrationCountry of birth x x x x x x x x x x x x x x x x x x x . .Citizenship . . . . x x x x x . . . . . . x x x x . .Year or period of arrival . . x . . . . . . . . . . . . . x x x . .
DisabilityDisability . . . x x . . . . . . . . . x . x x x . .Cause of disability . . . . x . . . . . . . . . x . . . . . .
Selected Variable Topic Availability, by Country and Census Year
Colombia France Kenya Mexico United States Vietnam
Coding Schemes and Category Availability for Occupation
Colombia France Kenya Mexico United States Vietnam
www.ipums.org/international 20
AANNOONNYYMMIIZZEESS
II
PP
UU
MM
SSii
» » Suppress geographical Suppress geographical detaildetail» » Blur/aggregate sensitive Blur/aggregate sensitive codescodes» » Convert dates to ages (blur Convert dates to ages (blur key vars.) key vars.) » » Swap cases between Swap cases between districtsdistricts
» » Scramble recordsScramble records
Using the highest standards Using the highest standards available:available:
administrative (license), administrative (license), legal, and legal, and
technical (US Census technical (US Census Bureau, Eurostat, & others)Bureau, Eurostat, & others)
www.ipums.org/international 21
‘‘statistical confidentiality’ statistical confidentiality’ shall mean the protection of shall mean the protection of
data related to single data related to single statistical units which are statistical units which are
obtained directly for obtained directly for statistical purposes or statistical purposes or
indirectly from administrative indirectly from administrative or other sources against any or other sources against any
breach of the right to breach of the right to confidentiality. It implies the confidentiality. It implies the prevention of non-statistical prevention of non-statistical
utilization of the data utilization of the data obtained and unlawful obtained and unlawful
(Thorogood, 1999)(Thorogood, 1999)--all used by IPUMS-International--all used by IPUMS-International
» 1. small sample size1. small sample size» 2. limited geographical detail2. limited geographical detail» 3. top and bottom coding of unique categories3. top and bottom coding of unique categories» 4. signed non-disclosure agreement4. signed non-disclosure agreement» 5. 5. prohibit redistribution of datasets to third prohibit redistribution of datasets to third
partiesparties» 6. 6. prohibit attempts to identify individuals or prohibit attempts to identify individuals or
the making any claim to that effectthe making any claim to that effect» 7. 7. require users to provide copies of require users to provide copies of
(Thorogood, 1999)(Thorogood, 1999)--all used by IPUMS--all used by IPUMSii and more and more
» 8. Age (constructed, where necessary)8. Age (constructed, where necessary)» 9. Never identify date of birth9. Never identify date of birth» 10. Never identify place of birth10. Never identify place of birth» 11. Migration: timing and place not identified in 11. Migration: timing and place not identified in
detaildetail» 12. Place of residence identified by major civil 12. Place of residence identified by major civil
» 13. Sensitivity analysis of variables by national 13. Sensitivity analysis of variables by national expertsexperts
» 14. Confidentiality assessment by national experts 14. Confidentiality assessment by national experts
www.ipums.org/international 25
International Monetary Fund’s International Monetary Fund’s General Data Dissemination General Data Dissemination
SystemSystem52 countries with uniform 52 countries with uniform
standardsstandards» All embrace strict standards of statistical All embrace strict standards of statistical
confidentialityconfidentiality» All prohibit disclosure of information which All prohibit disclosure of information which
may identify individuals or entitiesmay identify individuals or entities» And 37 of 52 countries distribute census And 37 of 52 countries distribute census
microdata samplesmicrodata samples» Why not Russia, Armenia, Azerbaijan Why not Russia, Armenia, Azerbaijan
Republic, Belarus, Georgia, Kazakhstan, Republic, Belarus, Georgia, Kazakhstan, Kyrgyz Republic, Moldova Republic, Kyrgyz Republic, Moldova Republic, Tajikistan, Turkmenistan, Ukraine, or Tajikistan, Turkmenistan, Ukraine, or Uzbekistan?Uzbekistan?
»protects privacy and protects privacy and confidentialityconfidentiality»assures proper use; assures proper use; »new sanction: loss of employment.new sanction: loss of employment.
Researcher selects Researcher selects »Countries, Countries, »Censuses,Censuses,»Cases/sub-populations, Cases/sub-populations, »Variables, and Variables, and »Sample densities--makes Sample densities--makes chronological &/or cross-national chronological &/or cross-national research possibleresearch possible
Open architecture software and Open architecture software and mirror sitesmirror sites
Web-based extraction Web-based extraction system system
www.ipums.org/international 27
IPUMS-Eurasia, 2003-2007: IPUMS-Eurasia, 2003-2007: 12 countries, >280 m. people12 countries, >280 m. people
» Scope: Scope: Eurasia Eurasia census microdata, 1989-present» Work Plan (contingent upon funding):Work Plan (contingent upon funding):
» Jan 2003: Sign licensing agreements with official agencies» Nov 2003: Obtain funding from US NIHNov 2003: Obtain funding from US NIH» 2004: Pay licenses/sign contracts to develop/translate 2004: Pay licenses/sign contracts to develop/translate
microdata & metadata microdata & metadata » 2005: Country expert teams design national 2005: Country expert teams design national
integrationsintegrations» 2006: MPC/expert teams design Eurasia integration2006: MPC/expert teams design Eurasia integration» 2007: MPC integrates microdata and metadata2007: MPC integrates microdata and metadata» 2008 and beyond: 2008 and beyond:
MPC disseminates to bona fide researchers who sign MPC disseminates to bona fide researchers who sign non-disclosure license. non-disclosure license. National census/research institutes disseminate via National census/research institutes disseminate via CDs/web.CDs/web.
www.ipums.org/international 28
On a millennial scale, On a millennial scale, censuses and census microdata censuses and census microdata
survive survive for only a short, but significant for only a short, but significant
periodperiod
www.ipums.org/international 29
IPUMS-Eurasia, 2003-2007: IPUMS-Eurasia, 2003-2007: What needs to be done now?What needs to be done now?
10% OK10% OK» License fee: $$$ proportional to sample densityLicense fee: $$$ proportional to sample density» Division of tasks (provisional): equitableDivision of tasks (provisional): equitable» Calendar (provisional): begin in 2003Calendar (provisional): begin in 2003» 1989 sample: OK? Or will a new one be drawn?1989 sample: OK? Or will a new one be drawn?» 1979 and 1970: do any microdata tapes still 1979 and 1970: do any microdata tapes still
exist?exist?
www.ipums.org/international 30
additional information at:additional information at:www.hist.umn.edu/~rmccaa/ipums-www.hist.umn.edu/~rmccaa/ipums-