IPUMS workshop * * * Robert McCaa, Professor of Population History University of Minnesota [email protected] additional information.

IPUMS workshopIPUMS workshophttps://international.ipums.org

* * ** * *Robert McCaa, Professor of Population HistoryRobert McCaa, Professor of Population History

University of MinnesotaUniversity of [email protected]

additional information at:additional information at:www.hist.umn.edu/~rmccaa/ipums-europewww.hist.umn.edu/~rmccaa/ipums-europe

The IPUMS team The IPUMS team (lack computer gurus, some researchers, & 3 PIs were away or too busy to pose!)(lack computer gurus, some researchers, & 3 PIs were away or too busy to pose!)

Steven Ruggles, Inventor of IPUMS, Professor of History and Director, Minnesota Population Center

IPUMS-Greece TimelineIPUMS-Greece Timeline

• May, 2003: Memorandum of Understanding signedMay, 2003: Memorandum of Understanding signed• July, 2005: Microdata samples entrusted for censuses July, 2005: Microdata samples entrusted for censuses

of 1971, 1981, 1991 and 2001of 1971, 1981, 1991 and 2001• April, 2006: Translations of documentation completedApril, 2006: Translations of documentation completed• Dec., 2006: Integration completed; dissemination Dec., 2006: Integration completed; dissemination

beginsbegins

Outline (65 slides, yikes!!)Outline (65 slides, yikes!!)

» 1. IPUMS goals1. IPUMS goals and milestones and milestones 11 slides 11 slides

» 2. Applying for Access2. Applying for Access 8 slides8 slides

» 3. Studying documentation3. Studying documentation 15 slides 15 slides

» 4. Creating an extract4. Creating an extract 9 slides9 slides» a. Selecting samplesa. Selecting samples

» b. Selecting variablesb. Selecting variables

» c. Selecting sub-populations c. Selecting sub-populations

» 5. Integrating microdata5. Integrating microdata 9 slides9 slides

» 6. Managing access: Users and Uses6. Managing access: Users and Uses 13 slides 13 slides

Project goalsProject goals

• IPUMS is a global partnership to IPUMS is a global partnership to (1) preserve census microdata and documentation,(1) preserve census microdata and documentation,(2) integrate census microdata samples, and(2) integrate census microdata samples, and(3) manage access to anonymized sample extracts for (3) manage access to anonymized sample extracts for researchers and policy makers, at no cost—regardless researchers and policy makers, at no cost—regardless of country of birth, residence or citizenshipof country of birth, residence or citizenship

Preservation: 1973 census tapes of Sudan at risk!

A Solution: Data recovery

Data recovery. Example: Bangladesh Bureau of Data recovery. Example: Bangladesh Bureau of Statistics (1981 census, 276 tapes to recover)Statistics (1981 census, 276 tapes to recover)

>3,000 tapes >3,000 tapes recovered: 1971 Germanyrecovered: 1971 Germany

1980 Mexico, 1980 Mexico, Mali 76, Sudan 73Mali 76, Sudan 73

and many moreand many more

MicrodataMicrodataon this tape on this tape

were recovered!!were recovered!!

Integration, Dissemination: March 2008Integration, Dissemination: March 2008

dark greendark green = = disseminating (26 countries, 80 disseminating (26 countries, 80 censuses, 200mpr)censuses, 200mpr)

green = harmonizing (34 countries, 95 censuses, green = harmonizing (34 countries, 95 censuses, 190mpr)190mpr)

lightest greenlightest green = negotiating = negotiating (see handout)(see handout)

Mollweide projection

MilestonesMilestones

» 1999: Founded by Steven Ruggles and Bob McCaa,1999: Founded by Steven Ruggles and Bob McCaa,––restrict access to trusted users, and apply corresponding restrict access to trusted users, and apply corresponding confidentiality techniquesconfidentiality techniques» 2002: 12002: 1stst release of integrated samples for 7 countries; >200 release of integrated samples for 7 countries; >200

users in first yearusers in first year

» 2008: Big hit! 79 countries signed; 70 entrusted data to 2008: Big hit! 79 countries signed; 70 entrusted data to IPUMS, datasets for more than 230 censuses, >150 entire IPUMS, datasets for more than 230 censuses, >150 entire datasetsdatasets

MilestonesMilestones

» 2007, 42007, 4thth release: release: » data for 26 countries, samples for 80 censuses, data for 26 countries, samples for 80 censuses,

» 202 million person records, 202 million person records,

» ~2,000 users~2,000 users

» 2009, 62009, 6thth release: release: » data for 40 countries, samples for ~130 censusesdata for 40 countries, samples for ~130 censuses

» >300 million person records>300 million person records

» thousands of usersthousands of users

» Note: data extracts are provided only to licensed users.Note: data extracts are provided only to licensed users.

2. 2. UsingUsing https://www.ipums.org/international:https://www.ipums.org/international:

1. Logon 1. Logon w/ passwordw/ password

2a. Study documentation2a. Study documentation2b. Design extract2b. Design extract

3. Receive email; 3. Receive email; logon with p/wordlogon with p/word

4. Download 4. Download extract (SSL extract (SSL encrypted)encrypted)

5. UnZip data5. UnZip data

(also SAS, (also SAS, STATA) STATA)

6. Analyze6. Analyze

- Warning -- Warning -

• IPUMS microdata are anonymized samples.IPUMS microdata are anonymized samples.– They are for advanced analysis and research. They are for advanced analysis and research. – Use of a statistical software is required.Use of a statistical software is required.– Statistical software provides great power.Statistical software provides great power.– “ “With great power, comes great responsibility.”With great power, comes great responsibility.”

• IPUMS samples are for analysis.IPUMS samples are for analysis.• IPUMS samples are IPUMS samples are not not official statistics.official statistics.

http://international.ipums.orghttp://international.ipums.org

Apply for access (see form and conditions of use)Apply for access (see form and conditions of use)

Study integrated documentation (variables)Study integrated documentation (variables)

Link to Official Statistical Agency home pagesLink to Official Statistical Agency home pages

Examine integrated metadata (samples)Examine integrated metadata (samples)

Construct a custom-tailored request: select Construct a custom-tailored request: select countries, years, sub-populations, & variablescountries, years, sub-populations, & variables

1.1. Respect “restricted-access” conditions of use: protect Respect “restricted-access” conditions of use: protect confidentiality; “share” data with only with registered usersconfidentiality; “share” data with only with registered users

2.2. Study both source documentation and metadata: Study both source documentation and metadata:

» source: census forms and instructions to enumerators source: census forms and instructions to enumerators

» metadata: samples, variables, comparability discussionsmetadata: samples, variables, comparability discussions

3.3. Construct extracts judiciously:Construct extracts judiciously:use “subsamp” (1% sample for testing) use “subsamp” (1% sample for testing) extract only needed countries, censuses, variables, sub-pops extract only needed countries, censuses, variables, sub-pops

4.4. Use weights:Use weights:either households or individuals (geographical strata = power)either households or individuals (geographical strata = power)

5.5. Analyze carefully:Analyze carefully:proper statistical techniques, keeping in mind data qualityproper statistical techniques, keeping in mind data quality

6.6. Cite properly: IPUMSCite properly: IPUMSii and National Statistical Agencies and National Statistical Agencies

7.7. Share publications: IPUMSShare publications: IPUMSii and National Statistical Agenciesand National Statistical Agencies

Dr. Bob’s 7 rules for using IPUMSDr. Bob’s 7 rules for using IPUMSii microdata microdata