Software Systems for Survey and Census Yudi Agusta Statistics Indonesia (Chief of IT Division Regional Statistics Office of Bali Province) Joint Meeting on Management of Statistical Information Systems (MSIS) – Daejeon, 26-28 April 2010 Statistics Indonesia
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Software Systems for Survey and Census
Yudi AgustaStatistics Indonesia(Chief of IT Division
Regional Statistics Office of Bali Province)
Joint Meeting on Management of Statistical Information Systems (MSIS) – Daejeon, 26-28 April 2010
Statistics Indonesia
Outline
BPS – Statistics IndonesiaThe Needed Software SystemsThe Case of The 2010 Indonesian Population
Census (Sensus Penduduk 2010 – SP2010)Problem Background and AnalysisAvailable and Applicable TechnologiesExpected ImplementationDesign of Software Systems
Some HurdlesFurther Advancement
BPS – Statistics Indonesia Vision: BPS is the Agent of Trustworthy Statistical Data for All Mission:
To strengthen the constitutional and operational base of the institution for carrying out effective and efficient statistical activities.
To deliver competent and professional statistic human resources, supported by the use of cutting edge information technology for the improvement of Indonesian statistics.
To improve the implementation of the universal standard for classification, concept and definition, estimation, and statistics ethical code in every statistical activity.
To improve the quality of statistical information services for all parties. To improve coordination, integration, and synchronization of statistical activities
conducted by government and private sectors, within the frame of effective and efficient National Statistics Systems.
Economic Growth Inflation Employment Poverty Farmer’s Terms of Trade Rice & Crops Production Export and Import
Setup of Census/Survey Methodology The Setup of Census Block, Sampling, Questionnaire
Design, and Variable Definition Data Collection
Census, Survey, Compilation of Administrative Records, Special Studies
Data Processing Data Dissemination and Analysis
Main Activities Output
The Needed Software Systems
Master File & Map Update & Browsing
Systems
Master File & Map Update & Browsing
Systems
Classification Standard Browsing Systems
Classification Standard Browsing Systems
E-Survey Systems
E-Survey Systems
Concept and Definition
Update and Browsing Systems
Concept and Definition
Update and Browsing Systems
CAPI Systems
CAPI Systems
CATI Systems
CATI Systems
Data Entry Systems
Data Entry Systems
Reporting Systems
Reporting Systems
Documents Handling Systems
Documents Handling Systems
OCR/OMR Data
Capturing Systems
OCR/OMR Data
Capturing Systems
Data Validation Systems
Data Validation Systems
Data Imputation Systems
Data Imputation Systems
Automatic Editing
Systems
Automatic Editing
Systems
Managerial Scheduling Systems
Managerial Scheduling Systems
Admin Communication Systems
Admin Communication Systems
Data Exploring Systems
Data Exploring Systems
Data Warehouse
Systems
Data Warehouse
Systems
Metadata Update and Browsing Systems
Metadata Update and Browsing Systems
Data Mapping Systems
Data Mapping Systems
Data Center Systems
Data Center Systems
Digital Map Update
Systems
Digital Map Update
Systems
Data Dissemina
tion Systems
Data Dissemina
tion Systems
Sampling, Weighting & Estimation Systems
Sampling, Weighting & Estimation Systems
The Needed Software Systems
Master File & Map Update & Browsing
Systems
Master File & Map Update & Browsing
Systems
Classification Standard Browsing Systems
Classification Standard Browsing Systems
E-Survey Systems
E-Survey Systems
Concept and Definition
Update and Browsing Systems
Concept and Definition
Update and Browsing Systems
CAPI Systems
CAPI Systems
CATI Systems
CATI Systems
Data Entry Systems
Data Entry Systems
Reporting Systems
Reporting Systems
Documents Handling Systems
Documents Handling Systems
OCR/OMR Data
Capturing Systems
OCR/OMR Data
Capturing Systems
Data Validation Systems
Data Validation Systems
Data Imputation Systems
Data Imputation Systems
Automatic Editing
Systems
Automatic Editing
Systems
Managerial Scheduling Systems
Managerial Scheduling Systems
Admin Communication Systems
Admin Communication Systems
Data Exploring Systems
Data Exploring Systems
Data Warehouse
Systems
Data Warehouse
Systems
Metadata Update and Browsing Systems
Metadata Update and Browsing Systems
Data Mapping Systems
Data Mapping Systems
Data Center Systems
Data Center Systems
Digital Map Update
Systems
Digital Map Update
Systems
Data Dissemina
tion Systems
Data Dissemina
tion Systems
Sampling, Weighting & Estimation Systems
Sampling, Weighting & Estimation Systems
The Needed Software Systems
Master File & Map Update & Browsing
Systems
Master File & Map Update & Browsing
Systems
Classification Standard Browsing Systems
Classification Standard Browsing Systems
E-Survey Systems
E-Survey Systems
Concept and Definition
Update and Browsing Systems
Concept and Definition
Update and Browsing Systems
CAPI Systems
CAPI Systems
CATI Systems
CATI Systems
Data Entry Systems
Data Entry Systems
Reporting Systems
Reporting Systems
Documents Handling Systems
Documents Handling Systems
OCR/OMR Data
Capturing Systems
OCR/OMR Data
Capturing Systems
Data Validation Systems
Data Validation Systems
Data Imputation Systems
Data Imputation Systems
Automatic Editing
Systems
Automatic Editing
Systems
Managerial Scheduling Systems
Managerial Scheduling Systems
Admin Communication Systems
Admin Communication Systems
Data Exploring Systems
Data Exploring Systems
Data Warehouse
Systems
Data Warehouse
Systems
Metadata Update and Browsing Systems
Metadata Update and Browsing Systems
Data Mapping Systems
Data Mapping Systems
Data Center Systems
Data Center Systems
Digital Map Update
Systems
Digital Map Update
Systems
Data Dissemina
tion Systems
Data Dissemina
tion Systems
Sampling, Weighting & Estimation Systems
Sampling, Weighting & Estimation Systems
1 2
3
4 5
The Case of
The 2010 Indonesian Population Census
(Sensus Penduduk 2010 – SP2010)
Problem Background
Communication Lag Question and Answer Suggestion and Confirmation Updated News & Files, Reporting
Miscommunication Unclear Information Misinterpretation False in Filing
Clarity of Process & Instruction Systems No Clear Procedures No Documentations
Document Handling Problem Bad Documentation
Lack of Accuracy Imprecise Implementation of
Validation Rules Lack of Post Processing Analysis
Uses of Census Data Lack of Preparation in Utilizing
Census Data Lack of Automated Tools in
Exploring Census Data
Extensive Changes/Consolidation In Terms of Concept and Definition, Scope, and Uses of Variables During The Conduct of Population Census
Occurring Errors in Data Collection - the Enumerators and Respondents Point of Views Very Large Scope of Processes Needed to be Supervised The Need of Faster, Accurate, and Reliable Data Dissemination Process
Problem Analysis – In Relation To The Possible Support of IT
Available & Applicable TechnologiesExtraction
Transformation Loading
Online Analytical
Processing
Implementation of Online Interaction System With Various Functions Online Enumeration Q&A Online Data Processing Q&A Online Validation Rule Suggestion and Confirmation/Approval Online Reporting System
Implementation of Document Handling System Record Keeping of Census/Survey Documents
Implementation of Post Processing Utility System Important Variables Analysis Double Counting Analysis Record Outlier Analysis
Implementation of Data Exploring System Comparison with Previous Data Comparison Between Area Online Analytical Processing (OLAP)
Information Systems
Expected Implementation
Intelligent Systems
Data Mining
Rule Base System
Optical Mark
Recognition
Optical Character
Recognition
Web-Based Online System
Design of Software Systems
Online Enumeration Q&A
Online Data Processing Q&A
Online Validation Rule Suggestion and Confirmation/Approval
Online Enumeration Reporting System
Online Data Processing Reporting System
Online Patch and Other Supporting Files Upload and Download
Online Interaction System
P0
OnlineInteraction System
a
HQEnumeration Admin
c
RegionalEnumeration Admin
b
HQ DataProcessing Admin
d
Reg. DataProcessing Admin
Enum_Question_Display
Enum_Answer_Display
Enumeration_Answer
Val_Rule_Suggested_Display
Val_Rule_Evaluated_TagVal_Rule_Implemented_Tag
Enum_Report_Display
Val_Rule_Evaluated_DisplayVal_Rule_Implemented_Display
Enum_Report_DisplayEnum_Report_Form
Enum_QuestionEnum_Report_File
Data_Proc_Question_DisplayVal_Rule_Implemented_Display
Enum_Report_Form
Program_PatchUpdated_Val_Rule_File
Data_Proc_Answer
Data_Proc_Report_Display
Data_Proc_QuestionVal_Rule_Suggestion
Data_Proc_Report_File
Val_Rule_Suggestion
Program_PatchUpdated_Val_Rule_File
Data_Proc_Report_DisplayVal_Rule_Implemented_DisplayVal_Rule_Evaluated_Display
Data_Proc_Answer_Display
Data_Proc_Analysis_Blank_File
Data_Proc_Analysis_Blank_FileData_Proc_Analysis_Filled_File
Data_Proc_Analysis_Filled_File
Design of Software Systems
P0
SP2010Document Handling System
c
InternalDoc Storage Admin
a
ExternalDoc Storage Admin
Docs_To_External_Storage_Info
Cut_Docs_Info
List_Released_Docs
Scanned_Docs_Info
List_Docs_To_Eksternal_Storage
List_Docs_To_Internal
Docs_To_Internal_Storage
List_Received_Batched_Docs
Received_Batched_Docs_InfoEdited_Docs_Info
Record Keeping of Documents in Each Process (Receiving Batching, Editing, Cutting, Scanning, Correction, Completion, Validation, and Releasing)
Documents Handling System
Design of Software Systems: SP2010
Scanning SystemRecognition SystemCorrection & Completion
SystemReleasing SystemValidation SystemSystem UpdateValidation Rules Update
P0
SP2010Data Processing System
a
ScanningRe-Scanning Admin
b
RecognizingAdmin
c
CorrectionCompletion Admin
d
ReleasingAdmin
e
ValidationAdmin
f
Reg DataProcessing Admin
SP2010_Documents Recognizing_Command
Re-Scanning_Command
Recognized_Records
Scanning_Command Recognizing_Parameters
Scanning_Parameters
Corrected_Completed_RecordsNon-Completed_Records
Corrected_Completed_Records
List_Released_Documents
Non-Completed_RecordsReleasing_Tags
Validated_Records
Scanned_Images
Program_Patch
Data_Proc_Report_FileReport_Display
Updated_Val_Rules
Data Processing System
Scanning PC
Correction and Completion PCs
CutterRecognition
PCDocuments
Storage Terminal
Releasing PC
Validation PCs
Enumeration Admin
Data Processing Admin
Proxy Server
Data Server
SP2010 Data Processing Network Systems Scanning & Recognition Admin
Correction & Completion Admin
Correction and Completion Staffs
Releasing & Validation Admin
Validation Staffs & System Admin
Users Involved
Design of Software Systems: SP2010
Double Counting Analysis
Record Outliers Analysis
Name and Address Analysis
Age AnalysisName and Sex
Analysis
Post Processing Utility System P0
SP2010Post Processing Utility
a
Reg. DataProcessing Admin
Corrected_Sex
Record_Outlier_Analysis
Corrected_Double_Counting_Record
Corrected Name
Utility_Program_Patch
Corrected_Address
Address_Analysis_CorrectionName_Analysis_Correction
Sex_Analysis_Correction
Corrected_Record
Age_Analysis_Correction
Double_Counting_Analysis
Corrected_Age
Design of Software Systems
Area Analysis
Time Series Analysis
Online Analytical Processing (OLAP)
Data Exploring System
P1
DataExploring System
a
Reg. DataProcessing Admin
b
RegionalEnumeration Admin
Area_Analysis_Report
Previous_Data_File_Input
OLAP_Table
OLAP_Table
Area_Analysis_ReportTime_Series_Analysis_Command
OLAP_Command
Time_Series_Analysis_Report
Area_Analysis_Command
Time_Series_Analysis_ReportArea_Analysis_Report
ETL_CommandOLAP_Command
Time_Series_Analysis_Command
Some Hurdles
Information Technology Blue PrintThere is still no general information technology
blue print within the organisationExpected by the year of 2012, there will be one
In-House or OutsourcingCommon practice: in-house development
Lack of standard, consistency, experience, and effectiveness
Outsourcing? Lack of Funding
Further Advancement
Managerial Support SystemsManagerial Scheduling Systems
Data Dissemination Supporting SystemsData Center SystemsData Warehouse SystemsMetadata Systems
A More General Systems for Handling Different Types of Surveys and CensusesGeneral DesignUniversal Development Engine
Thank You
Joint Meeting on Management of Statistical Information Systems (MSIS) – Daejeon, 26-28 April 2010
Statistics Indonesia
Related Documents
