Data Management & Data lifecycle • Survey Conception • Data System Architecture • Data collection management • Data Analysis & Dissemination
Data Management & Data lifecycle
• Survey Conception• Data System Architecture• Data collection
management• Data Analysis &
Dissemination
Data Management & Data Lifecycle
Type of info per usage
Global reports
Project report
Outreach material
Indicators (Focus)
Registration X X X X
IDP profiling X X X XProtection Incident monitoring X X X XProtection situation monitoring X X X XPopulation movement monitoring X X X X
Sectoral assessment X XPartners activities monitoring X XAbsorbtion capacity evaluation X X X
Introduction
Data Management & Data Lifecycle
From Data.. to InformationIntroduction
Operation Data Manager should be involved in all the steps of a “Data Lifecycle”.
SurveyConception
DataSystem
Architecture
DataCollection
management
Data Analysis &
Dissemination
Any break of this cycle ends with the failure of the system :• A data collection form that is ill-
designed either because it does not satisfy operational information requirements or is flawed from a technical standpoint
• A well designed survey with a poorly designed and therefore poorly maintained database
• A structurally well designed database with no data, as data collection cycles have not been integrated/respected
• A well populated database without implemented reports and queries and therefore no output
Data Management & Data Lifecycle
Before the Form…Survey Conception
• Avoid reinventing the wheel – check what has been designed and piloted before
• Consultation with all stakeholders – avoid duplication of efforts and assessment fatigue of beneficiaries
• Layers of data collection• Collect Simple base reference data first
• Embark on detailed info based on samples defined from the base reference
• Data collection frequency should vary according to how frequently the phenomena being tracked or measured changes
Data Management & Data Lifecycle
Good practices for Data Collection Forms
1. Questionnaires used in survey research should be clear and well presented.
2. Think about the form of the questions, 3. Keep the survey as short as possible.4. Make definitions of data elements
consistent with standard definitions and analytic conventions
5. Plan clearly how answers will be analyzed.
6. Test the survey for “understandability” and respondent effort through focus groups
Survey Conception
Data Management & Data Lifecycle
Data model
• Data models are the key for interoperability (i.e easy data exchange with partners)
• Implementing partners should not have to draft and decide on a core data model; it should be the same everywhere and just adapted locally where necessary; support (guidelines) need to be there
• Importance of a common referential
Data System Architecture
Beneficiaryregistration
Site / community
Assessment
Activity monitoring
Site
Who’s doing what where?
Multi sectoral assessment:-Health-Education-Water
Bio DataVulnerabilityNeeds
Demographics
Project activities descriptionPerformance Indicators
Base indicatorsDelivered Assistance
Infrastructure InventoryOrganization
Data Management & Data Lifecycle
System architectureData System Architecture
• Building an Interface for data collection:• Mobile • Offline desktop• Web/Server based• OCR* ready form (can be scanned)
• Integration of external data source (ETL**)• Offering analysis capacity (OLAP*** and
Stats)* Mechanical or electronic translation of scanned images of handwritten, typewritten or printed text into machine-encoded text
** Extract, transform, and load (ETL) is a process in database usage that involves Extracting data from outside sources, Transforming it to fit operational needs (which can include quality levels), Loading it into the end target (database or data warehouse)*** An OLAP (Online analytical processing) cube is a data structure that allows fast analysis of data.
Data Management & Data Lifecycle
Reports are part of the data system
Queries and tools to extract data from the databases need to be designed along with the database
Must give abilities for reporting officers to
- Set up queries and reports without high level IT knowledge
- To be clear on the standard indicators these queries should be based on
Data System Architecture
Data Management & Data Lifecycle
Data collection strategies
• Direct coordination with partners•ex : Somali protection cluster
• Establishment of a « data collection project »•ex : UNOPS Goma
• Specific Contract with a dedicated partner•Ex: CartONG in Uganda
Data collection management
Data Management & Data Lifecycle
Implementation matrix
UNHCRDirect Government
Implementing partner Project
Dedicated partner
Registration X X
IDP profiling X X XProtection Incident monitoring X XProtection situation monitoring X XPopulation movement monitoring X X X
Sectoral assessment X XPartners activities monitoring XAbsorbtion capacity evaluation X X X
Avoid conflict of interest
Data collection management
Data Management & Data Lifecycle
PDF reports and maps
• Targets mostly local partners and decision makers
• Can be disseminated through• mailing list (cf Somali protection)• Google group (cf Goma Update)• Website (cf ReliefWeb)
Data Dissemination
Data Management & Data Lifecycle
GeoPortal and Open Data API
GeoPortal: • is a tool to ensure institutional memory
and “Master Data” management• Can be a tool for desk officers to visualize
a situation and use map extracts in their reporting
Data API:• Can be used for global dissemination: cf
Worldbank Data API or Google public data
• Offers material for data journalism (e.g. computer assisted reporting on data through journalists)
Data Dissemination
Data Management & Data Lifecycle
Data, Law & License
For all data sets that do not fall under the “Guidelines for the Regulation of Computerized Personal Data Files” (for instance protection data) ….
…. The “Open database license” (ODBL) can give a legal frame to all our data collection activities
http://www.opendatacommons.org/licenses/odbl/1.0
Data Dissemination
http://www.unhcr.org/refworld/pdfid/3ddcafaac.pdf
Data Management & Data Lifecycle
Providing support for the 4 phases of the process
Conclusion
4 specific types of expertise that are difficult to combine in one profile:
•Statistician/Analyst: Creating a questionnaire and compiling analyzing the resulting statistics
• IS Architect: Building the information system
• Manager: Managing the stakeholder consultation process during the design phase, the collection in the field and dissemination of results
• Data journalist: Developing sound and sexy reports
Need to find where are the gap among the “Operation Data Management” officers network
Need to define the training & support need for each of those specific domains