Seminar on New Frontiers for Statistical Data Collection WP 30 Moving to common survey tools and processes – the ABS experience

Seminar on New Frontiers for Statistical Data Collection

WP 30 Moving to common survey tools and processes – the ABS experience

Jenine Borowik, Adrian Bugg, Brent Ross

Outline• ABS experience – progressive

convergence of data collection tools and methods

• ABS 2017 directions • Risks and issues • Conclusion

2

1960’s- Integration of Economic Statistics

-Integration of Household Statistics

1950’sIntroduction of Sample Surveys

19801950 1960 1965 1970 2010

1970’sFormal User Consultation Introduced

Major ABS Change

1990 2000

1992National Project Centres

1990’sIncreased use of Adm Data

1992Lotus Notes/WDBs

1995Analysis Unit Established-ABS Website-1st Confidentialised unit record file

2002Business Statistics Innovation Program

19641st Mainframe

2011Future Organisation Structure

1968-69Economic Census

ABS 2017

Information Management Transformation

Individual collection methods and systems for

• Household surveys • Business surveys• Population Census• Others• “cottage industries”

Progressive convergence of tools and methods

• CAPI / CATI• Paper and scanning• Establishment of ESDC (100-

400 office based interviewers)• Single field interviewer

workforce (300-900 field based interviewers)

• Administrative Data Acquisition Unit

• Integrated Collection and Dissemination Division

4

ABS experience• Existing purpose built capability• Specialised• Inflexible and vulnerable• High cost to maintain and change• Slow to change• Cycle of leap and legacy

5

Increasing cost & difficulty of

acquiring data

New competitors &

changing expectations

Rapid changes in the

environment

Competition for skilled

resources- low

unemployment

Reducing budget

Riding the big data

wave

Where are we headed?• Over the next 5 years we will

transform the way the ABS manages information from collection to dissemination

• Providing a better evidence base for use by Governments and the community

7

ABS 2017 Program Goals1. Reduce the cost and time of doing business

2. Grow the business through new statistical products and services

3. Deliver the first large scale digital Census (2016) on time, to budget and quality

while delivering Business As Usual. 8

How will we get there?• Through large scale Innovation across the whole ABS• We will:

– radically transform the way we acquire, collate, use, reuse and disseminate statistical information

• By:– industrialising, modernising and reengineering our business processes – reengineering our statistical infrastructure and the way we manage

information– developing capability needed to meet future needs– collaborating with other international NSIs– focussing on a limited number of centralised projects,

including the digital Census for 2016

9

1. Business Process Reengineering

• Change the way we acquire and process data – emphasis on e-collect for Census 2016, business and household surveys, including web and hand held devices

• Increase our access to and use of large administrative datasets

• Integrate and join data from a range of sources

10

2. Reengineering Statistical Infrastructure• New Core Information Management Infrastructure

tools – metadata driven systems based on agreed standards– One ABS data warehouse to store, manage data from

all sources– On-line data that can be queried– Management of statistical workflows– International collaboration to build next generation

statistical tools11

Expected Changes in our statistical cycle

12

Acquire@ABS High Level Overview

BusinessesIndividualsHouseholds Governments

On lineOn site On the go

Receive data

Transform & Load data

Digitise data

Design, Build & Test Collection Instruments, Processes & Systems

Support Providers

Manage Provider Information

Run Collection

Manage Workforce & Workload

Despatch Instruments /

Requests for Info

On the phoneABS

CHANNELS/MODES

PROVIDERS

On paperen masse

(system to system)

Collection Futures for Census, Household & Business Surveys

Population Census

Household Surveys

Business Surveys

Enterprise Solution

Enterprise Ready

Line of business

- utilising coreinfrastructure

Line of business

- silo / legacy

ABS 2017 Strategy

Freeze,Minimise

investment

Minimise -where

enterprise is undesirable

(business case doesn't exist)

Candidate for future

upgrade

ABS 2017 Infrastructure

Functional Areas Level of DevelopmentAddress Register Enterprise Solution

CAI Data Capture Enterprise Solution

Contact Services Enterprise Solution

eForm Application Enterprise Solution

eForm Data Load Enterprise Solution

eForm Platform Enterprise Solution

Event and Operation Management Enterprise ReadyFrame Create, Maintenance and Sample

Select Enterprise Solution

Knowledge Services Enterprise ReadyPaper Forms Data Capture RFT and

implementation Enterprise Solution.

Provider Management Enterprise Ready

Provider Portal Enterprise Solution

Provider Portal Web Services Mix of Enterprise Ready and Enterprise

Solution

Questionnaire Development Enterprise Solution

Spreadsheet Data Capture Enterprise Solution

Workforce Portal Enterprise Solution

Workforce Portal Web ServicesMix of Enterprise Ready and Enterprise

Solution

Workload Management Enterprise Ready

Risks and issues• New stovepipes with standard tools• Future organisational structure – “Operations Division”• Don’t want to build if we don’t need to• Standards versus flexibility• Cultural change and resistance• Achieving the savings• One size may not fit all• Trade offs will need to occur in some business areas• We can’t resolve all problems

17

Thank you

Further information:

– Jenine Borowik [email protected]– Adrian Bugg [email protected]

18