Integration of Annual Economic Collections – The Australian Experience ICESIII, Canada, 2007 Presented by Eden Brinkley.

Integration of Annual Economic Collections – The Australian

Experience

ICESIII, Canada, 2007Presented by Eden Brinkley

Overview of presentation

• Overview of ABS annual collections– Current collections– Client concerns

• Annual Integrated Collection (AIC)– Motivation – Understanding client needs– Basic AIC model– Design and estimation– Editing processes and systems– Developing a common culture

• Key learnings

Current annual collections

• Annually– Economic Activity Survey (economy wide)– Manufacturing Industry Survey– Mining and Utilities Survey– Selected Service Industries Surveys

• Biennially– Information and Communications Technology Survey

• Every 6-7 years– Construction Industry Survey– Retail Industry Survey– Wholesale Industry Survey

Client concerns with current collections

• Lack of coherence amongst the annual collections – Releasing statistics, nominally for the same industry, that did

not accord

• Need for operational efficiency and simplicity– Annual collections operated as individually managed projects

with different methodologies, systems and processes

• National Accounts Branch felt the annual estimates were not always ‘fit for purpose’– Lack of coherence with quarterly collections– Outputs not always aligned with National Accounts needs– Lack of sectors splits and regular product detail

Annual Integrated Collection (AIC)

• Established a project in 2003 to integrate annual collections into a single system known as the AIC– Common designs directly focussed on meeting

client needs– Common systems infrastructure– Changed culture and motivation for staff

• focus on a ‘big picture’ program

• Phased in progressively over 4 years from the 2004-05 reference year

Approach to integration

• Started with a ‘bottom up’ approach– Progressively align elements of the various

collections in the program (e.g scope, estimation methodologies, etc.)

• Moved to a ‘top down’ approach– Start with a comprehensive prioritized

assessment of client needs– Design collections to meet these needs

Key client needs

• National Accounts requirements (must have)– Financial data across economy at Supply Use

industry level annually– Financial product level data every 1 to 9 years,

depending on priority

• Other client requirements– A range of financial, structural and activity data

• By fine industry levels, business size, and broad geographic region

– Alternative views of the data across industries• e.g. satellite accounts

Broad AIC model

• A two-part integrated collection vehicle

– An annual economy-wide core collection

– A rolling program of more detailed but less frequent industry-specific collections linked to the core

• Administrative by-product data used, as much as possible, to complement the directly collected survey data– Decision criterion: “is the resulting data fit for

purpose?”

AIC collection program (example)

SU = Su p p ly U se an d IO = In p u t/O u tp u t

In du stry Y ear 1 Y ear 2 Y ear 3 Y ear 4 Y ear 5 Y ear 6 Y ear 7 Y ear 8 ...

C ore program

A ll in du stries

R oll in gprogram

S U in du stry A IO in du stry A

IO in du stry B

IO in du stry C

S U in du stry B

S U in du stry C IO in du stry DIn du stryclass A

In du stryclass B

IO in du stry E

SU = Supp ly /U se and IO = Input/O utput

Design constraints

• The sample is designed to meet the following constraints:

– Core collection to have a sample size of around 15,000 units each year

– Rolling program of collections to have a further 10,000 units each year

– Sample for the rolling program is effectively a top-up of the core sample for the relevant industries

– Industry estimates from the core and rolling program to be the same/aligned wherever possible

Scope and coverage

• Scope should ideally align with the scope of Australian National Accounts– All Australian based business activities of business entities

operating during the reference period– In practice there are some exclusions (e.g. households

institutional sector)

• Continuing to explore the inclusion of government units in core estimates for the first time

• Coverage is restricted to businesses on ABS Business Register at 30th June each year

Frame

• Drawn from the ABS Business Register

• ABS Business Register has two populations– ABS maintained complex population

• Approximately 15,000 units• Accounts for about half of total operating income, and

about a third of employment

– Australian Taxation Office maintained simple population

• Almost 2 million units• No feedback allowed

Sample design

• Stratification– Industry x sector x broad region x size

• Design variables– Industry value added, Compensation of

employees, Gross operating surplus, Gross fixed capital formation

• Reliable estimates of movement are the first priority– In practice the design is mostly based on level

estimates

Estimation

• Generalised regression (GREG) estimation used for directly collected survey data– Business Activity Statement (BAS) data used as

benchmarks

• BAS data also substitute for very small businesses in scope of the AIC– Substitution leads to challenges in modelling data

items not found on the tax file (e.g. employment)

• Preliminary estimates to National Accounts nine months after reference period– Publish within 12 months

AIC Estimation Framework

Complete Enumeration StreamData source: AIC surveyPopulation: 3,500 units (0.2% of total)Survey Sample: 3,500 units (55% of BAS Total sales)

Industry

Po

pu

lati

on GREG Estimation Stream

Data source: AIC survey, Benchmark Source: BASPopulation 1.2 million (63.2% of total)Survey Sample: 12,000 units (44% of BAS Total sales)

Employment* > 300

BAS Substitution Stream Data Source: BAS dataPopulation: 0.7 million units (36.6% of total)Survey Sample: 0 units (1% of BAS Total sales)

Maximum of 2.5% of Industry class turnover

Integrated outputs

Clie nt Re quire m e nts(ide ntifie d by the AIC

progra m se ttingproce ss)

Annua l'core '

re quire m e nts

Additiona l'rolling'

re quire m e nts

Estim a tion Inte gra te d suite of AIC outputs(de signe d to m e e t clie nt re quire m e nts)

Da ta Source

Adm inby-product

Dire ctlycolle cte d

surve y da ta

GREGEstim a tion

Da taSubstitution

M ode lling/synthe tic

e stim a tion

Core outputs- Broa d fina ncia ls- Broa d de m ogra phics

Rolling outputs- De ta ile d fina ncia ls- De ta ile d de m ogra phics- P roduct a nd Activity da ta- Cha ra cte ristics da ta

Instrument design

• Aiming to rationalize both data content and the different form types used across AIC collections – Understanding client requirements

• Mapping project established to map SNA concepts to questions on forms– Success in developing improved ‘Standard

Question Wording’ for 2006-07 – More rigorous process again for 2007-08

• Aiming to investigate use of tailored forms for groups of similar businesses

Integrated infrastructure

• ABS has already built the basis of an integrated end-to-end systems environment

• Main components include:– Input Data Warehouse– Intelligent forms scanning facilities– Blaise for data capture and editing– Standard estimation and imputation system– Provider management system– Central repository for storing aggregate data– Central metadata repository

• AIC is both influencing and leveraging off this integrated end-to-end environment

Editing processes and systems

• AIC core and rolling collections steadily implementing best practice– Developing detailed editing strategies and common

processes for each collection

• Aiming to deliver ‘fit for purpose’ data at minimum cost

• Editing re-engineering supported by– Editing Guide and training for AIC collection areas– Editing workshops for AIC collections– Common use of end-to-end systems developments

• Mainly focused on new micro editing tools to date

Micro editing process

1. Prepare for micro editing process

2. Clean data for selective editing

3. Identify and treat critical stream

4. Apply automatic amendments

Editing strategy and other metadata

Other data to support micro editing

Captured unit record data for current cycle

Automatic editingand imputation

Selective editing

Initial editing

Plan and prepare

Sufficient data for macro editing

No

Yes

Other enablers

• Editing strategies critically reviewed by peers• Formal senior level sign-off gives legitimacy to

new approaches being taken• Analysts focussed on validating the highest

priority outputs for key clients• Clerical attention focussed only on the

important units• Earlier system specifications meant systems

were delivered on time

Gains from improved editing

0

10

20

30

40

50

2003-042004-052005-06

Percentage of units edited

0

5

10

15

20

2003-042004-052005-06

Nominal cycle time Months after ref period

0

50

100

150

200

2003-042004-052005-06

Nominal staff months

Future editing related work

• Integrated approach to imputation and outliering

• Adoption of common macro editing processes and tools

• Integrated output and dissemination strategy• More emphasis on ensuring common

process flows across collections generally • Improved documentation (e.g. AIC quality

manual)

Common AIC culture

Common Common cultureculture

Central coordination

team

Senior management

support

Standard tools and

frameworks

Clear goals and priorities

One virtual team

Increased

communication

Key learning points

• Focus on priority client needs at each stage of the collection design and conduct to ensure outputs are ‘fit for purpose’

• From time to time take a green-fields/top down approach to collection design

• A good collection frame is the foundation for the production of quality statistics

• Work to maximize use of administrative data for design, estimation, data supplementation, etc.

• Integrated processes and systems will lead to both improved efficiency and data quality

• Build systems from the perspective of the end user

• Build a unified culture – one which involves compromise for the greater good

Questions?