Census Bureau Seasonal Adjustment Software and Research
U S C E N S U S B U R E A UU S C E N S U S B U R E A U
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filters and Filter Diagnostics– Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions
State-Space Models using Sampling Error Data
Non-Gaussian “Structural” State-Space Models for More Stable Resistance to “Outliers”
Statistical Research Division Time Series Group Research and “X-12-ARIMA+” Programming
Brian.C.MonsellKellie.C.WillsWilliam.R.Bell (honorary)David.F.Findley (honorary)Donald.E.Martin (Part-time, Howard University)Trang.Ta.Nguyen (1-year in-house visitor)John.Alexander.Aston (2-year Post-Doc from
Imperial College, London)S.J.M. Koopman (Fellow, Free Univ. of Amsterdam)
Economic and Statistical Programming DivisionTime Series Methods Branch
Research and SAS, Excel Programming
Catherine.C.HoodKathleen.M.McDonald.JohnsonGolam.FarooqueRoxanne.Feldpausch
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Software:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses sets of series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
X-12-ARIMA
Improvements over StatsCan’s X-11-ARIMA• regARIMA models (including outliers, user-defined
regressors, etc.) vs. ARIMA models• Much more extensive automatic options for
modeling, including trading day, holiday est., additive vs. multiplicative adjustment
• More diagnostics (e.g. spectra, revisions)• Specialized output files, e.g. log files for users
favorite diagnostics, from many X-12-Graph (SAS, but for non-SAS-users)
X-11 Seasonal Adjustment
RegARIMA Models(Forecasts, Backcasts, and Preadjustments)
Modeling and Model Comparison Diagnostics and Graphs
Seasonal Adjustment Diagnostics and Graphs
REGARIMA Model
transformation ARIMA Process
Regressors for trading day and holiday or calendar effects, additive outliers,temporary changes, level shifts, rampsuser-defined effects
Leap-year adjustment, or“subjective” strike adjustment, etc.
ttt
t ZXDY
log
Xt
Dt
Types of Regression Variables Available in X-12-ARIMA
Outlier and Trend-Change EffectsAdditive (or Point) OutliersTemporary Change Outliers
Level shifts, Ramps Seasonal Effects
Calendar month indicators*Trigonometric Seasonal (Sines-Cosines)*
Calendar EffectsTrading Day (Flows or Stocks)*Leap-year February*, Length of Month*Shifting Holidays (e.g. Easter)
Constant Term User-Defined Effects
*Two-regime option availableNote: Regression coefficients can be fixed
X-12-ARIMA Releases
Ver. 0.2.10 July (Statistics Canada options)Ver. 0.3 Summer (TRAMO-type automatic
ARIMA model selection)-based on information gleaned from
TRAMO code provided by Victor Gomez
Ver. 1.0 End of year (Better organized output and manual, more testing etc.)
“X-12-ARIMA/SEATS”• Offers both x11{ } and seats{ } commands to provide “X-
11” or SEATS type seasonal adjustments with X-12-ARIMA diagnostics as well as SEATS diagnostics
• Is being updated from SEATS2000 to SEATS2001&2002 (with support from Agustin Maravall and Gianluca Caporello)
• Schizophenic (duplicate) output, currently• Distribution for research and testing to statistical
agencies and central banks in 2003
Diagnoses from X-12-ARIMA/SEATS
1. Spectrum diagnostic reveals source of “Invalid Decomposition” problem
X-12-A/SEATS COMMAND FILE
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}#x11{}seats{}
Message from seats{ } run:
• NOTE: Spectral plot for the seasonally adjusted series cannot be done when SEATS cannot perform a signal extraction.
Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.3846 0.12087
Seasonal MA Lag 12 -0.3665 0.12612
10*LOG(SPECTRUM) of the regARIMA model residuals Spectrum estimated from 1990.Jan to 1995.Oct.+++++++I+++++++++++++++++++++++++++++ -22.11I * I * I * I * * -23.34I * * I * * * I * * T I * * *T -24.57I * * *T I * * *T I * * * *T I * * * *T* -25.81I * * * *T* I * * * *T*
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}check{}x11{}#seats{}
X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24
WARNING: At least one visually significant trading day peak has been found in one or more of the estimated spectra.
G.1 10*LOG(SPECTRUM) of the differenced, transformed seasonally adjusted data. Spectrum estimated from 1990.Jan to 1995.Oct. ++++++++++I+++++++++++++++++++++++++++++++++++ I T I T I T -20.10I T I T I T I T -22.01I T I T I T
series {file="serie.txt"format="tramo"}
transform{function=log}outlier{critical=3.7}arima{model=(0 1 1)(0 1 1)}regression{variables=td}check{}seats{}
X-12-ARIMA/SEATS Seasonal Adjustment ProgramVersion Number 0.3s Build 24
Reading input spec file from metalss.spc Storing any program output into metalss.out Storing any program error messages into metalss.err
WARNING: At least one visually significant seasonal peak has been found in one or more of the estimated spectra.
Standard Parameter Estimate Errors ----------------------------------------------------- Nonseasonal MA Lag 1 0.1995 0.12871
Seasonal MA Lag 12 0.3843 0.15795
X-12-ARIMA Diagnoses for SEATS
2. T/S Practice of adding outliers to improve kurtosis, etc. can substantially increase the size of revisions of the initial seasonal adjustments:
Example (from Catherine Hood) US Exports of Passenger Cars: History diagnostic shows cost to revisions of adding outlier regressors to reduce kurtosis
Outline of Talk: SoftwareX-12-ARIMA and its Evolution to “X-12-ARIMA/SEATS”Windows version ( Jurgen Doornik & GiveWin)Supporting software
Genhol (holiday regressors)SAS Utilities:X-12-Graph (14+ types of diagnostic graphs)Interface (simplifies analyses of many series)X-12-Write (easy prod./modif. of .spc files)X-12-Review (1 page diagnostic summaries)
Genhol
• From holiday date file: Generates regressor matrices and associated command files to enable X-12-ARIMA estimation of complex moving holiday effects (e.g. for Easter, Ramadan, etc.).
• Regressors for up to three intervals:– before-the-holiday interval– surrounding-the-holiday interval– past-the-holiday interval (“recovery” interval)
Proportionality Regressors: An Example
• Assume :– An effect interval is 10 days long, and this year
2 of its days fall in January and 8 in February.The interval regressor’s values for this year will
be:– 0.2 in January– 0.8 in February– 0.0 for the rest of the year
Interface Program (SAS): for seasonal adjustment of sets of series
Example: Seasonally Adjusted Total U.S. Imports = sum of 140 component series, c. 80% of which are seasonally adjusted.
What is the effect on the month-to-month changes and quality diagnostics of the S. A. Total Imports if the seasonal adjustment options are changed for 5 of the component series?
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)– Filter Diagnostics – Automatic modeling: TRAMO vs. X-12’s “TRAMO”– Revisions
Filters and Filter Diagnostics
• Filter (spectral) diagnostics needed– To understand limitations/issues with short
series (finite filter diagnostics, also for concurrent adjustments, trends)
– To decide between closely competitive models
Paper by David Findley and Donald Martin.
0 1 2 3 4 5 6
Cycles per year
0.0
0.5
1.0
1.5
Squ
ared
gai
nSquared gain of symmetric SEATS filters
Parameter values -- 0.4,0.8
infinite109 months49 months
Outline of Talk: Research
TRAMO/SEATS Evaluation & Improvement for X-12-ARIMA\SEATS (also for short series)
• Automatic modeling: TRAMO vs. X-12’s “TRAMO”
• Accuracy– Results from simulated series
• Revisions– Results from Census Bureau series
ESMPD’s Automatic Modeling Study
• First presented at the International Forecasters Symposium, June 2001
• Continuation of this work to appear at the ASA meetings, August 2002, in a paper by Kathleen McDonald-Johnson, et al.
Series
306 time series from the US Census Bureau’s Import/Export series and Retail Sales
Results
• 88 series (29%) with same regARIMA model• 27 series (9%) with same differencing and same
regressors but different ARMA choices• 123 series (40%) with same differencing, but
different regressors• 32 series (10%) with different nonseasonal
differencing order (but sometimes offset by a constant)
• 36 series (12%) with different seasonal differencing order
Conclusions
• TRAMO’s weakness is the procedure for deciding about trading day modeling– TRAMO developers are aware of our results
• X-12-ARIMA has a problem with choosing nonparsimonious models– Monsell has already implemented some
changes, including a unit root test.
Why Are Different Models Chosen?
• Model estimation method is different – TRAMO : Hannan-Rissanen and m.l.e conditional on
AR part of model– X-12-ARIMA : Exact MLE
• Model residuals are different, which can lead to different choices of outliers
• Outlier procedure itself is different – TRAMO removes insignificant outliers after each
iteration• TRAMO uses approximate BIC
Accuracy: X-12-ARIMA vs T/S (ESMPD)
• Results from 54 simulated series were first presented at the ASA meetings, August 2000– Continuation of the first SEATS studies,
beginning in 1997
The Simulated Series
• Fifty-four series– Six different trends – three from SEATS and
three from X-12– Six different seasonal factors – three from
SEATS and three from X-12– Irregular sampled from three sets of irregular
factors combined from SEATS and X-12
Results of Accuracy Study
• SEATS performed better on the majority of series with large irregulars if the series are 9+ years long, but most adjustments were not acceptable.
• Both programs did better than expected on the short series, but X-12-ARIMA adjustments were usually better than SEATS adjustments on series 4-7 years long
Revisions: X-12-ARIMA vs T\S
• New ESMPD study using X-12-SEATS on Census series. “Final” results will be presented at the ASA meetings, August 2002.– Can we identify characteristics in the series
that will indicate if its “linearized” series will be a better candidate for a model-based adjustment than for an X-11 filter adjustment or vice versa?
Methods
• Use X-12-SEATS to get revision diagnostics from both an X-11/X-12-type adjustment and a SEATS adjustment– Used TRAMO to get the ARIMA model, and
then used either an x11 or a seats “spec”
Very Preliminary Results
• 260 US Import/Export series• Only a very small subset (18 series) where
we can see definite differences in the revision diagnostics for the seasonal adjustment
An Observation: Series with
– Large revisions in X-12 and smaller revisions in SEATS had generally large values for 12 (most greater than 0.95) and values for X-12’s I/S ratio < 5.
– Large revisions in SEATS and smaller revisions in X-12 had generally 0.4 < 12 < 0.6 and values for I/S > 6.
In both cases, smaller revisions are associated with more constant seasonal factor estimates
Next Steps
• Look at more series• Look at more diagnostics/characteristics of
the series to try to find patterns, not just revisions
Outline of Talk: Research
Projects almost ready to yield results:
State-Space Models using Sampling Error Data Non-Gaussian “Structural” State-Space Models
for More Stable Resistance to “Outliers”
State-Space Models with Sampling Error Statistics: Bell and Nguyen
100+ Disaggregate Construction series with “high” sampling error variancesConsider model-based adjustment with
regARIMA+observation errormodels that incorporate sampling error variance and autocovariance estimates to achieve acceptable or better seasonal adj’s.
(Need state-space for model est. & seas adj.)
Non-Gaussian “Structural” State-Space Models for More Stable Resistance to
“Outliers” Koopman and Aston X-12-ARIMA and T\S use outlier regressors
identified by t-statistics and critical values. Identifications can change as new data arrive, causing seasonal adjustment revisions.
Use heavy tailed non-Gaussian models instead of critical values. (Hard to estimate such models, simplest for “Harvey’s structural models”)
More Information
WWW site for X-12-ARIMA (papers and software):
www.census.gov/srd/www/x12a
Thanks to Catherine Hood for supplying some of these slides.