arXiv:1408.2757v1 [stat.ME] 12 Aug 2014 Bayesian Lattice Filters for Time-Varying Autoregression and Time-Frequency Analysis Wen-Hsi Yang 1 , Scott H. Holan 2 , Christopher K. Wikle 3 Abstract Modeling nonstationary processes is of paramount importance to many scientific disciplines including environmental science, ecology, and finance, among others. Consequently, flexible methodology that provides accurate estimation across a wide range of processes is a subject of ongoing interest. We propose a novel approach to model-based time-frequency estima- tion using time-varying autoregressive models. In this context, we take a fully Bayesian approach and allow both the autoregressive coefficients and innovation variance to vary over time. Importantly, our estimation method uses the lattice filter and is cast within the partial autocorrelation domain. The marginal posterior distributions are of standard form and, as a convenient by-product of our estimation method, our approach avoids undesirable matrix in- versions. As such, estimation is extremely computationally efficient and stable. To illustrate the effectiveness of our approach, we conduct a comprehensive simulation study that com- pares our method with other competing methods and find that, in most cases, our approach performs superior in terms of average squared error between the estimated and true time- varying spectral density. Lastly, we demonstrate our methodology through three modeling applications; namely, insect communication signals, environmental data (wind components), and macroeconomic data (US gross domestic product (GDP) and consumption). Keywords: Locally stationary; Model selection; Nonstationary; Partial autocorrelation; Piecewise stationary; Sequential estimation; Time-varying spectral density. 1 CSIRO Computational Informatics, Ecosciences Precinct, GPO Box 2583, Brisbane QLD 4001, Aus- tralia, [email protected]2 (To whom correspondence should be addressed) Department of Statistics, University of Missouri, 146 Middlebush Hall, Columbia, MO 65211-6100, [email protected]3 Department of Statistics, University of Missouri, 146 Middlebush Hall, Columbia, MO 65211-6100, [email protected]
49
Embed
Bayesian Lattice Filters for Time-Varying Autoregression ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
arX
iv:1
408.
2757
v1 [
stat
.ME
] 1
2 A
ug 2
014
Bayesian Lattice Filters for Time-Varying
Autoregression and Time-Frequency Analysis
Wen-Hsi Yang1, Scott H. Holan2, Christopher K. Wikle3
Abstract
Modeling nonstationary processes is of paramount importance to many scientific disciplinesincluding environmental science, ecology, and finance, among others. Consequently, flexiblemethodology that provides accurate estimation across a wide range of processes is a subjectof ongoing interest. We propose a novel approach to model-based time-frequency estima-tion using time-varying autoregressive models. In this context, we take a fully Bayesianapproach and allow both the autoregressive coefficients and innovation variance to vary overtime. Importantly, our estimation method uses the lattice filter and is cast within the partialautocorrelation domain. The marginal posterior distributions are of standard form and, as aconvenient by-product of our estimation method, our approach avoids undesirable matrix in-versions. As such, estimation is extremely computationally efficient and stable. To illustratethe effectiveness of our approach, we conduct a comprehensive simulation study that com-pares our method with other competing methods and find that, in most cases, our approachperforms superior in terms of average squared error between the estimated and true time-varying spectral density. Lastly, we demonstrate our methodology through three modelingapplications; namely, insect communication signals, environmental data (wind components),and macroeconomic data (US gross domestic product (GDP) and consumption).
2(To whom correspondence should be addressed) Department of Statistics, University of Missouri, 146Middlebush Hall, Columbia, MO 65211-6100, [email protected]
3Department of Statistics, University of Missouri, 146 Middlebush Hall, Columbia, MO 65211-6100,[email protected]
relatively larger spectra (or fluctuations) for the early period around 1950. Then, due to the
oil crisis, another fluctuation comes out in the period of 1970–1980. The fluctuation at the
end of the sample reflects the 2008–2009 worldwide financial crisis. See Koopman and Wong
(2011) for further discussion. Using the same MCMC iterations as Section 4.1, Figures 11c
and 11d present the associated posterior standard deviations.
5 Discussion
This paper develops a computationally efficient method for model-based time-frequency anal-
ysis. Specifically, we consider a fully Bayesian lattice filter approach to estimating time-
varying autoregressions. By taking advantage of the partial autocorrelation domain, our
approach is extremely stable. That is, the PARCOR coefficients and the TVAR innovation
variances are specified within the lattice structure and then estimated simultaneously. No-
tably, the full conditional distributions arising from our approach are all of standard form
and, thus, facilitate easy estimation.
The framework we propose extends the current model-based approaches to time-frequency
analysis and, in most cases, provides superior performance, as measured by the average
squared error between the true and estimated time-varying spectral density. In fact, for
slowly-varying processes we have demonstrated significant estimation improvements from
using our approach. In contrast, when the true process comes from a piecewise AR model
the approach of Davis et al. (2006) performed best, with our approach a close competitor and
performing second best. This is not unexpected as the AutoPARM method is a model-based
segmented approach and more closely mimics the behavior of a piecewise AR.
In addition to a comprehensive simulation study we have provided three real-data ex-
amples, one from animal (insect) communication, one from environmental science, and one
from macro-economics. In all cases, the exceptional time-frequency resolution obtained us-
22
ing our approach helps identify salient features in the time-frequency surface. Finally, as a
by-product of taking a fully Bayesian approach, we are naturally able to quantify uncertainty
and, thus, use our approach to draw inference.
Acknowledgments
This research was partially supported by the U.S. National Science Foundation (NSF) and
the U.S. Census Bureau under NSF grant SES-1132031, funded through the NSF-Census
Research Network (NCRN) program. The authors would like to thank the Reginald Cocroft
lab for use of the insect communication data analyzed in Section 4.1. The authors would
like to thank Drs. Richard Davis, Hernando C. Ombao, and Ori Rosen for providing their
codes. Finally, we also thank Dr. Mike West for generously providing his code online.
Table 1: The mean, ASE and standard deviation, sdASE, of the ASE values for the sim-ulations presented in Section 3. Note that the bold values represent the approach havingminimum ASE.
Figure 2: (a) and (b) depict one realization along with the true time-frequency representationof the time-varying AR(2) process (TVAR2), respectively (Section 3.1). (c) illustrates thebox-plots of the average squared error (ASE) values corresponding to the time-frequencyrepresentation of the TVAR2 for all of the approaches considered.
Figure 3: (a) and (b) depicts one realization along with the true time-frequency representa-tion of the time-varying AR(6) process (TVAR6), respectively (Section 3.2). (c) illustratesthe box-plots of the average squared error (ASE) values corresponding to the time-frequencyrepresentation of the TVAR6 for all of the approaches considered.
Figure 4: (a) and (b) depict one realization along with the true time-frequency representa-tion of the piecewise AR process (PieceAR), respectively (Section 3.3). (c) illustrates thebox-plots of the average squared error (ASE) values corresponding to the time-frequencyrepresentation of the PieceAR for all of the approaches considered.
27
0 0.2 0.4 0.6 0.8 1
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
Time
(a)
AdaptSPEC AutoPARM AutoSLEX WPK1999 BLFFix BLFDyn
0.5
1
1.5
2
2.5
3
3.5
4
(b)
AS
E
Figure 5: (a) depicts one realization of the simulated insect communication signals (Sim-Bugs), Section 3.4. (b) illustrates the box-plots of the average squared error (ASE) valuescorresponding to the time-frequency representation of the SimBugs for all of the approachesconsidered.
28
0 0.2 0.4 0.6 0.8 1
−0.4
−0.2
0
0.2
0.4
(a)
Fre
quen
cy
(b)
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
−80
−70
−60
−50
−40
−30
−20
Time
Fre
quen
cy
(c)
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
1
2
3
4
5
6
7
Figure 6: (a) An example of typical signal corresponding to a successful mater (Section 4.1).(b) and (c) present posterior mean and standard deviation of the TVAR(6) spectral repre-sentation of the signal in plot (a).
Figure 7: (a) shows the BLF-scree plot of the treehopper communication signal. (b) depictsthe first six time-varying estimated PARCOR coefficients. (c) and (d) show the estimatedtime-varying coefficients and innovation variances of the TVAR(6) model.
Figure 8: (a) and (b) show daily time series (1964-1994) of east/west and north/southcomponents of wind, respectively. Both components are measured in meters per second(m/s).
31
Fre
quen
cy
(a)
1970 1980 19900
0.1
0.2
0.3
0.4
0.5
10
15
20
25
30
(b)
1970 1980 19900
0.1
0.2
0.3
0.4
0.5
5
6
7
8
9
10
Year
Fre
quen
cy
(c)
1970 1980 19900
0.1
0.2
0.3
0.4
0.5
0
1
2
3
4
Year
(d)
1970 1980 19900
0.1
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Figure 9: (a) and (c) display the posterior mean and standard deviation of time-frequencyrepresentations of the wind east/west component by fitting a TVAR(3) model. (b) and (d)display the posterior mean and standard deviation of time-frequency representations of thewind north/south component by fitting a TVAR(4) model.
32
1950 1960 1970 1980 1990 2000 2010
7.5
8
8.5
9
9.5
Year
Inde
x V
alue
s
(a)
1950 1960 1970 1980 1990 2000 2010
−0.02
−0.01
0
0.01
0.02
0.03
Year
log
diffe
renc
e
(b)
1950 1960 1970 1980 1990 2000 2010
−0.02
0
0.02
0.04
Year
log
diffe
renc
e
(c)
Figure 10: (a) displays the logarithm of GDP (solid line) and consumption (dash line) in fromthe first quarter of 1947 to the first quarter of 2010. (b) and (c) present the log difference ofGDP and the log difference of consumption.
33
Time
Fre
quen
cy
(a)
1950 1960 1970 1980 1990 2000 20100
0.1
0.2
0.3
0.4
0.5
−11.5
−11
−10.5
−10
−9.5
−9
−8.5
−8
Time
(c)
1950 1960 1970 1980 1990 2000 2010
−11.5
−11
−10.5
−10
−9.5
−9
−8.5
Time
Fre
quen
cy
(b)
1950 1960 1970 1980 1990 2000 20100
0.1
0.2
0.3
0.4
0.5
0
0.1
0.2
0.3
0.4
0.5
0.6
Time
(d)
1950 1960 1970 1980 1990 2000 20100
0.2
0.4
0.6
0.8
1
Figure 11: (a) and (b) display the time-frequency representations of the log difference ofGDP and consumption series, respectively. (c) and (d) show the standard deviations ofthe time-frequency representations of the log difference of GDP and consumption series,respectively.
34
Appendix A: Sequential Updating and Smoothing
To complete the Bayesian estimation of the forward and backward PARCOR coefficients,
as well as the time-varying innovation variances, we use dynamic linear models (DLMs)
(see, West and Harrison, 1997; Prado and West, 2010). Specifically, we provide the details
and formulas for analysis of {α(m)t,m} and {σ2
f,m,t}. The analysis of {β(m)t,m } and {σ2
b,m,t} follow
similarly.
For t = 1, . . . , T , given the values of f(m−1)t and b
Step 7. Given the set of estimated values {α(m)t,m}, {β(m)
t,m }, for m = 1, . . . , P , use (7) and
(8) iteratively to get the set of estimated {a(P )t,m}, m = 1, . . . , P , as well as set {σ2
t = σf,P,t}.
38
Appendix B: Lattice Filter Structure
Figure 12a illustrates the lattice structure for an AR(P ) model. Given α(m)m and β
(m)m ,
m = 1, . . . , P , recursive use of (2) and (3) can produce forward and backward prediction
errors for the forward and backward AR(m) models (i.e., f(m)t and b
(m)t ). Alternatively,
Figure 12b illustrates the lattice structure for a TVAR(P ) model. Different from Figure 12a,
the forward and backward PARCOR coefficients are time dependent. Given α(m)t,m and β
(m)t,m ,
m = 1, . . . , P , this case presents recursive use of (5) and (6) to produce forward and backward
prediction errors for the forward and backward TVAR(m) model.
39
(a)
xt
f(0)t
b(0)t
f(1)t
b(1)t
f(P )t
b(P )t
-α(1)1
-β(1)1
-α(P )P
-β(P )P
(b)
xt
f(0)t
b(0)t
f(1)t
b(1)t
f(P )t
b(P )t
-α(1)t,1
-β(1)t,1
-α(P )t,P
-β(P )t,P
Figure 12: (a) Graphical representation of the lattice filter for a stationary AR model. (b)Graphical representation of the lattice filter for a TVAR model.
40
Appendix C: Supplemental Figures for Case Studies
This section contains supplemental figures associated with Section 4 of the main text (Case
Studies). These figures are not strictly necessary to illustrate the intended applications.
Nevertheless, we include them here for further potential scientific insight.
Figures 13a and 14a provide BLF-scree plots associated with the east/west and north/south
components of the wind signal. The first three time-varying estimated PARCOR coefficients
of the east/west and north/south components of the wind signal are provided in Figures 13b
and 14b. The estimated time-varying coefficients of the TVAR(3) model for the east/west
component are given in Figures 13c and 13d whereas the estimated time-varying coefficients
of the TVAR(4) model for the north/south component are given in Figures 14c and 14d.
Finally, the BLF-scree plot, time-varying estimated PARCOR coefficients and time-varying
innovation variances for the difference of the logarithm GDP and Consumptions series are
Figure 13: (a) shows the BLF-scree plot of the east/west component of the wind signal. (b)depicts the first three time-varying estimated PARCOR coefficients. (c) and (d) show theestimated time-varying coefficients and innovation variances of the TVAR(3) model.
Figure 14: (a) shows the BLF-scree plot of the north/south component of the wind signal.(b) depicts the first four time-varying estimated PARCOR coefficients. (c) and (d) show theestimated time-varying coefficients and innovation variances of the TVAR(4).
Figure 15: (a) shows the BLF-scree plot of the difference of logarithm GDP series. (b) depictsthe first time-varying estimated PARCOR coefficients. (c) and (d) show the estimated time-varying coefficients and innovation variances of the time-varying AR(1).
Figure 16: (a) shows the BLF-scree plot of the difference of logarithm consumption series.(b) depicts the first two time-varying estimated PARCOR coefficients. (c) and (d) show theestimated time-varying coefficients and innovation variances of the time-varying AR(2).
45
References
Cocroft, R. B. and McNett, G. D. (2006). “Vibrational communication in treehoppers(Hemiptera: Membracidae).” In Insect Sounds and Communication: Physiology, Ecologyand Evolution, S. Drosopoulos and M. F. Claridge (Eds.), 305–317. Taylor & Francis.
Cressie, N. and Wikle, C. K. (2011). Statistics for Spatio-Temporal Data. John Wiley &Sons.
Davis, R. A., Lee, T. C. M., and Rodriguez-Yam, G. A. (2006). “Structural break estimationfor nonstationary time series models.” Journal of the American Statistical Association,101, 473, 223–239.
Everitt, R. G., Andrieu, C., and Davy, M. (2013). “Online Bayesian inference in sometime-frequency representations of non-stationary processes.” IEEE Transactions of SignalProcessing , 61, 5755–5766.
Feichtinger, H. G. and Strohmer, T. (1998). Gabor Analysis and Algorithms: Theory andApplications . Birkhauser.
Fitzgerald, W. J., Smith, R. L., Walden, A. T., and Young, P. C. (2000). Nonlinear andNonstationary Signal Processing . Cambridge University Press.
Godsill, S. J., Doucet, A., and West, M. (2004). “Monte Carlo smoothing for nonlinear timeseries.” Journal of the American Statistical Association, 99, 465, 156–168.
Grochenig, K. (2001). Foundations of Time-Frequency Analysis . Birkhauser.
Hayes, M. H. (1996). Statistical Digital Signal Processing and Modeling . John Wiley & Sons.
Haykin, S. (2002). Adaptive Filter Theory (4th ed). Pearson.
Holan, S. H., Wikle, C. K., Sullivan-Beckers, L. E., and Cocroft, R. B. (2010). “Modelingcomplex phenotypes: generalized linear models using spectrogram predictors of animalcommunication signals.” Biometrics , 66, 3, 914–924.
Holan, S. H., Yang, W.-H., Matteson, D. S., and Wikle, C. K. (2012). “An approach foridentifying and predicting economic recessions in real-time using time-frequency functionalmodels.” Applied Stochastic Models in Business and Industry , 28, 485–499.
Kitagawa, G. (1988). “Numerical approach to non-Gaussian smoothing and its applica-tions.” In Proceedings of the 20th Symposium on the Interface, ed. J. M. E.J. Wegman,D.T. Gantz. Interface Foundation of North America.
46
— (2010). Introduction to Time Series Modeling . Chapman & Hall/CRC.
Kitagawa, G. and Gersch, W. (1996). Smoothness Priors Analysis of Time Series . Springer.
Koopman, S. and Wong, S. (2011). “Kalman filtering and smoothing for model-based signalextraction that depend on time-varying spectra.” Journal of Forecasting , 30, 147–167.
Mallat, S. G. (2008). A Wavelet Tour of Signal Processing: The Sparse Way . AcademicPress.
Ombao, H., Raz, J., Von Sachs, R., and Malow, B. (2001). “Automatic statistical analysisof bivariate nonstationary time series.” Journal of the American Statistical Association,96, 454, 543–560.
Oppenheim, A. V. and Schafer, R. W. (2009). Discrete-Time Signal Processing . PrenticeHall Signal Processing.
Percival, D. B. and Walden, A. T. (2000). Wavelet Methods for Time Series Analysis .Cambridge University Press.
Prado, R. and Huerta, G. (2002). “Time-varying autogressions with model order uncer-tainty.” Journal of Time Series , 23, 5, 599–618.
Prado, R. and West, M. (2010). Time series: modeling, computation, and inference. CRCPress.
Rencher, A. C. (2002). Methods of Multivariate Analysis (2nd ed). John Wiley & Sons.
Rosen, O., Stoffer, D. S., and Wood, S. (2009). “Local spectral analysis via a Bayesianmixture of smoothing splines.” Journal of the American Statistical Association, 104, 485,249–262.
Rosen, O., Wood, S., and Stoffer, D. S. (2012). “AdaptSPEC: Adaptive spectral estimationfor nonstationary time series.” Journal of the American Statistical Association, 107, 500,1575–1589.
Shumway, R. H. and Stoffer, D. S. (2006). Time Series Analysis and Its Applications (2nded). Springer.
Vidakovic, B. (1999). Statistical Modeling by Wavelets . John Wiley & Sons.
West, M. and Harrison, J. (1997). Bayesian Forecasting and Dynamic Models (2nd ed).Springer.
West, M., Prado, R., and Krystal, A. D. (1999). “Evaluation and comparison of EEGtraces: latent structure in nonstationary time series.” Journal of the American StatisticalAssociation, 94, 446, 375–387.
47
Wikle, C. K., Madden, R. A., and Chen, T.-C. (1997). “Seasonal variation of upper tro-pospheric lower stratospheric equatorial waves over the tropical Pacific.” Journal of theAtmospheric Sciences , 54, 1895–1909.
Wolfe, P. J., Godsill, S. J., and Ng, W. J. (2004). “Bayesian variable selection and regular-ization for time–frequency surface estimation.” Journal of the Royal Statistical Society:Series B (Statistical Methodology), 66, 3, 575–589.
Wood, S., Rosen, O., and Kohn, R. (2011). “Bayesian mixtures of autoregressive models.”Journal of Computational and Graphical Statistics , 20, 1, 174–195.
Yang, W.-H., Wikle, C. K., Holan, S. H., and Wildhaber, M. L. (2013). “Ecological pre-diction with nonlinear multivariate time-frequency functional data models.” Journal ofAgricultural, Biological, and Environmental Statistics , 18, 3, 450–474.