Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7 Robust Estimation for Circular Data using Claudio Agostinelli [email protected]Dipartimento di Statistica Universit` a Ca’ Foscari di Venezia San Giobbe, Cannaregio 873, Venezia Tel. 041 2347446, Fax. 041 2347444 http://www.dst.unive.it/~claudio 16 June 2006 Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 1/ 72 Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7 Outline 1 Circular data Introduction Example: Wind direction dataset Models for Circular data 2 Robust Statistics What is it? Outliers in Circular data Our point of view Minimum Distance Estimators Weighted Likelihood Estimators 3 Conclusions Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 2/ 72 Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7 Circular Data Circular data arises in many diverse scientific fields like in Natural, Physical, Medical and Social Sciences where some or all the measurements are directions. The two main ways correspond to the two principal circular measuring instruments: • the compass e.g. wind directions and directions of migrating animals. • the clock. e.g. arrival times (on a 24-hour clock) of subjects. Because of the nature of the circular observations, the analysis (descriptive and/or inferential) cannot be carried out with standard methods for observations on Euclidean space. Some references: Mardia and Jupp [2000], Jammalamadaka and SenGupta [2001] and Jupp and Mardia [1989]. Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 3/ 72 Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7 5% 10% 15% 20% 25% 30% 35% 40% N E S W + ● ● ● ● (0, 1] (1, 2] (2, 3] (3, 8.02] Wind rose of the dataset from Col de la Roa (Italy) meteorological station in March and April 2001 between 3.00 and 4.00 o’clock (n=310). R code Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 4/ 72
15
Embed
Claudio Agostinelli claudio@unive - R · Claudio Agostinelli { useR!2006 Conference { Wien ver. 1.0 { 16 June 2006 21/ 72 Focus Session on Robust Statistics is Friday, 3:00pm at HS
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Dipartimento di StatisticaUniversita Ca’ Foscari di Venezia
San Giobbe, Cannaregio 873, VeneziaTel. 041 2347446, Fax. 041 2347444http://www.dst.unive.it/~claudio
16 June 2006
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 1/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Outline
1 Circular dataIntroductionExample: Wind direction datasetModels for Circular data
2 Robust StatisticsWhat is it?Outliers in Circular dataOur point of viewMinimum Distance EstimatorsWeighted Likelihood Estimators
3 Conclusions
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 2/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Circular Data
Circular data arises in many diverse scientific fields like in Natural,Physical, Medical and Social Sciences where some or all themeasurements are directions. The two main ways correspond tothe two principal circular measuring instruments:
• the compass
e.g. wind directions and directions of migrating animals.
• the clock.
e.g. arrival times (on a 24-hour clock) of subjects.
Because of the nature of the circular observations, the analysis(descriptive and/or inferential) cannot be carried out with standardmethods for observations on Euclidean space.Some references: Mardia and Jupp [2000], Jammalamadaka andSenGupta [2001] and Jupp and Mardia [1989].
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 3/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
5%
10%
15%20%25%30%35%40%
N
E
S
W +
●
●
●
●
(0, 1]
(1, 2]
(2, 3]
(3, 8.02]
Wind rose of the dataset from Col de la Roa (Italy) meteorological
station in March and April 2001 between 3.00 and 4.00 o’clock (n=310).R code
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 4/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
N
E
S
W + ●●●●●●●●●
●●
●
●●●●●●●●
●●●●●
●●●●●●●●●
●●●●●
●●●●●●●●
●●●●●●●●
●●●●
●●●●●●
●●●●●●●
●●●
●●●
●●●●●●●
●●●
●●●
●●●●●●●●●●●
●●●●●●
●●●●●●
●●●●
●●●●●●●
●●●●●●●●●
●●●●●●
●●●●●●●
●●●●●
●●●●●●●●
●●●●●●●
●●●●●
●●●●●
●●●●●
●●●●●●●
●●●●
●●●●
●●●●
●●●●●●●
●●●●●●●
●●●●
●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●
●
●●●●
●
●●
●
●
●
●●● ●●●●● ● ●●
●● ●
●●●●
●●●●●●●
●
●●●●
●●●●●●●●●●●●●●●●●
N
E
S
W +
N
E
S
W +
Stack Circular Plot (Left) and estimated density (Right, together with a
Rose Diagram) of the Wind direction dataset. R code
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 5/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Models for Circular dataSome examples are:
• Von Mises (Normal Circular) distribution:
m(x ;µ, κ) =1
2πI0(x)exp(κcos(x − µ)), κ ≥ 0
• Wrapped Normal distribution:
m(x ;µ, ρ) =1
2π
1 + 2∞∑
p=1
ρp2cos p(x − µ)
, ρ = exp
(−σ2
2
)• Wrapped Cauchy distribution:
m(x ;µ, ρ) =1
2π
1− ρ2
1 + ρ2 − 2ρcos(x − µ), ρ = exp (−σ)
• Cardioid distribution:
m(x ;µ, ρ) =1 + 2ρcos(x − µ)
2π, −1
2< ρ <
1
2Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 6/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
N
E
S
W +
f*(x)VMWNWC
MLE estimators for the Wind direction dataset using
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 7/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Robust Statistics
Robust Statistics is a discipline which deals with the inferenceproblem in the case of misspecification of the model with respectto the distribution of the data. A“robust”estimator is able to welldescribe the parameters, or some features, of the“majority” (or“bulk”) of the data. And to quote:
• Hampel et al. [1986], pag. 56: “whereas in classical statisticsthe model has to fit all the data, in robust statistics it may beenough that it fit the majority of the data, the remainderbeing regarded as outliers”;
Some references: Collett [1980], Lenth [1981], Upton [1993] andSenGupta and Laha [2001]. A survey is He [1992].
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 8/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Outliers in Circular data
• In Euclidean setting the outliers are defined as:
Barnett et al. [1994], pag. 7: “... an outliers in a set of data tobe an observation (or subset of observations) which appears tobe inconsistent with the remainder of that set of data”;Hawkins [1980]: an outlier is “an observation that deviates somuch from other observations as to arouse suspicions that itwas generated by a different mechanism”;
The definition is based on some“geometric” distancebetween observations.
• In Circular setting: the sample space is bounded and theparametric space is often bounded too (e.g. the parametricspace of the mean direction in the Von Mises distribution isevery interval of length 2π, in general, [0, 2π)).
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 9/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Outliers in Circular Data: Pearson Residuals
In our approach, outliers are observations that are highly unlikelyto occur under the assumed model [see Markatou et al., 1995].
This definition is well adapted in the circle since it is based on a“probabilistic” distance. One way to measure this discrepancy isto use the Pearson Residuals [Lindsay, 1994] defined as follows
δ(x , θ, f ∗) =f ∗(x)
m∗(x ; θ)− 1
where f ∗(x) is a non parametric density estimator based on thedata and m∗(x ; θ) is a smoothed version of the density of themodel. Note that δ(x , θ, f ∗) = δ(x mod (2π), θ, f ∗).
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 10/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
0
π
2
π
3π
2
+
δ(x)
f(x)
0.02 m(x, π
3, 20)
Pearson Residuals (δ(x ; θ = (0, 5), f (x))) for
f (x) = 0.98m(x ; 0, 5) + 0.02m(x ;π/3, 20) where m(x ;µ, κ) is the density
of a Von Mises distribution. R code
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 11/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Minimum Distance EstimatorsFor continuous models the Power Divergence Measure [Cressie andRead, 1988] between the densities f ∗(x), m∗(x , θ) is
∫ 2π
0
(f ∗(x)m(x ;θ)
)α+1
α (α + 1)m(x ; θ) dx
or using the Pearson Residual function (in the unsmoothed model)δ(x ; θ, f ∗): ∫ 2π
0G (δ(x ; θ, f ∗))m(x ; θ) dx
where G (δ(x ; θ, f ∗)) = (δ(x ;θ,f ∗)+1)α+1
α (α+1) .Examples are:
• α = −1/2: Hellinger distance;
• α → −1: Kullback–Leibler divergence;
• α = −2: Neyman’s Chi–Square.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 12/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Numerical Calculation of Distances
Since f ∗(x) and m∗(x ; θ) and their transformations are 2π periodicfunctions the Power Divergence Measure is computed easily andfast by Fast Fourier Transform and Parseval’s formula. In fact, letφk(·) the Fourier transform, we get
2π
∫ 2π
0f ∗(x)α+1 m∗(x ; θ)−α dx =
∞∑k=−∞
φk
(f ∗(x)α+1
)φk
(m∗(x ; θ)−α
)
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 13/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Weighted Likelihood Estimating Equations
The estimating equations of WLEE is a modified version of theMLE equations where at each score is associated a weight definedas follows
w(x ; θ, f ∗) =A(δ(x ; θ, f ∗)) + 1
δ(x ; θ, f ∗) + 1
where A(δ) is the Residual Adjustment Function, with the formrelated to the Power Divergence Measure given by
A(δ) =(δ + 1)α+1 − 1
α + 1
Hence the WLEE estimator is the solution of
n∑i=1
w(xi ; θ, f∗)u(xi ; θ) = 0
where u(x ; θ) is the score function for the model.Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 14/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Example: Von Mises distribution
The WLEE for the Von Mises distribution is µ = arctan∗( Pn
i=1 w(xi ;µ,κ) sin(xi )Pni=1 w(xi ;µ,κ) cos(xi )
)κ = A−1
(Pni=1 w(xi ;µ,κ) cos(xi−µ)Pn
i=1 w(xi ;µ,κ)
)where
• A(κ) = I1(κ)/I0(κ) is the ratio of the two modified Besselfunctions of the first kind with order zero and one;
• arctan∗ is the“quadratic-specific” inverse of the tangent thatprovides the unique inverse of the tangent in [0, 2π).
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 15/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 61/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
> kappaklbw <- sapply(bw, temp, dir = dir, alpha = -1)> kappancsbw <- sapply(bw, temp, dir = dir, alpha = -2)> plot(bw, kappahdbw, xlab = "bandwidth", ylab = expression(kappa),+ main = "", type = "l", col = 4, ylim = range(kappahdbw,+ kappaklbw, kappancsbw))> lines(bw, kappaklbw, col = 5)> lines(bw, kappancsbw, col = 6)
Return
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 62/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
C. Agostinelli. Inferenza statistica robusta basata sulla funzione diverosimiglianza pesata: alcuni sviluppi. PhD thesis,Dipartimento di Scienze Statistiche, Universita di Padova, 1998.
C. Agostinelli. Robust estimation for circular data. Working Paper2004.3, Dipartimento di Statistica, Universita di Venezia,Venezia, 2004a.
Z.D. Bai, C.R. Rao, and L.C. Zhao. Kernel estimators of densityfunction of directional data. Journal of Multivariate Analysis, 27:24–39, 1988.
V. Barnett and T. Lewis. Outliers in Statistical Data. John Wiley,New York, 1994.
D. Collett. Outliers in circular data. Applied Statistics, 29(1):50–57, 1980.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 63/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
N. Cressie and T.R.C. Read. Multinomial goodness–of–fit tests.Journal of the Royal Statistical Society, Series B, 46:440–464,1984.
N. Cressie and T.R.C. Read. Cressie–Read Statistic, pages 37–39.Wiley, 1988. In: Encyclopedia of Statistical Sciences,Supplementary Volume, edited by S. Kotz and N.L. Johnson.
D.L. Donoho and R.C. Liu. The“automatic” robustness ofminimum distance estimators. Annals of Statistics, 16(2):552–586, 1988.
D.E. Ferguson, H.F. Landreth, and J.P. McKeown. Sun compassorientation of the northern cricket frog, Acris crepitans. AnimalBehaviour, 14:45–53, 1967.
P. Hall, G.S. Watson, and J. Cabrera. Kernel density estimationwith spherical data. Biometrika, 74(4):751–762, 1987.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 64/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
F.R. Hampel, E.M. Ronchetti, P.J. Rousseeuw, and W.A. Stahel.Robust Statistics: The Approach based on Influence Functions.Wiley, 1986.
D. Hawkins. Identification of Outliers. Chapman & Hall, 1980.
X. He. Robust statistics of directional data: A survey.Nonparametric Statistics and Related Topics, pages 87–95, 1992.
X. He and D.G. Simpson. Robust direction estimation. The Annalsof Statistics, 20(1):351–369, 1992.
H. Hendriks. Nonparametric estimation of a probability density ona riemnnian manifold using fourier expansions. The Annals ofStatistics, 18(2):832–849, 1990.
S.R. Jammalamadaka and A SenGupta. Topics in CircularStatistics, volume 5 of Multivariate Analysis. World Scientific,2001.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 65/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
P.E. Jupp and K.V. Mardia. A unified view of the theory ofdirectional statistics, 1975–1988. International StatisticalReview, 57(3):261–294, 1989.
P.T. Kim. Deconvolution density estimation on so(n). The Annalsof Statistics, 26(3):1083–1102, 1998.
J. Klemela. Estimation of densities and derivatives of densitieswith directioinal data. Journal of Multivariate Analysis, 73:18–40, 2000.
E.M. Knorr and R.T. Ng. A unified notion of outliers: Propertiesand computation. In Proc. KDD, pages 219–222, 1997. Anextended version of this paper appears as: Knorr, E.M., Ng,R.T. (1997) A unified approach for mining outliers. In: Proc.CASCON, pag. 236–248.
D. Ko. Robust estimation of the concentration parameter of thevon mises–fisher distribution. The Annals of Statistics, 20(2):917–928, 1992.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 66/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
D. Ko and T. Chang. Robust m-estimators on spheres. Journal ofMultivariate Analysis, 45:104–136, 1993.
D. Ko and P. Guttorp. Robustness of estimators for directionaldata. The Annals of Statistics, 16(2):609–618, 1988.
R.V. Lenth. Robust measures of location for directional data.Technometrics, 23(2):77–81, 1981.
B.G. Lindsay. Efficiency versus robustness: The case for minimumhellinger distance and related methods. Annals of Statistics, 22:1018–1114, 1994.
K.V. Mardia and P.E. Jupp. Directional Statistics. Wiley, 2000.
M. Markatou, A. Basu, and B.G. Lindsay. Weighted likelihoodestimating equations: The continuous case. Technical report,Department of Statistics, Columbia University, New York, 1995.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 67/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
M. Markatou, A. Basu, and B.G. Lindsay. Weighted likelihoodestimating equations with a bootstrap root search. Journal ofthe American Statistical Association, 93:740–750, 1998.
C. Park, A. Basu, and B.G. Lindsay. The residual adjustmentfunction and weighted likelihood: A graphical interpretation ofrobustness of minimum disparity estimators. ComputationalStatistics and Data Analysis, 39(1):21–33, 2002.
V.R. Prayag and A.P. Gore. Density estimation for randomlydistributed circular objects. Metrika, 37:63–69, 1990.
E. Ronchetti. Optimal Robust Estimators for the ConcentrationParameter of a von Mises–Fisher Distribution, chapter 6, pages65–74. Wiley, 1992. In: The art of Statistical Science, edited byK.V. Mardia.
A. SenGupta and A.K. Laha. The slippage problem for the circularnormal distribution. Australian and New Zealand Journal ofStatistics, 43(4):461–471, 2001.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 68/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
G. Upton. Outliers in circular data. Journal of Applied Statistics,20:229–235, 1993.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 69/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7
These slides are prepared using LATEX, beamer class and Sweavepackage in R . They are compiled with R ver. 2.2.0 running underOS darwin7.9.0 and packages circular ver. 0.3-5, wle ver. 0.9-2.
Claudio Agostinelli – useR!2006 Conference – Wien ver. 1.0 – 16 June 2006 71/ 72
Focus Session on Robust Statistics is Friday, 3:00pm at HS 0.7