..
M-ESTIMATORS AND L-ESTIMATORS OF LOCATION:UNIFORM INTEGRABILITY AND ASYMPTOTICRISK-EFFICIENT SEQUENTIAL VERSIONS
by
JANA JURECKOVADepartment of Probability &Statistics
Charles UniversityPrague 8, Czechoslovakia
and
PRANAB KUMAR SENDepartment of BiostatisticsUniversity of North Carolina
Chapel Hill, NC 27514, U.S.A.
Institute of Statistics Mimco Series No. 1280
APRIL 1980
M-ESTIMATORS AND L-ESTIMATORS OF LOCATION; UNIFORMINTEGRABILITY AND ASYMPTOTIC RISK-EFFICIENT SEQUENTIAL VERSIONS
by
JANA JURECKOVADepartment of Probability &Statistics
Charles UniversityPrague 8, Czechoslovakia
and
PRANAB KUMAR SENlDepartment of BiostatisticsUniversity of North Carolina
Chapel Hill, NC 27514, U.S.A.
ABSTRACT
Sequential M- and L-estimators of location minimizing the risk
asymptotically as the cost of one observation tends to 0 are con-
structed. Their asymptotic risk efficiencies are shown to coincide
with the asymptotic efficiencies of the respective non-sequential
estimators; this enables to construct the asymptotically minimax
sequential M- and L-estimators in the model of contaminacy. The
asymptotic distributions of the stopping times are derived for both
types of estimators. The theorems on uniform integrability and moment
convergence of (non-sequential) M- and L-estimators, developed as the
main tools for the proofs, have an interest of their own.
AMS 1970 Subject Classifications: Primary 62L12; Secondary 60F25
Key Words &Phrases: M-estimator, L-estimator, asymptotic risk-
efficiency, sequential point estimator, moment convergence.
lResearch of this author was supported by the National Heart, Lungand Blood Institute, Contract NIH-NHLBI-7l-2243 from the NationalInstitutes of Health.
-2-
1. INTRODUCTION
Nonparametric sequential point estimation of location has
received considerable attention during the recent past. Ghosh and
Mukhopadhyay (1979) and Chow and Yu (1980) have considered asymptotically
risk-efficient sequential point estimation of the mean of a population
based on the sequence of sample means and variances, while Sen and
Ghosh (1980) have extended the theory to general U-statistics. Sen
(1980) has considered the problem of estimating the location of
symmetric (but unknown) distribution based on a general class of rank
order (or so called R-) estimators and established the asymptotic risk
efficiency of the proposed sequential procedure. In the classical
non-sequential case, the R-estimators form one of the three main
groups of robust competitors of classical estimation procedures; the
other two major groups are formed by M-estimators and L-estimators
[viz., Huber (1973, 1977)]. The theory of asymptotically risk-
efficient (sequential) point estimation based on a broad class of M-
and L-estimators is developed in the current paper. Unifor,m integra
biZity and moment-convergence properties of these M- and L-estimators
playa fundamental role in this context.
Along with the preliminary notions, the proposed sequential point
estimation procedures are outlined in Section 2. Section 3 is devoted
to the study of uniform integrability and moment convergence of the
M-estimators. Parallel results for the L-estimators are considered in
Section 4. These results are then applied in the proofs of main
theorems of Section 5 concerning the properties of the proposed
sequential procedures. In particular, the Section 5 deals with the
asymptotic risk-efficiency and with the asymptotic normality of the
"
•
-3-
allied stopping times. Similarly as in the case of R-estimators
[Sen (1980)], it is shown that the asymptotic risk efficiencies of
sequential estimators coincide with the asymptotic efficiencies of
their non-sequential versions. This among others enables to extend
the asymptotic minimax properties of M- and L-estimators in the
model of contaminacy to the sequential case.
2. THE PROPOSED SEQUENTIAL PROCEDURES
Let {X., i ~ I} be a sequence of independent and identically1
distributed random variables (i.i.d.r.v.) with distribution function
(d.f.) F8
(x) ==F(x -8), xsR == (_00,00), where F (unknown) is symmetric
about 0 and 8 is the unknown location parameter to be estimated. Let
T be a sui table estimator of 8 based on Xl' ... , Xn and assume thatn
2 2 exists for all (2.1 )~e
'J == nE (T - 8) n ~ nO'n n
for some nO (~l) and
2 2'J -+ 'J as n +00, 0 < 'J < 00.
n
We conceive the loss (in estimating 8 by Tn)
2~ (a, c) == a(Tn - 8) + cn,
(2.2)
(2.3)
where a and c are positive constants. Then the risk is
-1 2An(a, c) == E~(a, c) == n a'Jn +cn. (2.4)
We like to minimize (2.4) by a proper choice of n. The optimal choice
of n generally depends on the unknown F, for any fixed c as well
as asymptotically as c 1- O. In this asymptotic case, the optimal
and A ()(a,c)~2'J&nO c
lim q(c)/r(c) ==1. This suggestsdObe a sequence of estimates of 'J
choice of n is no(c), where!<
no(c) ~b'J, b == (a/c) 2
where q(c) ~r(c) denotes that
following procedure: Let {Vn}
and let n' be an initial sample size (~2) and h(>O) be an
(2.5)
the
arbitrary constant. Define
N =min{n ~n':c
-4-
a stopping number N(= N )c" -h }n~b(v +n ) , c>O
n
by
(2.6)•
based on Xl, ... ,XN .c
and consider the sequential point estimator TNc
The risk of estimating 8 by TN is thenc 2
A*(a, c) = aE(TN -8) +cENc (2.7)c
We are primarily interested in showing that
A*(a,c)/A ()(a,c)-d as c+o, (2.8)nO c
which means that the sequential procedure is asymptotically (as c +0)
equally risk-efficient as the optimal non-sequential one, if v were
known.
The convergence (2.8) has been studied by more authors [referred
to in Section 1] in the case that {T }n
is either the sample mean,
U-statistic or some case of R-estimator. In the current paper, we
shall show that (2.8) holds for a broad classes of M-estimators
and L-estimators (i.e., the estimators of maximum-likelihood type and
of linear combination of order statistics type, respectively).
An M-estimator M of 8 is a solution of the equation·n
(2.9)
with respect to t, where ~ is some nondecreasing score function
(so that S (t)n
is \ in t) . More precisely, is defined by
where
M = (M* +M**)/2,n n n
M* = sup{t: S (t) >O} and M** =inf{t: S (t)<O}.n n n n
(2.10)
(2.11 )
Under suitable regularity conditions on ~ and on F [viz., Huber
(1964)], to be specified later on,
1 2L{n~ (Mn - 8) } ~ N(0, v (M)) as n -T 00
where
(2.12)
-5-
V~M) =cr~M/Y (M)' cr~M) = flO1jJ2 (x)dF (x) ,_00
00
Y(M) =y(1jJ, F) = f {-f'(x)/f(x)}1jJ(x)dF(x) (>0)
(2.13)
(2.14)
d d2
and f' (x) =dxf(x) =~ (x) is assumed to exist almost everywhere.dx
In Section 3, we shall show that (2.1) and (2.2) hold for M-estimators
generated
estimate
by a class of bounded 1jJ-functions.
2V(M) as follows. Let
2 -1 n 2s (M) =n L· l1jJ (X. - M ), n ~ 1,n 1= 1 n
In this case, we shall
(2.15)
let <P be the standard normal d.f. and let <P(-TE) =E, O<E<l.
M sup{t: -~net) >To./ 2Sn (M)}= n Sn
M+ inf{t: -~n (t) < -To./ 2Sn (M)}= n Sn
Put
(2.16)
where 0.(0 <a < 1)
+ -dn(M) =Mn -Mn(~O), (2.17)
is some preassigned number. Then, it follows from
Jure~kova (1977) that as n increases,
pvn(M) =v'n dn (M/ 2To./2 -+ v(M) =cr(M/Y(M); (2.18)
in fact, stronger convergence properties of vn(M) have been studied
v "by Jureckova and Sen (1980 a, b). The stopping number defined by
" " (M)(2.6), corresponding to {Vn} = {Vn(M)} is denoted by Nc ' so that
(M) ,,-h }Nc =min{n ~n': n ~b(Vn(M) +n )
and we shall show in Section 5 that (2.8) holds for
The L-estimator Ln of location e
(2.19)
{l\1 (M)}.N
is typically c of the form
L = L~ lC'X .n 1= n1 n,l
where Xn,l ~ ... ~ X are the order statistics corresponding ton,n
Xl' ... ,Xn and
(2.20)
-6-
c . = c . 1 ~ 0, V 1 ~ i ~ n, and \~ 1c . =1.nl n,n-l+ ll= nl (2.21 )
Denote
J (t)n
= n.c . for i - 1 < t ~!. . 1nl n n ' 1 = , ... ,n (2.22)
and suppose that
J n (t) -+ J (t) a. S., t E: (0, 1), J (1 - t) = J (t) ~ 0, (J (t) dt = 1.o (2.23)
Then, under some regularity conditions [viz., Huber (1969)],
(2.24)
where0000
V~L) =J f [F (x i\ y) -F (x) F (y) ]J (F (x) ) J (F (y)) dxdy (2.25)
We proceed to estimate V~L) by
n-l n-lv2
(L) = L L c .c .{n(i i\j) -ij}(X . 1 -X .)(X . 1 -X .), (2.26)n i=l j=l nl nJ n,l+ n,l n,J+ n,J
where under suitable regularity conditions [viz., Sen (1978)]/\
as n-+ oo (2.27)
asymptotic normality results pertaining to the vn(L) are due to
Gardiner and Sen (1979). The stopping number, defined by (2.6), for
is denoted by so that
(2.28)N?) =min{n ~n': n ~b(Vn(L) +n-h)}.
We shall show in Section 5 that (2.8) holds for {L (L)} correspondingN
to a class of J-functions which vanish outside of a c compact
subinterval of (0, 1) .
In the remaining of this section, we state the basic regularity
conditions on F, ~ and J, pertaining to our study. Sections 3 and
4 are devoted to the study of the uniform integrability and the moment
convergence of {Mn} and {Ln}, respectively; these results are used
-7 -
in Section 5 in the proofs of (2.8) for the corresponding sequential
procedures.
Assumptions 'on F: We assume that F has an absolutely continuous
density f such that f(x) =f(-x), V xe:R and
f(x) is \ in x for x ;:::0. (2.29)
Moreover, F is supposed to have finite Fisher information, i.e.,00
reF) =f {f'(x)/f(x)}2dF (x) <00 (2.30)
and we assume that there exists a positive number l (not necessarily
an integer or ;::: 1), such that
(2.31)
.e_00
Assumptions on~: We assume that ~ is nondecreasing and skew-
symmetric, i.e.,
~(x) = -~(-x) is ! in x e:R+ = [0, 00),
and that there exists a positive number k such that
~(x) =~(k) ·sign x for Ixl ;:::k.
Moreover, suppose that ~ could be decomposed in the absolutely
continuous and step components, i.e.,
~ (x) =~l (x) +~2 (x) 'I x e: R
(2.32)
(2.33)
(2.34)
where ~l (x) is absolutely continuous [inside (-k, k)] and ~2 is
pure step-function having a finite number of jumps inside (-k, k);
we denote the jump-points by a. ,J
and let for
a. 1 <x<a.,J - J
where a =-ko and a 1 =k.m+ Then the
constant Y(M) defined in (2.14) is equal to00
YeM) =f ~i(x)dF(x) +I;=l((3j -Sj_l)f(aj ) >0._00
(2.35)
-8-
Put
£.c£.(x) = Ixl F(x)[l -F(x)], x e:R; ct=sup C,e(x)
xe:R
where £. is given by (2.31). Then
(2.36)
c* < 00£. ' lim c£. (x) = 0
x-+±ooand
We assume that
c.=c '1;?:0,nl n,n-l+1
(0 <0.0 <2)
and that there exist an 0.0
such that
c . =c . =0nl n,n-l+l
Moreover, denoting
for i s k wheren as n -+00. (2.38)
i - 1 iJ (t) =n'c . for --<t Sn nl n n'
we assume that
i=l, ... ,n (2.39)
lim J (t) = J (t ) a . s ., t e: [0, 1]nn-+oo
where the function J(t) has bounded variation on [0, 1] and
J(t)=J(l-t);?:O, te:[O,l], fJ(t)dt=l.
oIn the context of L-estimators, the assumption of finite Fisher
information may be replaced by a weaker assumption
sup If' (x) I < 00 •
-1 -1F (aO)SXSF (1-0.0)
3. MOMENT CONVERGENCE OF M-ESTlMATORS
(2.40)
(2.41)
(2.42)
Uniform integrability and some moment-convergence properties of1.<
{n 2 (M -e)} are studied here. The following lemmas are needed in then
sequel but they have an interest of their own.
-9-
LEMMA 3.1. Under the regularity conditions on ~ and F of Section 2~
for every
(3.1 )
where
(3.2)
PROOF. Note that for every t > 0,
Po{ /Ill Mn I > t} = Po{m Mn > t} + Po{vn Mn
< - t} = 2Po{m Mn > t}, (3 .3)
where, by (2.9) - (2.11),
PO{m Mn
>t} ~po{n-1Sn(t//Il) ~O}
= Po {n -1 S (tim) -)l (t) ~ -)l (t)}, (3 .4)n n n
and
- )In(t) = -Eon- 1Sn (tlrn) = -EO~(X1 - (tim))
= EO[~(X1) -~(X1 - (tim))] =r)()~(X)d[F(X) -F(x + (tim))]
= j""[F(X + (tim)) - F(X)]d~(X)oo_00
J
k .= [F(x + (thin)) -F(x)]d~(x)
-k
= r[F(X + (tim)) -F(x) +F(-x + (tim)) -F(-x)]d~(x)o [as ~(x) +~(-x) =0, 'V x]
= r[F(X + (tim)) -F(x - (t/rn))]d~(x) [as F(x) +F(-x) =1, V x]
ok
= (2tlm)J f(x+(et/rn))d~(x) [where lei <1]
o
? (2t/m)f(k + (t/rn))[~(k) -~(O)] [as f(x) is \ in x, x ~ 0]
[as ~(O) = 0]? ( 2t I /Il) f (k + c 1) ~ (k), 'V t E: (0, c /n]!,;
= (2c2
) ~ (k) (tl In) .
Therefore, by (3.3) - (3.5), for every O<t <c/n,
(3.5)
where
-10-
II In !.:PO{lnM >t}$2Po{-L· lZ, ~(2c2)~l/J(k)(t/ln)}n n 1= n1 (3.6)
2l/J(k) with probability
Z . =l/J(X. - .-!..) - E l/J(X. _.-!..)n1 1 rn Olin'
are independent r.v. with mean 0, bounded by
i =l, ... ,n (3.7)
1. Hence, using Theorem 2 of Hoeffding (1963) on r.v.· (3.7), the
desired result follows from (3.6). Q.E.D.
LEMMA 3.2. Under the regularity conditions on l/J and F of Section 2~
for every t > 0,
(3.8)
where
m = I-n; IJ and ° =1n for n = 2m, ° =0 for n = 2m + 1, m~ 1. (3.9)n
PROOF. We consider only the case n = 2m; the proof for n = 2m + 1 is
analogous. Note that for every t > 0,
PO{/i1"!M I >t}=2PO{v'nM >t}$2PO{X l~-k+(t//i1")} (3.10)n n n,m+
where X $ ~ "$ X are the order statistics corresponding ton,l n,n
Xl'" .,Xn · Since the right hand side of (3.10) equals to that of (3.8)
the proof of (3.8) is complete. Q.E.D.
For any a E: [0, 1], put
p (a) = 4a (1 - a) , so that 0 $ P(a) $ 1.
LEMMA 3.3. For every n(~l) and a >~ ,
( Jfl. n-m-on-l m n n
2n m _ 0n u (1 -u) du $ 2(p(a))
a
where m, on and pea) are defined by (3.9) and (3.11).
(3.11)
(3.12)
PROOF. We shall again prove (3.12) for n = 2m only; the proof for
-11-
n = 2m + 1 is analogous. Note that by repeated partial integration,
the left-hand side of (3.12) reduces to
2L~:~[~)ai(1 _a)n-i ~ 2P{B(n, a) ~ ~}
~2P{~(n, a) - a~ }-a} , (3.13)
where B(n, a) is a binomial r.V. with parameters (n, a). Since
1a >2' by using Theorem 1 of Hoeffding (1963), we may bound the right
hand side of (3.13) by n2 [pea)] . Q.E.D.
We are now in a position to prove the main theorems of this
Section.
THEOREM 3.1. For every r > 0, there exists an n « (0)r
such that,
under the regularity conditions (2.29) - (2.34),
Eo{nr/2IMnlr} <00 " uniformly in n;::n .r
(3.14)
PROOF. Let cl
>k >0, where k is defined by (2.33). Note that
Eo{nr/2IMnlr} = fOOrtr-lpo{rnIMnl >t}dt
o
fOO )rtr-1PO{rn!Mnl >t}dt = I nl + I n2 ,
clm
Then, by Lemma 3.1,
c l In -c t 200 _ct2
f2 r-l J 2 r-lI
nl~ 2ret dt ~ 2r e t dt < 00
a 0
uniformly in n=1,2, .... On the other hand, if we let
nr
= [f] + 1, where l is defined by (2.31)
use (2.37) and Lemmas 3.2 and 3.3, we get
say. (3.15)
(3.16)
(3.17)
-12-
(3.18)
1n2
:5:zrj'X> tr-l[p(F(~k + (tm)))]ndt
clm00
r / 2f r-l, n:5:2rn u [p(H-k+u))] du
c1
{n-n -b} foo
:5:2r(CV(r-l)/i nr / 2 [p(F(-k+cl)}] r [4F(y)(1-F(y))]bdy
C -k1
•
where cl <00 is given by (2.37) and b is any number satisfying
I < 00n2
This completes
Hence,
n +00.o as
1F(cl-k»F(O)=Z' so that
n-n -br(F (c
2- k)) is uniforml y
Finally,
r/2n
and it converges to
C 1 > k , it ho 1d s
and hence,
n ~ nr
P(F(C1
-k)) <1
b > 1/t. Since
bounded for 11 ~ n and converges to 0 as n +00.r
(4F(y)(l-F(y)))bdy <00 by (2.36) and (2.37).fooc
1-k
uniformly in
the proof of the theorem.
LEMt~ 3.4. Under the reguZarity conditions on ~ and F,
where
1 n ~m(M -8) - rn L' l~(X' -8) =0 (n-'I)
11 Y(M) n 1= 1 P
is defined by (2.14).
(3.19)
PROOF. The Lemma was proved in Jure~kova (1980) [Theorem 3.3].
LEMMA 3.5. Under the reguZarity conditions on ~ and F of Section 2,
h I-~\n 1 2r ..+" "1' b"1 +" Lt e sequence n Li=l~ (Xi - 8 '1.-S un'1.-J orm",y '1.-ntegra ",e Jor n = 1,2, ...
and
(3.20)E8In-~L~_1t)J(X. _e)12r~ a2(~)(2r); , r=l, 2, ...1- 1 r! 2
as n ~ 00-, where a (M) is defined by (2.13).
PROOF. Since E8~ (Xl - 8) = 0 and I~ (y) I :5: ~ (k) < 00, V Y £ R, moments of
all orders of t)J(X1
-8) exist. Hence, the result follows directly
from the moment convergence result of von Bahr (1965).
-13-
THEOREM 3.2. Under the reguZarity conditions (2.29) - (2.34),
lim E8 (rn 1Mn _el)2r = (0(M/'Y(M))2r(2r)I/(2rr!)n-+<x>
ho Zds for r = 1 ,2, . .. .
(3.21)
PROOF. It follows directly from the uniform integrability in Theorem
3.1, from (3.19) and from Lemma 3.5. In fact, we need not confine
ourselves to even integer 2r. Since, by the Jensen inequality,
-~ n rIn Ii:lljJ(Xi -8)1 is uniformly integrable for any r' E:[2r -2, 2r],
we may prove on parallel lines that as n -+00,
(3.22)
_00for any fixed real r > O.
LEMMA 3.6. For any E: > 0 and 0 > 0, there exist positive constants
c and nO such that
p{ Is~ -0~1 >d ~cn-l-o, 'rf n ~nO'
PROOF. Let us define
02 -lIn 2s = n . lljJ (X. - 8), n ~ 1.n 1= 1
Since ljJ2(X. -8), i =l, ... ,n, are i.i.d. bounded valued r.v., by1
(3.23)
(3.24 )
Theorem 1 of Hoeffding (1963), for every E: > 0 there exists an n > 0
such that
I
02 21
-2nnp{ sn -00 >E:/2}$2e , V n~1.
Again, by virtue of Theorem 3.1,
I 1 I I 1 ~ -1-0p{ MnI >r;} = p{ rn Mn > T'n d $ cn , 'rf n ~ nO
2By (2.32) - (2.34), ljJ is of bounded variation on R, so that
ljJ2 (y) = ljJ~ (y) + ljJ; (y),
where ljJi, ljJ2 are nonnegative and ljJi is ~ while ljJZ is ! in
(3.25 )
(3.26)
(3.27)
Y E: R. Henc e ,2 2 I 1 ·11 I 1 1 Isup IljJ (y +t) -ljJ (y) 1$ ljJi(y -ZE:) -ljJi(y +ZE:) + ljJz(Y +ZE:) -ljJZ(Y -22 ) .
Itl$~ (3.28)
(3.29)
-14-
Since, by (2.33), ~i and ~2 are bounded, by using (3.28) and the
Markov inequality, we obtain that for every £' >0,
{ 11\,n 2 1\' 2 II} -1-0
P sup -L'-l~ (X. +t) --L~ (X.) >2£' =:; c'n1 n 1- 1 n 1 .
jt\=:;2 E
(3.23) then follows from (2.13), (2.15), (3.24), (3.25), (3.26) and
(3.29). Q.E.D.
THEOREM 3.3. Under the regularity conditions of (2.29) - (2.34), for
every E > 0 and 0 > 0, there exist a C, 0 < C < 00 and an nO « (0)
such that
(3.30)
PROOF. Since ~v ~
is nondecreasing, exploiting technique of Jureckova
(1969), we obtain that for every fixed K«oo) and £ >0, there
exist an m(= mKE
) and a set of points
that
( - K =:;) t 1 < ... < t (=:; K).m
such
!-< !-<sup In- 2{Sn (n- It) - Sn (O)} + ty (M) J
ltl~K
+ max \In 1.1 (t.) + t. Y(M) I + £/2,l~j ~m n J J
where 1.1.(t) is defined by (3.5) andJ
sup Irn 1.1n (t) + ty (M) I -r 0 as n +00,
It\::::k
(3.31)
(3.32)
The random variables
i = 1, ... ,n are independent for any fixed j (= 1,2, ... ,m) and
EZ q) = 0 E(Z q)) 2 = 0 (n-\) ,nl ' n1
•
-15-
Proceeding as in Theorem 3.1 of Jure~kova and Sen (1980b), we claim that
under (2.32) - (2.34),
(3.33)
Thus, by Markov inequality,
1-< 1-<p{ max In- 2s (n- 2t.) -S (0) -rnjl (t.)1 >E'}
l::;;j::;;m n J n n J
::;; I~ I P{ln-12s (n-~t.) -S (0) -rnlJ (t.)1 >E'} =0(n-q/2
) (3.34)J= n J n n J
where, given 0, we may choose q so large that q/2~1+o. From
(3.31), (3.32) and (3.34), we obtain that for every E >0, 0>0,
p{ sup In-~[S (n-\) -S (0)] +t (U) I >d ::;;cn-1 - o , V n ~nO (3.35)It I::;;K n n y .'1
and (3.31) follows readily from (3.23), (2.15) - (2.18) and (3.35).Q.E.D.
4. MOMENT CONVERGENCE OF L-ESTIMATORS
First, we consider the following theorem on a. s. representation
of L .n
THEOREM 4.1. Let L be an L-estimator of the form (2.20) withn
c ., 1::;; i ::;; n,n1
satisfying (2.21) and (2.38). Let d.f. F have
the absolutely continuous symmetric density satisfying (2.29) and
(2.41). Then> POI' any n ~ nO' there exists a sequence {Y. }~+1J' n1 1=1
of i.i.d. random variables with standard normal distribution such that
L _~
In'2(L - 8) - (n + 1)n
n+l 1
I a .Y . I = 0 (n -~log n) asj=l nJ nJ a.s.
n -roo , (4.1)
(4.2)
where
nI
,n+l 1 -1 ia . = b. - L' 1 --1b ., b. = c . / f (F (--1) ), 1::;; i ::;; n .nJ . . nl 1= n + n1 n1 n1 n +
1=J
PROOF. We may put 8 = 0 without loss of generality. Note that by
(2.29) and (2.41), for every
-16- .
sup {F(x)[1 -F(x)]o\f 1 (x)/f2
(x)l}-1 -1
F (B) <x<F (I-B)(4.3) •
2 -1 I I~ [4/f (F (B))] sup flex) ~YB<oo
F-1(S)~x~F-l(I-B)
where YS
(> 0) may depend on B. As such, the condition (3.2) in
Theorem 6 of CxBrgB and R6vesz (1978) holds for the case of
-1 -1F (S) ~ x ~ F (1 - S), and hence, we may virtually repeat the proof of
their lbeorem 3 [using our (4.3) instead of their more stringent (3.2)]
and claim that for every n(~ nO)' there exists a Brownian Bridge
{B (t): O~t~l} such thatn
1;sup jf(x)q (x) -B (F(x))1 = O(n- 210gn) (4.4)
-1 -1 n n a.s.F (B) ~x~F (I-B)
where
J2 -1 i-Iq (x) = n [X . - F (F(x))] for-- < F(x)
n n,1 n1 ~ i ~ n. (4.5)
By (4.4), (4.5), we have for n-+ oo
max jrn[X. -F-1(~1)] -B(i/(n+l))f(F-1 (i/(n+1))!k 1<' < -k n, 1 n + .+ -l-n 1
n n -~= O(n 10gn), (4.6)a.s.
so that by (2.20), (2.21) and (2.38),
IrnL -L~lb.B(i/(n+l))1 = O(n-1z10gn) (4.7)n 1= n1 n a.s.
t E:R+} is a standard Wiener process on R+,
of i.i.d. random variables with them
standard normal distribution such that W (m) = L Y ., m = 12, ...n i=l n1
with b. given by (4.2).n1
{W (t) = (t + 1) B (-t~I):n n +n+1
thus there exist {y .}. 1n1 1=
Therefore,
Ii1+T B (~1) = rn+1 [W (.-2-1) - n +i lWn (1)]nn+ nn+
i . n+1= '\ Y . __1_ '\ Y
L. nJ n + 1 . L. nJ"j =1 ]=1
(4.8 )
i=1,2, ... ,n,
-17 -
so that
• \,n B (_1_' ) = 1 \,n b [\,i Y _1_' \,n+1 y ]L.i=l n n + 1 rn+T L.i=l ni L.j =1 nj + n + 1 L.j =1 nj
(4.1) then follows from (4.7) and (4.9). Q.E.D.
1 n+1= rn+I I a .• y .,
n + 1 j =1 nJ nJ
(4.9)
LEMr~\ 4.1. Under (2.21), (2.38) - (2.41) and (2.42), it hoZds for any
positive integer r,
1 · E[ 1 \,n+1 y ]2r 2r (2r)!1m " L. a· = v • --~ .~ j=l nj nj (L) 12rn r.
where v~L) is defined by (2.25).
(4.10)
PROOF. Note that -~\,n+1(n + 1) . L.' 1a .• Y .
J = nJ nJhave a normal distribution
with mean zero and variance
-l In+1 2 *2(n+1) "laO V Ln ' say.J = n1
To prove (4.10), it thus suffices to show that
. *2 2llmvLn =v(L)'n~
but it readily follows from (4.2), (2.21), (2.38) - (2.40).
(4.11)
(4.12)
LE~~ 4.2. Under the assumptions of Therorem 4.1, for any positive
integer r, there exists a C > 0 and an integer n such thatr r
EO(1n L )2r:o;C <00 'V n ~n . (4.13.)n r rn l'
PROOF. Regarding (2.20) and the fact that .I cniF- (n ~1) =0, we1=1
get by Jensen inequality that under e = 0,
(InIL - eI)2r = (rnIL I) 2r :0; ( I c ". rnIX . _ F-1 (_i_) I)2rn n i=l n1 n,1 n + 1
n-k .\' n (rnIX _ F-1 (_1_) I)2r
$ L.i=k +lc ni n ni n + 1n
so thatn-k .
Eo(rniL 1)2r $L. kn 1c .E(lnlx . -F-1(~1) 1)2r <C* <00n 1= + n1 . n1 n + r
n
holds for n ~ n, as it follows from Theorem 2 of Sen (1959).r
(4.14 )
-18-
THEOREM 4.2. Let L be an L-estimator of the form (2.20) with then
coefficients satisfying (2.21), (2.38) - (2.41). Let d.f. F have
the absolutely continuous symmetric density satisfying (2.29) and
(2.42). Then~ for every positive integer r,
lim Ee
[/il (Ln
- e)] 2r = v~~) (;r) !n+oo 2 r!
(4.15)
PROOF. It follows directly from Theorem 4.1, Lemma 4.1 and Lemma 4.2.
Let2 be the asymptotic variance (2.25) of m(L - 8) and\i (L) n
letA2
be its estimator (2.26) . Then we shall need the followingvn(L)
THEOREM 4.3. Under the assumptions of Theorem 4.2, to any E > 0 and
o> 0, there exist C > 0 and nO such that~ for n:2: no'
-1-0p{ Iv~ (L) - v~L) I > d ~ Cn (4.16)
PROOF. Let Fn be the empirical d.f. of Xl' ... ,Xn
. Then by (2.25),
(2.26), (2.39) and (2.40),
-F (x)F (y)]J (F (x))J (F (y))n n n n n n
- [F(x "y) -F(X)F(Y)]J(F(X)J(f(y))}dXdY.
(4.17)
Now, for every n > 0,
I I-2nn2
P{supF(x)-F(x) >n}~2e ,Vn:2:l,xER n
-1 } { -1 } np{X k I <F (aO) -n =P Xn,n-k >F (1 -aO) +n ~ [pen)] ,n, n+ n
(4.18)
Vn2':l,
(4.19)
where 0 < p (n) < 1. Al so, excepting at countably many points (with
Lebesgue measure 0), J(t) has a derivative (with respect to t)
(4.18),while on this compact region,
(4.19) and the fact that c -c =0 V i~k, with probabilityni n,n-i+I' n
we may 'replace the domain R2
in (4.17) by
•
-19-
(2.39) - (2.40) and the boundedness of IJ (t) I lead us to the desired
result when J(t) is continuous inside [aO' 1 -ao]' Next, let us
suppose that J(t) has only a finite number of saltus on [aO' 1 -aO]'
Excluding small neighborhoods of these saltus points, repeating the
above proof and finally using the boundedness of IJ(t) I for these
neighborhoods, the proof follows. Finally, J(t) is a function of
bounded variation, and hence, for any n >0, J(t) can have only a
finite number of points of discontinuities at which its saltus is
greater than n. Therefore, the proof for the case of a finite number
of points of discontinuity extends to that of a countable number. Q.E.D.
5. PROPERTIES OF SEQUENTIAL M- AND L-ESTI~~TORS
Let Xl' X2 ,· .. be i.i.d. random variables distributed according
to the d.f. F(x -8) such that F is symmetric and satisfies.eregularity conditions (2.29) - (2.31). Unless otherwise stated, T
n
will denote either M-estimator generated by the ~-function satisfying
(2.32) - (2.35) or the L-estimator with the coefficients satisfying
(2.21), (2.38) - (2.41). 2v will denote the asymptotic variance of
m(T - 8)n
and its estimator (2.15) or (2.26), respectively. Let
Nc be the stopping variable defined in (2.6) and let TN be thec
estimator based on Xl" ",XN .c
THEOREM 5. 1 • Under regu Zari ty conditions of Seeton 2, for any h > 0
(in (2.6)),
and
pN/nO(c) -->-1 and E(N/no(c))--+l as c i- 0
where nO(c) is defined by (2.5);
InO(c)(TN -8)/v 11+ N(O, 1)c
lim{A*(a, c)/"A () (a, c)} ;::1ci-O nO C
(5.1)
(5.2)
(5.3)
-20-
so that b ~OO as c~O. Then by (26), .,
are given by (2.7) and (2.4), respectively.where
PROOF.
A* (a" c) and
Put b = f~) ~,
N 2 bll (l+h)c with probability 1. (5.4 )
f
For every c > 0 and E: 0 < E < 1, put
nc*=[bl/(l+h)], [()(l)]nlc = nO c - E (5.5)
where we choose c so small that n' Sn* <n <n (c) <n .c Ie 0 2c Then, by
(2.6), Theorem 3.3 and Theorem 4.3 (on noting that nib sV(l -E),
V n S nlc
) , as c to,
P{N $nl
} =p{v Sn/b, for some n: n* $n snl
}c . c nee
sP{v sV(l -E), for some n: n* Snsnl
} (5.6)nee
n$Ln:~*p{lvn -vi >EV} =o[(n~)-o] =0(co/ (2(1+h))) ~o.
cSimilarly, noting that n/b 2v(1 + E), V n 2n2c' we have for n 2n2c'
{A -h }P(N? n) =P m<b(V +m ), V n'Sm$nc m
S p{v -v <n} (where n >0) (5.7)n
$ p{ Iv -vi >n} =O(n- l - o) by Theorems 3.3 and 3.4.n
pThen (5.6) and (5.7) imply that N/n
O(c) --+ 1 as c to. Moreover,
if n $nlc ' then n/nO(c) <1 -E and, by (5.7),
E{Nc·I(Nc >n 2c )}/nO(c) -+- O. as c to, so that (ENc)/nO(c)-l
as c to. Thj s proves (5.1).
Lemma 3.4 and Theorem 4.1 ensure that the distribution of
L 2{n 2 (T -8)} is asymptotically normal, N(O, v) as well as the
n
"uniform continuity in probability" of this sequence in the sense of
Anscombe (1952). Hence, (5.2) follows from Anscombe (1952) theorem.
Finally, to prove (5.3), we may follow the ideas of the proof of
Theorem 3.2 of Sen (1980), where the Lemmas 3.1 -3.6,4.1,4.2 and
Theorem 3.1 -3.3, 4.1 -4.3 of Sections 3 and 4 provide the analogous
'e-21-
tools to apply the same technique in the current context. Hence, for
intended brevity, the details are omitted. Q.E.D,•
and
It hasv
been proved by Jureckov§ and Sen (1980 a, b) that
{d A
L n (Vn(M) -V(M))/S}~ N(O, 1) as n+ oo (5.8)
if1
d=2
1(5.10)
if d=4
sup {ndlvm(M)-Vn(M)\}~O as 0+0 (5.9)m:lm-nj<on
where v(M) and vn(M) are given by (2.13) and (2.18), respectively,
ljJ = ljJl + ljJ2 with ljJl being the absolutely continuous component and
ljJ2 the step-function component and where d =} if ljJ2:: 0 and d =}
if ljJ2 $0, and
{
2 2 2222 _ [(0 /40 (M)) + v 0 1 - (r;/ Y1) ] / 00
S - m 2I· 1(f3· -S· 1) f(a.)
J= J J- J
where
0; = fooljJ4
(X)dF(X) -0~M)' oi = f
oo
(ljJl(X))2dF (X) -Y~~~_00 _00
r; = fooljJ2 (x) ljJ I (x) dF (x) - 0~fvl) Y(M)
_00
(5.11)
(5.12)
(in the case ljJ:: ljJl '(2)
where 0 (M) and Y(M) are given by (2.13)
and (2.14), respectively), and a 1 , ... ,am
are the jump-points of ljJ2
with jumps (S. - S. 1),J J-
THEOREM 5.2. If the constant h in (2.6) satisfies h > d, then,
under the regularity conditions on F and ljJ of Section 2, as c + 0,
PROOF. Note that by (2.6), whenever N >n',. cA A -h
bVN
$ Nc $ b(VN
-1 + (Nc -1) )c c
(5.13)
(5.14 )
so that if we put nOc
= [bv] + 1, we get from (2.5) and (5.14)
-22-
A -hb(v
N-V) :::;N
C-n
Oc:::;b(V
N-1 -v) +b(N
c-1) , (5.15)
c c pwhenever Nc >n'. Now, by Theorem 5.1, Nc/n
Oc--+ 1, while for
d -hd<h, n
oc(N
c-1) -+0 as ci-O, and
-1 d -l+db nac~vnOc as ci-a. (5.16)
Thus, regarding that na(c) ~nac' (5.13) follows from (5.8), (5.9),
(5.15) and (5.16). Q.E.D.
REMARK. Theorem 5.2 shows that, if 1/J2 :: a, the rate of convergence to
the asymptotic normal distribution in (5.13) is faster than in the case
1/J 2 :: a.
Let us now consider the asymptotic distribution of the stopping
variable corresponding to the L-estimator. Gardiner and Sen (1979)
proved
(5.17)
and
•
e.
as o i- a (5.18)
where are given by (2.25) and (2.26), respectively, and
with
2 f1fl -1 -1K =J (Si\t-st)La(s)Lo(t)dF (s)dF (t)
o a(5.19)
LO
(t) = Ll (t).J (1) (t) - L1
(1 - t)J (I) (1 - t), o :::; t :::; 1 (5.20)
THEOREM 5.3.
J(l)(t) =t·J(t),
and
L1 (t) = 2r-t uJ (u) d F-1 (u) ,
oIf the constant h in (2.6) satisfies
(5.21)
then~
under the Y'egularity conditions on F and J of Section 2~ as c i- 0,
(5.22)
..
-23-
PROOF. (5.22) follows from (5,17) and (5.18) similarly as in the proof
of Theorem 5.2 .
6. ASYMPTOTIC MINIMAX PROPERTY OF SEQUENTIALM- AND L-ESTIMATORS
Let Xl' X2 , ... be a sequence of i.i,d. random variables distri
buted according to d.f. F(x -8) where F is symmetric but generally
unknown; F is only supposed to belong to an appropriate neighborhood
F of a given d.f. G. Suppose that the loss incurred in estimating
8 by Tn
is given by (2.3). Let T denotes the set of sequences
{T} of translation equivariant estimators, asymptotically normallyn
.e
distributed and such that the minimum asymptotic risk
(2.5) exists and satisfies
limp2 ( )/(4ac)] =,}(Tn , F) 'r/ FE: FctO nO c
>. () (a, c)nO cin
(6.1 )
where 2v (T , F)n
is the asymptotic variance of if F (x - 8)
is the underlying d.f., and for which there exists sequential point
estimation procedure TN with the risk satisfyingc
lim[>.*(c)/>. ()(a, c)] =1.c+O nO c
Then we may consider the limit
rce(Tn , F) = lim >.*(c)
dO
(6.2)
(6.3)
as a measure of efficiency of the sequential point estimator TN ifc
F is the underlying distribution.
Similarly as in the non-sequential estimation procedures, for an
appropriate family F of distributions, there may exist an M- or
L-estimator providing the saddle-point of the function (6.3) over
T xF. We shall formulate such result for the case that F represents
-24-
the contaminated distribution G; it is an extension of the Huber's
(1964) result to the sequential case. •Let F* be the set of distribution functions
1
F~ ={F = (1 -s)G +sH: H sH} (6.4)
where S'C [0,1) is a fixed number, G is a symmetric d.f. which has
twice continuously differentiable density g, g is strongly unimodal,
I(G) <00 and ] Ix\idG(x) <00 for some i > 0, while H is the set of
all absolutely continuous symmetric d.f.'s with f\x1idH(X) <00 for
some i >0. Then, for the M- and L-estimators considered in this paper,
(6.1) -(6.2) hold for FsFi. (6.1) -(6.2) also hold for the sample
mean provided we assume that F sF;, where F; = {F = (1 - s)G + sH: H s HZ}
and H; =(I-I sH: ]X2dH(X) <oo} while G satisfies the same condition as
in (6.4). Further, for translation-equivariant U-statistics, (6.1) -
(6.2) hold [c.f., Sen and Ghosh (1980)] for
where H*3
is the class of all d.f.'s
for which the kernel (generating the U-statistics) has finite i-th
absolute moment for some .e. > 2). Finally, (6.1) - (6.2) hold for a
general class of R-estimators [c.f. Sen (1980)] with absolutely
continuous (but possibly unbounded) score function, provided
where F*4 is a sub-class of .F~
F sF;-sfor which sup f (x) (F (x) [1 - F (x)]) < 00,
x
for some s >1/6. It is easy to verify that the intersection of F~
F*4
is a non-null set.
be the set of distributions defined in (6.4)THEOREM 6.1.
and let Tl
Let F*1
be the set of sequential estimators satisfying (6.1)
and (6.2) for F s Fi . Then there
Tel) and a sequential L-estimatorN
c
exist a sequential M-estimator
(2) d F*TN an FOE 1 such thatc
,
-25-
e(TN
' FO) ~e(T~i), FO) = [I(Fo)]~~e(T~i), F)c c c
ho Zds faY' i =1 ,2, V TN £ T1 and F £ Fi . .c
(6.5)
PROOF. It follows from J-Juber (1964) that the asymptotic variances of
M-estimators with the ~-functions satisfying (2.32) - (2.34) have a
saddle-point which corresponds to the density
(1 _E)g(k)eq(x+k), x <-k
fo(x) = (l - E)g(X) , -k ~ x ~ k (6.6)
(l - E)g(k)e-q(X+k), k~x
and the corresponding minimax M-estimator is the maximum-likelihood
estimator corresponding to fO' i.e.,
xER, i.e., (6.7)
where q = -
-q
g I (x)g(x)
q
and k
x < -k
-k < x < k
k<x
is related to E according
(6.8)
(6.9)
It follows from Theorem 5.1 that the sequential M-estimator
corresponding to ~O and to the stopping rule (2.19) is the
of (6.5).
T(l)N(M)
csolution
Moreover, it follows from Jaeckel (1971) that the L-estimator
T(2) corresponding to the weight function.n
(6.10) .
is asymptotically equivalent to
(2) .T (L) with the stopping rule
Nc
alternative solution of (6.5).
-26-
T(l)n •
N (L)c
The sequential L-estimator
given by (2.28) is then an •
•
REFERENCES
ANSCOMBE, F.J. (1952). Large sample theory of sequential estimation.Froc. Camb. PhiZ. Soc. ~~, 600~607 .
CHOW, Y.S. and YU, K.F. (1980). The performance of a sequentialprocedure for the estimation of the mean. (Unpublished),
CSORGO, M. and REVESZ, P. (1978). Strong approximations of thequantile process. Ann. Statist. 2, 882-894.
GARDINER, J.C. and SEN, P.K. (1979). Asymptotic normality of avariance estimator of a linear combination of a function oforder statistics. Zeit. Wahrsch. Verw. Geb. ~Q, 205-221.
GHOSH, M. and MUKHOPADHYAY, N. (1979). Sequential point estimationof the mean when the distribution is unspecified. Comm.Statist. A2' 637-652.
HOEFFDING, W. (1963) .random variables.
Probability inequalities for sums of boundedJour. Amer. Statist. Assoc. ~~, 13-29.
HUBER, P.J. (1964). Robust estimation of a location parameter.Ann. Math. Statist. ~2' 73-101.
HUBER, P. J . (1969). Theorie de l' inference statistique robuste.Seminaire de mathematiques superieures. Universite de Montreal.
•HUBER, P.J. (1973) .
and Monte Carlo .Robust regression: Asymptotics, conjecturesAnn. Statist. 1, 799-821.
IIUBER, P.J. (1977). Robust methods of estimations of regressioncoefficients. Opert. Statist. Ser. Stat. ~, 41-53.
JAECKEL, L.A. (1971). Robust estimates of location: Symmetry andasymmetric contamination. Ann. Math. Statist. ~~, 1020-1034.
JURECKOVA, J. (1969). Asymptotic linearity of a rank statistic inregression parameter. Ann. Math. Statist. ~Q, 1889-1900.
JURECKOVA, J. (1977). Asymptotic relations of M-estimates andR-estimates in linear regression model. Ann. Statist. ~, 464-672.
v ..JURECKOVA, J. (1980). Asymptotic representation of M-estimators of
location. Opert. Statist. Ser. Stat. 11, (in press) .
JURECKOVA, J. and SEN, P.K. (1980a). Invariance principles forsome stochastic processes relating to M-estimators and theirrole in sequential statistical inference. Sankhya~ Series A(under consideration for publication).
-28-
JURECKOVA, J. and SEN, P.K. (1980b). Sequential procedures based onM-estimators with discontinuous score functions, Institute ofStatistics Mimeo Series No. l279. The University of NorthCarolina, Chapel Hill, NC.
SEN, P.K. (1959). On the moments of sample quantiles. CalcuttaStatist. Assoc. Bull. 2, 1-19.
SEN, P.K. (1978). An invariance principle for linear combinationsof order statistics. Zeit. Wahrsch. Verw. Geb. ~£, 327-340.
SEN, P.K. (1980). On nonparametric sequeritial point estimation oflocation based on general rank order statistics. Sankhya~ SeY'. Ai£ (to appear).
SEN, P.K. and GHOSH, M. (1980). Sequential point estimation ofestimable parameters based on U-statistics (under considerationfor publication).
von BAlm, B. (1965). On the convergence of moments in the centrallimit theorem. Ann. Math. Statist. ~£, 808-818.
•