Likelihood Ratio Test for Order Restrictions against all ...rms.scu.ac.ir/Files/Articles/Journals/Abstract/... · Abouzar Bazyari, Rahim Chinipardaz and Ali Akbar Rasekhi Department

International Journal of Statistics and Systems ISSN 0973 - 2675 Volume 6, Number 1 (201 I), pp. 41-56 O Research India Publications http://www.ripublication.com/ijss.htm

Likelihood Ratio Test for Order Restrictions against all Alternatives in Multivariate Normal Distribution

Abouzar Bazyari, Rahim Chinipardaz and Ali Akbar Rasekhi

Department of Statistics, Shahid Chamran University, Ahvaz, Iran

Abstract

The problem of testing the isotonic of several p -variate normal mean vectors against all alternatives is considered. This is a multivariate extension of Robertson and Wegrnan (1978). In the present paper, two cases are considered. First, it is assumed that the covariance matices are known and second that they have an unknown scale factor. For both cases, we propose the test statistic, critical values and estimate the power of tests. The p -values are obtained by simulation study.

Keywords: Isotonic regression, Likelihood ratio test, Multivariate normal distribution, Simulation.

Introduction Suppose that X,, , X,,, . . . , X,, are random variables fiom a p -variate normal

distribution with mean vector p, and nonsingular covariance matrix Z, , N , (p, ,Z, ) , v = 1,2,. . . , k . Throughout this paper except in section 5, Z, is assumed to be known. . Consider the likelihood ratio test for testing

against all alternatives, where p, I p j means that all the elements of p - p, are

non-negative. Let H, be the hypothesis placing no restriction on p's. Then it is desired to test

the hypothesis HI against the alternative hypothesis H, - H, . Such tests may be used in some fields. Their applications can be found in clinical

trails design to test superiority of a combination therapy (Laska and Meisner, 1989

Abouzar Bazyari et a1

and Sarka et al., 1995). More examples may be found in the case of ordered treatment means or the testing in which a treatment is better than control when the responses are ordinal.

The extensive literature concerning this problem has been appeared by Barlow et al. (1972), Robertson et al. (1988) and Silvapulle and Sen (2005). The last reference includes some applicable examples as well as some new methods considered by authors in this area.

Such tests started by Bartholomew (1959a, b) who derived a likelihood ratio test for homogeneity of k univariate normal means against ordered alternatives. Chacko (1963), Shorack (1967) and Barlow et al. (1972) followed the problems with ordered parameters. Kudo (1963) considered a p -dimensional normal distribution with unknown mean p = (4, p2 , . . . , pp ) and known covariance matrix E . The problem of

testing was H, : p = 0 against the restricted alternative H, : p, 2 0 (i = 1,. . . , p), where the inequality is strict for at least one value of i . He obtained a statistic based on the likelihood ratio criterion and discussed its existence and geometric nature and also give a scheme for its computation.

Perlman (1969) studied this problem assuming that C is completely unknown. Kudo and Choi (1975) generalized this work where the alternative hypothesis is that the mean vector lies in a convex polyhedral cone. Sasabuchi (1980) considered the problem that the mean vector lies on the boundary of a convex polyhedral cone against the mean vector corresponds in the interior. It was a complete generalization of Inada (1978) who studied it for the bivariate case (see also Sasabuchi, 1988a, b).

The problem of testing the homogeneity of several multivariate normal means against the unrestricted alternative hypothesis was given by Anderson (1984). Sasabuchi et al. (1983) extended Bartholomew's (1959a) problem to multivariate normal mean vectors with known covariance matrices. They obtained the likelihood ratio test statistic and proposed an iterative algorithm for computing the bivariate isotonic regression. Kulatunga and Sasabuchi (1 984) studied its null distribution only in some special situations. Kulatunga et al. (1990) proposed some test procedures when the covariance matrices are not diagonal, and studied them by simulation. Sasabuchi et al. (1992) generalized the iterative algorithm to multivariate isotonic regression.

Sasabuchi et al. (2003) considered this problem of testing in case that the covariance matrices are common but unknown. He proposed a test statistic, studied its upper tail probability under the null hypothesis and estimated its critical values. Sasabuchi (2007) then gave some more powerful tests than Sasabuchi et al. (2003). In all cases, the authors examined the hypothesis

which seems to be easier than given in (1). In fact, the hypotheses given in this paper are the multivariate versions of those

given by Robertson and Wegman (1978) who obtained the likelihood ratio test statistic for testing the isotonic of several univariate normal means against all

Likelihood Ratio Test for Order Restrictions

alternative hypotheses. We are interested in a multivariate extension of Robertson and Wegman's (1978) problem in both cases when the covariance matrices are known and when they have an unknown scale factor. We obtain the test statistic based on the likelihood ratio method, the critical values and power of test in both cases. Our results include the results given by Robertson and Wegman (1978) as a special case.

The paper is organized as follows. In section 2, we review the results given by Barlow et al. (1972) and Sasabuchi et al. (1983). We describe the problem, compute the likelihood ratio test when the covariance matrices are known and study its null distribution.

In section 3, for bivariate and tivariate normal distribution, the critical values are given. The power of test for different values of significance levels and its exact p -value are estimated by simulation in section 4. In section 5, we obtain the likelihood ratio test when the covariance matrices have an unknown scale factor.

Review the results and likelihood ratio test Suppose that X,,,X,,,, . . .,X,,, are random variables from a p -variate normal

distribution with mean vector p, and nonsingular covariance matrix 2, , N , (p,, , Z, ) , v = 1,2,. . . , k . Throughout this paper, except in section 5, C, is assumed to be known.

Definition (Sasabuchi et al., 1983). Given p-variate real vectors X, , X, , . . . , Xk and

p x p positive definite matrices El , E :, , . . . , C , a p x k real matrix ( fil , fi, , . . . , fi, ) is said to be the multivariate isotonic regression (MIR) of X, , X,, . . . , Xk with weights

zy l , c;', . . . ,c,' if (fi, I k, I . . I fi,) and (fi, , fi, , . . ., fi,) satisfies

where fiv's can be computed with iterative algorithm proposed by Sasabuchi et al. (1983).

In fact, this definition includes the definition given by Barlow et al. (1972) for univariate variables.

Now, consider the problem of testing

Hl :pl I p 2 I . . . I p , versus H, -H, ,

where pv S p means that all the elements of pj - pv are non-negative.

The likelihood ratio test for testing H, against H, - H, is based on

Abouzar Bazyari et al

A likelihood ratio test rejects H , for small values of h.

Theorem 1. Suppose that C, is known. The likelihood ratio test for H, against

H , - H, rejects H, for the large values of

where ff, is the maximum likelihood estimate of y, under the alternative

hypothesis and ( b l , b2,. . . , b k ) is the multivariate isotonic regression (MIR) of - - - 1 X 1 , X 2 , . . . with weights n , ~ ; ' , n 2 z i 1 , . . . , n , ~ ; ' and Rv = -CX, ,

nv j=l

Proof. With some easy manipulations we have

where X, and pv are as before.

Simplifying the second term


and hence we have

The distribution of the T statistic Without the loss of generality, for the simplicity, we suppose that n, = 1 for

v = 1,2,. . ., k , then the likelihood ratio test is

where Xv is a column vector for v th population.

Consider another hypothesis: H, : p, = p2 = ...= p k .

Then it is clear that H, c H, c H,. In fact, H, is the least favorable among

hypotheses satisfying H1 with the largest type I error probability. Suppose that Cv is diagonal with diagonal elements ovl , o,, , . . . , ovp , v = 1,2,. . . , k .

Then the likelihood ratio test statistic leads to

where ,Cii,p2,, . . . ,P, is the univariate isotonic regression of XI,, X,,, . . ., X, with -1 weights q l , 4,. . ., ob .

The distribution of T when Cv is diagonal The T statistic can be written as


The null distribution of T with this form is given by the following theorem.

Theorem 2. If H, be true, then for t > 0 , the null distribution of T is

where 1 = $1, . i=l

Proof. Robertson and Wegman (1978) showed that when H , is true the distribution

of (T,2)i for i = 1,2,. . . , p is given as

where P(li, k ) is the probability that the isotonic regression function ,Lvi takes

exactly li distinct values and we note that

We can write

Let T z O .


Since (TI,), given I , has x2 distribution with k - I i degrees of freedom, so given

L = I , . . . L = 1 , ( ) is distributed as x2 with ( - 1 degrees of i=l i=l

freedom. For T = 0,

Application for two populations To illustrate this, we consider first the case k = 2 . Then we have

Now, for tests of significance the probability is

2

= p(Xl i > x ~ , ) . P ( ~ 0; (xi - x , ) ~ 2 t ) . &=I

1 Under the hypothesis H , : p, = p2 , P(Xl i > X, , ) = - . The distribution of

2 2

o 1 ( - x i 2 is x2 with one degree of freedom. So, if H, be true, we have v=l

finally

1 P((T,2)i 2 t ) = - P(X: 2 t ) .

2

For larger values of k the method is similar.


Therefore, we have

If the probability P(L, = 1,) is found, then the distribution of T will be determined. To compute these probabilities see Barlow et al. (1972).

Theorem 3. Suppose that s are common and diagonal. If H , be true, then

where Q(l,k,p) is the convolutions of the probabilities P(1,k) used in the univariate order restricted inference and defined as

Q(1, k, p) = P(L, = I,, . . . , L, = I,) by Kulatunga (1984). I, +. . .+I,=/

Critical values of the test statistic In this section we compute the critical values of the test statistic, T , for some of the bivariate and trivariate nonnal distributions. These values are given in tables 3.1 and 3.2.

Table 3.1. Critical values of the test statistic, T , for p = 2 .


Table 3.2: Critical values of the test statistic, T , for p = 3 .


Simulation study to compute the power and p -value This section is devoted to a simulation study to estimate the power of test in bivariate normal distribution. The results may be generalized to more dimensional normal distributions. Consider the simple order Hl : pl 5 p, - < I pk . Use the critical values from the table 3.1 for k = 2,3 and two significance levels a = .05 and a = .01. We suppose that the covariance matices are diagonal. Based on 1000 replications of multivariate normal the power is calculated. In both cases the mean vectors are p, =(/%,/v+l), v = 1 , 2 ,..., k, k = 2 , 3 and

Table 4.1: Simulation of power for the test statistic, T . significance level, a = .05, .O1 and alternatives p, = (/%, f i + 1) .


Following the simulation method for computing the exact p -value of the test statistic, T , for testing H, against H , - H, :

1) Generate independent observations from bivariate normal N2 (p, , C,) , where

(po,Co) can have any arbitrary values and the covariance matrices are diagonal.

2) The T statistic is estimated. 3) 'Repeat the previous two steps N = 10000 times, and estimate the p -value by

M - where M is the number of times the simulated value of T statistic in the N

second step is not less than its sample value. In this simulation study, to estimate the p -value of the test, we consider the

3 1 1 1 1 mean vectorsp, = ( b , b + l ) , P=3,2,-,I,-,- ,... - - 1 ,..., -, v=1,2 ,..., k and

2 2 3 '10'20 70 k = 2,3 . The results are given in table 4.2.

Table 4.2. Simulation of p-value for the test statistic T .

Likelihood ratio and its null distribution when the covariance matrices have an unknown scale factor Suppose that the covariance matrices have the form C, = 0 2 ~ , , v = 1,2,. . . , k , where

A, 's are known nonsingular matrices and o2 is unknown.


Let a2 is an estimator for a2 which is independent of X,'s and the estimator

@ is distributed with x2 distribution with q degrees of fieedom. o2

Then the likelihood function is the joint density function of (X, c?~), is

where c, and c, are positive constants which is independent of p and 02.

The likelihood ratio test rejects HI for the small values of

sup L HI (k+q)l2 A=-- - ( * 2 ) - 9

sup L 0, %-HI

where &; and &: are the maximum likelihood estimates of a2 under HI and H2 -HI respectively.

and

Theorem 4. The likelihood ratio test rejects HI for large values of

Now, suppose that the matrices A,'s are diagonal with diagonal elements

dv,,dv2,. . .,d, , v = 1,2,. . ., k , then the T statistic can be written.of the form


where h i , b2,, . . . , b, is the univariate isotonic regression of XIi , XZi,. . ., X, with 1 1

weights - - 1 - y . . . , . dli ' d2i d,

Theorem 5. Suppose that A, 's are diagonal with diagonal elements d,, , dv2 , . . . , d, , v = 1,2,. . . , k . If H , be true, then for t > 0 , the null distribution of T is

where 1 = 2 1 , and B(n,B) is the Beta distribution with parameters a and /I. i=l

Proof. - P(T 2 t ) = x P(L, = I,, . . . , Lp = lp)P(T 2 t / L, = I,, . . . , L, = 1,)

. .

1 Now, since - ( - x i 2 given L = 1 . L = 1 is distributed as

i=1 v=1 dvj

f12x&-11 and independent of qB2 which is distributed as a 2 x i . Therefore, the ?= statistic is distributed as B, , . The proof of the second part is easily.

y(@-l),:q

Theorem 6. Suppose that A,,'s are common and diagonal. If H , be true, then

The critical values of the ?= statistic for trivariate normal distribution for when q = 2 are computed in table 5.1.


Table 5.1. Critical values of the test statistic, F , for p = 3, q = 2 . a

References

[I] Anderson, T. W. (1984). An Introduction to Multivariate Statistical Analysis. 2nd edition, Wiely, New York.

[2] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and B d , H. D. (1972). Statistical Inference under Order Restrictions: The Theory and Application of Isotonic Regression. Wiely, New York.

[3] Bartholomew, D. J. (1959a). A test of homogeneity for ordered alternatives. Biometrika, 46,36-48.

[4] Bartholomew, D. J. (1959b). A test of homogeneity for ordered alternatives II. Biometrika, 46, 328-33 5.

[5] Chacko, V. J. (1963). Testing homogeneity against ordered alternatives. The Annals of Mathematical Statistics, 34,945-956.

[6] Inada, K. (1978). Some bivariate tests of composite hypotheses with restricted alternatives. Reports of the Faculty of Science, Kagoshima University (Mathematics, Physics and Chemistry), 11,25-3 1.


Kudo, A. (1963). A multivariate analogue of the one-sided test. Biometrika, 50,403-41 8. Kudo, A. and Choi, J. R. (1975). A generalized multivariate analogue of the one-sided test. Memoirs of the Faculty of Science, Kyushu University, 29(2), 303-328. Kulatunga, D. D. S. (1984). Convolutions of the probabilities P(1, k) used in order restricted inference. Memoirs of the Faculty of Science, Kyushu University. Series A, Mathematics, 38,9- 1 5. Kulatunga, D. D. S. and Sasabuchi, S. (1 984). A test of homogeneity of mean vectors against multivariate isotonic alternatives. Memoirs of the Faculty of Science, Kyushu University. Seies A, Mathematics, 38, 15 1 - 1 6 1. Kulatunga, D. D. S., Inutsuka, M. and Sasabuchi, S. (1990). A Simulation Stud) of Some Test Procedures for Testing Homogeneity of Mean Vectors against Multivariate Isotonic Alternatives. Technical Report, Kyushu University. Laska, E. M. and Meisner, M. J. (1989). Testing whether identified treatment is best. Biometries, 45, 1 139-1 15 1. Perlman, M. D. (1969). One-sided testing problems in multivariate analysis. The Annals of Mathematical Statistics, 40(2), 549-567. Robertson, T. and Wegrnan, E. T. (1978). Likelihood ratio tests for order restrictions in exponentioal families. The Annals of Statistics, 6(3), 485-505. Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiely, New York. Sarka, S. K., Snapinn, S., and Wang, W. (1995). On improving the min test for the analysis of combination drug trails (Corr: 1998V60 p180-18 1). Journal of Statistical Computation and Simulation, 51, 197-2 13. Sasabuchi, S. (1980). A test of a multivariate normal mean with composite hypotheses determined by linear inequalities. Biometrika, 67,429-439. Sasabuchi, S. (1988a). A multivariate test with composite hypotheses determined by linear inequalities when the covariance matrix is completely unknown. Memoirs of the Faculty of Science, Kyushu University. Series A, Mathematics, 42(1), 37-46. Sasabuchi, S. (1988b). A multivariate test with composite hypotheses determined by linear inequalities when the covariance matrix has an unknown scale factor. Memoirs of the Faculty of Science, Kyushu University. Series A, Mathematics, 42(1), 9-1 9. Sasabuchi, S. (2007). More powerful tests for homogeneity of multivariate normal mean vectors under an order restriction. Sankhya, 69(4), 700-71 6. Sasabuchi, S., Inutsuka, M. and Kulatunga, D. D. S. (1983). A multivariate version of isotonic regression. Biometrika, 70,465-472. Sasabuchi, S., Inutsuka, M. and Kulatunga, D. D. S. (1992). An algorithm for computing multivariate isotonic regression. Hiroshima Mathematical Journal, 22,551-560. Sasabuchi, S., Tanaka, K. and Takeshi, T. (2003). Testing homogeneity of multivariate normal mean vectors under an order restriction when the


covariance matrices are common but unknown. The Annals of Statistics, 31(5), 1517-1536.

[24] Shorack, G. R. (1967). Testing against ordered alternative in model I analysis of variance: normal theory and nonparametric. The Annals of Mathematical Statistics, 38, 1740- 1752.

[25] Silvapulle, M. J. and Sen, P. K. (2005). Constrained Statistical Inference: Inequality, Order, and Shape Restrictions, Wiely, New York.

Likelihood Ratio Test for Order Restrictions against all ...rms.scu.ac.ir/Files/Articles/Journals/Abstract/... · Abouzar Bazyari, Rahim Chinipardaz and Ali Akbar Rasekhi Department

Documents