STATISTICAL INFERENCE PART III

STATISTICAL INFERENCEPART IIIEXPONENTIAL FAMILY, FISHER INFORMATION, CRAMER-RAO LOWER BOUND (CRLB)*

*EXPONENTIAL CLASS OF PDFSX is a continuous (discrete) rv with pdf f(x;), . If the pdf can be written in the following formthen, the pdf is a member of exponential class of pdfs of the continuous (discrete) type. (Here, k is the number of parameters)

*REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFSWe have a regular case of the exponential class of pdfs of the continuous type ifRange of X does not depend on .c() 0, w1(),, wk() are real valued functions of for .h(x) 0, t1(x),, tk(x) are real valued functions of x.If the range of X depends on , then it is called irregular exponential class or range-dependent exponential class.

*EXAMPLESX~Bin(n,p), where n is known. Is this pdf a member of exponential class of pdfs? Why?

Binomial family is a member of exponential family of distributions.

*EXAMPLESX~Cauchy(1,). Is this pdf a member of exponential class of pdfs? Why?

Cauchy is not a member of exponential family.

*EXPONENTIAL CLASS and CSSRandom Sample from Regular Exponential Classis a css for .If Y is an UE of , Y is the UMVUE of .

*EXAMPLESLet X1,X2,~Bin(1,p), i.e., Ber(p).

This family is a member of exponential family of distributions.is a CSS for p.is UE of p and a function of CSS.is UMVUE of p.

*EXAMPLESX~N(,2) where both and 2 is unknown. Find a css for and 2 .

*FISHER INFORMATION AND INFORMATION CRITERIAX, f(x;), , xA (not depend on ).Definitions and notations:

*FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X:The Fisher Information in the random sample:Lets prove the equalities above.

*FISHER INFORMATION AND INFORMATION CRITERIA

*FISHER INFORMATION AND INFORMATION CRITERIA

*FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X:The Fisher Information in the random sample:Proof of the last equality is available on Casella & Berger (1990), pg. 310-311.

*CRAMER-RAO LOWER BOUND (CRLB)Let X1,X2,,Xn be sample random variables.Range of X does not depend on .Y=U(X1,X2,,Xn): a statistic; doesnt contain .Let E(Y)=m().

Let prove this!

*CRAMER-RAO LOWER BOUND (CRLB)-1Corr(Y,Z)1

0 Corr(Y,Z)21

Take Z=(x1,x2,,xn;)Then, E(Z)=0 and V(Z)=In() (from previous slides).

*CRAMER-RAO LOWER BOUND (CRLB)Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

*CRAMER-RAO LOWER BOUND (CRLB)E(Y.Z)=m(), Cov(Y,Z)=m(), V(Z)=In()

The Cramer-Rao Inequality(Information Inequality)

*CRAMER-RAO LOWER BOUND (CRLB)CRLB is the lower bound for the variance of an unbiased estimator of m().When V(Y)=CRLB, Y is the MVUE of m().For a r.s., remember that In()=n I(), so,

*LIMITING DISTRIBUTION OF MLEs : MLE of X1,X2,,Xn is a random sample.

*EFFICIENT ESTIMATORT is an efficient estimator (EE) of if T is UE of , and,Var(T)=CRLBY is an efficient estimator (EE) of its expectation, m(), if its variance reaches the CRLB.An EE of m() may not exist.The EE of m(), if exists, is unique.The EE of m() is the unique MVUE of m().

*ASYMPTOTIC EFFICIENT ESTIMATORY is an asymptotic EE of m() if

*EXAMPLESA r.s. of size n from X~Poi(). Find CRLB for any UE of .Find UMVUE of .Find an EE for .Find CRLB for any UE of exp{-2}. Assume n=1, and show that is UMVUE of exp{-2}. Is this a reasonable estimator?

*EXAMPLEA r.s. of size n from X~Exp(). Find UMVUE of , if exists.

SummaryWe covered 3 methods for finding UMVUE:Rao-Blackwell Theorem (Use a ss T, an UE U, and create a new statistic by E(U|T))Lehmann-Scheffe Theorem (Use a css T which is also UE)Cramer-Rao Lower Bound (Find an UE with variance=CRLB)*

***********************