Detection Chia-Hsin Cheng. Wireless Access Tech. Lab. CCU Wireless Access Tech. Lab. 2 Outlines Detection Theory Simple Binary Hypothesis Tests Bayes.

Detection

Chia-Hsin Cheng

2

Wireless Access Tech. Lab.

CCU Wireless Access Tech. Lab.

Outlines

Detection TheorySimple Binary Hypothesis TestsBayes CriterionThe MAP CriterionThe ML CriterionNeyman-Pearson CriterionM HypothesesComposite HypothesisGLRT (Generalized LRT)The General Gaussian ProblemCourse Information

3



Detection Theory

Example: Know Signal in Noise Problem

11

00

()(), 0, if () is transmittted()

()(), 0, if () is transmittted

stnttTstrt

stnttTst

We are faced with the problem of decision which of two possible signals was transmitted.Detection problem: observes r(t) and guess whether s1(t) or s2(t) was sent.

Decision rule

decision

4



5



Classical Detection Theory

The source generates outputs of two choices (hypotheses), H0 and H1.We do not know which hypothesis is true.

The transition mechanism can be viewed as a device that knows which hypothesis is true.(i.e., channel model, likelihood function)

Based on this knowledge, it generates a point in the observation space according to some probability law.

6



Classical Detection Theory

Example: symbol rate sampling

Example: over the symbol rate sampling

We confine our discussion to problems in which the observation space is finite-dimensional. The observations consist of a set of N numbers and can be represented as a point in a N-dimensional space.

7



After observing the outcome in the observation space, we shall guess which hypothesis is true, and to accomplish this, we develop a decision rule that assign each point in the observations space to one of the hypotheses.

8



Simple Binary Hypothesis Tests

We assume that the observation space corresponds to a set of N observations: , or in a vector r ,

The probabilistic transition mechanism generates points in accord with the two known conditional probability densities

and . The objective is to use this information to develop a suitable decision rule.

Example:

1 2 3, , , ..., Nr r r r1 2 3[ , , , ..., ]T

Nr r r rr

2

1

0

(0, )

: 1

: 0

n N

H r n

H r n

1

0

2 2| 1

2 2| 0

1( | ) exp( ( 1) / 2 )

21

( | ) exp( / 2 )2

r H

r H

p r H r

p r H r

9



Decision Criteria

In the binary hypothesis problem, we know that either H0 or H1 is true. Each time the experiment is conducted, one of things can happen:

10



Bayes Criterion

Two assumptions

1. A Priori probabilities are known

2. Costs are assigned Cij:

We should like to design our decision rule so that on the average the cost will be as small as possible.

Average cost = risk

11



Bayes Criterion (cont.)

Because the decision rule must say either H1 or H0,we can view it as a rule for dividing the total observation space Z into two parts Z0 and Z1. When an observation falls in Z0, we say H0, and whenever and observation falls in Z1,we say H1.

Optimal Bayes test: design Z0 and Z1 to minimize

12




The risk function of (1) can be written in terms of the transition probabilities and the decision regions:

We shall assume throughout our work that the cost of a wrong decision is higher than the cost of a correct decision, i.e.,

(3)

(2)

13




To find the Bayes test, we must choose the decision regions Z0 and Z1 in such a manner that the risk will be minimized. Because we require that a decision be made, this means that we must assign each point R in the observation space Z to Z0 or Z1 .

Thus Rewriting (2), we have

Observing that

Substituting into (5)

(4)

(5)

(6)

0 1Z Z

00

| 01 ( | )HzP H r R

10

| 11 ( | )HzP H r R

14



Bayes Criterion (cont.)Then, we have

The first two terms of (7) represent the fixed cost. The assumptions in (3) imply that the two terms inside the brackets are positive, the second term is larger than the first should be included in Z0 because they contribute a negative amount to the integral.

•The decision regions are defined by the statement:

(7)

(8)

fixed cost

>0

>0

H1

15




The quantity on the right of (9) is the threshold of the test and is

denoted by :

(10)

(9)

Likelihood ratio:

16




The Bayes criterion leads us to a likelihood ratio test (LRT)

Because the natural logarithm is a monotonic function, and both sides of (11a) are positive, an equivalent test is (log LRT)

(11a)

(11b)

17



The MAP Criterion

A priori (before we observe R = r): P0 and P1

A posteriori (after we have observed R = r):

When C10=C01=1, C00=C11=0

Form (9) and dividing by Pr(R)

MAP(maximum a posteriori probability) Criterion:

0

1

| 0 0

| 1 1

( | ) Pr( true| )

( | ) Pr( true| )

H

H

P H H

P H H

r

r

R R = r

R R = r

1

0

1 | 1

0 | 0

( | ) / ( )

( | ) / ( )H r

H r

P p H P

P p H P

r

r

R R

R R

1H

0H

10 00

01 11

( )1

( )

C C

C C

1

0

| 1

| 0

( | )

( | )H

H

P H

P Hr

r

R

R

1H

0H

1

( )condition prob.: ( | )

( )

( ) ( ) ( | )

( ) ( ) ( )( | )

( ) ( )

P A BP A B

P B

P A B P B P A B

P A B P B P A BP B A

P A P A

18



The ML Criterion

The possible likelihoods of r: and When P0 = P1 =1/2, C10=C01=1,and C00=C11=0

1| 1( | )Hp Hr R0| 0( | )Hp Hr R

ML(maximum likelihood) Criterion:

1

0

| 1

| 0

( | )( )

( | )H

H

P H

P H r

r

RR

R

1H

0H

1

MAP Criterion = ML Criterion (when all Pi are the same)

Form (9)

19



Bayes Criterion Example

Example: We assume that under H1 the source output is a constant

voltage m and that under H0 the source output is zero. Before observation that voltage is corrupted by an Gaussian noise.

because the noise samples are Gaussian.

20



Bayes Criterion Example (cont.)

21




The likelihood ratio test is

Thus, the log LRT is

or , equivalently

22




If C00 = C11 = 0 and C01 = C10 = 1, the risk function of (5) reduces to the probability of error

i.e., the Bayes test is minimizing the total probability of error.When the decision regions are chosen, the values of the integrals in

(5) are determined. We denote the probabilities of false alarm, detection, and miss, respectively, as

23




For any choice of decision regions, the risk function can be written from (5) as

Because

Then

24



Neyman-Pearson Criterion

In many physical situations, it is difficult to assign realistic costs or a priori probabilities. A simple procedure to bypass this difficulty is to work with the conditional probabilities PF and PD.

Min. F

>0 <0For >0,

25



Neyman-Pearson Criterion (cont.)

For any positive value of an LRT will minimize F. (A negative value of gives an LRT with the inequalities reversed)

Thus F is minimized by the likelihood radio test

To satisfy the constraint we choose so that , i.e.,

Observe that decreasing is equivalent to increasing Z1; thus PD increase as decreases.

FP '

(12)

Solving (12) for gives the threshold.

26




21

???

27



Q-function

Gaussian (normal) distribution

erfc-function

Q-function

The pdf of a Gaussian or normal distribution:

22

1 1( ) exp ( ) ,

22f x x

x

2( , )N

2

0

2erf ( ) exp( ) ;

xx t dt

22

erfc( ) exp( ) 1 erf ( )x

x t dt x

21( ) exp( ) 1 ( )

221

= erfc( )2 2

x

tQ x dt x

x

21( ) exp( ) ;

22

x tx dt

The cumulative distribution function (CDF) of a N(0,1)

The complementary CDF of a N(0,1)

28



21 1(0,1), ( ) exp

22N f x t

21( ) exp( )

22x

tQ x dt

xt

0

21( ) exp( )

22

x tx dt

( ) 1 ( )

1(0)

2

Q x Q x

Q

Q-function (cont.)

29



ux0 2 x

2 22

1 1( , ) ( ) exp ( )

22N f x x

d x 2

2

2

2

1 1exp ( )

22

1exp( )

22

1exp( )

22

( )

x

x

d

u du

tdt

tdt

dQ

1u

t dt du

( ) 1 ( )

1(0)

2

Q x Q x

Q

Q-function (cont.)

30




31



0 1 0 1 0 1( | ) ( | )r rP P sayH H true PP sayH H true

32



10 ln gausses and 1F DH P P

0gausses and 0F DH P P

Receiver operating characteristic(ROC)

33



Summary

Using either a Bayes criterion or a Neyman-Pearson criterion, we find that the optimum test is a likelihood ratio test. Thus, regardless of the dimensionality of the observation space, the test consists of comparing a scalar variable with a threshold.

In many cases, construction of the LRT can be simplified if we can identifies a sufficient statistic.

A complete description of the LRT performance was obtained by plotting the conditional probabilities PD and PF as the threshold was varied.

( ) R

34



M Hypotheses

In the simple M-ary test, there are M source outputs, each of which corresponds to one of M hypotheses. As before, we are forced to make a decision.The Bayes criterion assigns a cost to each of the alternatives, assu

mes a set of a priori probabilities , and minimizes the risk.

The cost Cij denotes that the i-th hypothesis is chosen and the j-th hypothesis is true.

0 1 1, , ..., MP P P

35



M Hypotheses (cont.)

The risk function for the M-ary hypothesis problem is

To find the optimum test, we vary the Zi to minimize R. We

consider the case of M=3 below.

1 1 1 1

|0 0 0 0

(say | true) ( | )j

i

M M M M

j ij r i j j ij H jZi j i j

P C P H H P C p H d

r R R

36



Noting that Z0=Z-Z1-Z2, because the regions are disjoint, we obtain

0 01 2

| 0 | 01 ( | ) ( | )H Hz zp H d p H d r rR R R R

0( )I R

1( )I R

2( )I R

37



38



Bayes criterion

We see that the decision rules correspond to three lines in the 1, 2 plane.

It is easy to verify that these three lines intersect at a common point.

The optimum Bayes test becomes

(I)

(II)

(III)

Choose H1

(I0>I1)

(I0<I1)

(I0>I2)

(I0<I2)

(I1<I2)

(I1>I2)

39



(I)

(II)

(III)

40



1

41



42



When , MAP ML0 1 1 =, ..., = MP P P

43



Some Points of M-ary Detection

The minimum dimension of the decision space is no more than M-1. The boundaries of the decision regions are hyperplanes in the(1, …, M-1).

A particular test of importance is the minimum total probability of error test. Here we compute the a posteriori probability of each hypothesis Pr(Hi|R) and choose the largest.

44



Composite Hypothesis

Example:

45



Composite Hypothesis (cont.) If is a random variable with a know pdf and the probability density of

on the two hypotheses as the likelihood ratio is

Above ex: Let = M

Reduce the problem to a simple hypothesis-testing problem ( knowing a pdf of )

(14)

46



Composite Hypothesis (cont.)

Example (continued) We assume that the probability density governing m on H1 is

Then,

Integrating and taking the logarithm of both sides, we obtain

47



GLRT (Generalized LRT)

Using ML (maximum likelihood) estimate the value of under the two hypotheses (H0, H1), the result is called a generalized likelihood ratio test:

where 1, ranges over all in H1 and 0, ranges over all in H0. In other words, we make a ML estimate of 1 , assuming that H1 is true. We then evaluate for and use this value in the numerator. A similar procedure gives the denominator.

ˆ 1| 1( | )p r R

48



The General Gaussian Problem

Definition: A set of random variables are defined as jointly Gaussian if all their linear combinations are Gaussian random variables.Definition: A vector r is a Gaussian random vector when its

components are jointly Gaussian random variables.Definition: A Gaussian random vector r is characterized by its mean m

and covariance matrix , i.e.,

1 2 3, , , ..., Nr r r r

1 2 3, , , ..., Nr r r r

Λ ~ ( , )Nr m Λ

49



/ 2 1/ 2 1|

1( | ) [(2 ) | | ] exp[ ( ) ( )]

2i

N T TH i i i i ip H r R K R m Q R m

(16)

0 0 0

1 1 1

: ~ ( , )

: ~ ( , )

H r N m K

H r N m K

(15)

• Consider the following binary hypothesis problem

50



(17)

(17)

Equal covariance matrices:

51



we defined d as the distance between the means on the two hypothesis when the variance was normalized to equal one.

22 1 0

0

[ ( | ) ( | )]

( | )TE l H E l H

dvar l H

m Q m

The performance of this binary detection problem depends on d

(18)

52



PD

0Tm Qm 1

Tm Qmr

d

53



Case1. Independent Components with Equal Variance.

Substituting to (18)

22

1 Q=

K I I

222 2

1 1

T Td

d

m Q m m m m

m

2

1T Tl

m QR m R

We see that d corresponds to the distance between the two mean-value vectors divided by the standard deviation of Ri.

54



Case 2. Independent Components with Unequal Variance.Case 3. A general case.

PLS refer to textbook pp.101-107.

55



Course Information

Instructor: Chia-Hsin Cheng Room: 525 Tel:05-2720411 ext23240 E-mail: [email protected]

Text book: H.L. Van Trees, Detection, Estimation and Modulation Theory, Wiley, 20

01, pt. I, Chap1~ chap2.

Reference books: S.M. Kay, Fundamentals of Statistical Signal Processing: Detection Theor

y, Prentice Hall, 1998, pt. II. H.V. Poor, An Introduction to Signal Detection and Estimation, 2nd ed., S

pringer-Verlag, 1994.

56



Ultra wideband

UWB First Reading [1]Moe Z. Win & Robert A. Scholtz, “Impulse Radio: How it works”, IEEE C

ommunication Letters,February 1998. [2] R. A. Scholtz, “Multiple access with time-hopping impulse modulation,”in

Proc. MILCOM, Oct. 1993. [3] Durisi, G. Romano, “On the validity of Gaussian Approximation to Charac

terize the Multiuser Capacity of UWB TH PPM,” IEEE Conference on Ultra Wideband Systems and Technologies. Digest of Papers , Baltimore, USA,pp.157 - 161, 2002.

[4]“PlusON Technology Overview”, http://www.timedomain.com ,July 2000. [5] K. Mandke et al., “The Evolution of Ultra Wide Band Radio for Wireless

Personal Area Networks,” High Frequency Electronics, September 2003, pp. 22-32.

Detection Chia-Hsin Cheng. Wireless Access Tech. Lab. CCU Wireless Access Tech. Lab. 2 Outlines Detection Theory Simple Binary Hypothesis Tests Bayes.

Documents

ccu wireless access

decision rule decision

risk slide

decision regions z

h1h1 slide

b slide

problem of decision

map criterion