Christopher Dougherty EC220 - Introduction to econometrics (review chapter) Slideshow: type 2 error and the power of a test Original citation: Dougherty,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Christopher Dougherty
EC220 - Introduction to econometrics (review chapter)Slideshow: type 2 error and the power of a test
Original citation:
Dougherty, C. (2012) EC220 - Introduction to econometrics (review chapter). [Teaching Resource]
This version available at: http://learningresources.lse.ac.uk/141/
Available in LSE Learning Resources Online: May 2012
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. This license allows the user to remix, tweak, and build upon the work even for commercial purposes, as long as the user credits the author and licenses their new creations under the identical terms. http://creativecommons.org/licenses/by-sa/3.0/
http://learningresources.lse.ac.uk/
TYPE II ERROR AND THE POWER OF A TEST
A Type I error occurs when the null hypothesis is rejected when it is in fact true. A Type II error occurs when the null hypothesis is not rejected when it is in fact false.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
1
TYPE II ERROR AND THE POWER OF A TEST
We will see that, in general, there is a trade-off between the risk of making a Type I error and the risk of making a Type II error.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
2
TYPE II ERROR AND THE POWER OF A TEST
We will consider the case where the null hypothesis, H0: m = m0 is false and the actual value of m is m1. This is shown in the figure.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
3
TYPE II ERROR AND THE POWER OF A TEST
If the null hypothesis is tested, it will be rejected only if X lies in one of the rejection regions associated with it.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
4
TYPE II ERROR AND THE POWER OF A TEST
To determine the rejection regions, we draw the distribution of X conditional on H0 being true. The distribution is marked with a dashed curve to emphasize that H0 is not actually true.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
5
TYPE II ERROR AND THE POWER OF A TEST
The rejection regions for a 5 percent test, given this distribution, are marked on the diagram.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
6
TYPE II ERROR AND THE POWER OF A TEST
If X lies in the acceptance region, H0 will not be rejected, and so a Type II error will occur. What is the probability of this happening? To determine this, we now turn to the actual distribution of X, given that m = m1. This is the solid curve on the right.
0
0 Xm0
m1
f (X )
m0- 1.96s.d. m
0+1.96s.d.
acceptance region(5% test)
rejection region rejection region
2.5% 2.5%
7
TYPE II ERROR AND THE POWER OF A TEST
The probability of X lying in the acceptance region for H0 is the area under this curve in the acceptance region. It is the shaded area in the figure. In this particular case, the probability of X lying within the acceptance region for H0, thus causing a Type II error, is 0.15.
0
0
f (X )
rejection region rejection regionacceptance region(5% test)
m0 Xm
0- 1.96s.d. m
0+1.96s.d. m
1
true distribution
false distribution
0.15
8
TYPE II ERROR AND THE POWER OF A TEST
The probability of rejecting the null hypothesis, when it is false, is known as the power of a test. By definition, it is equal to 1 minus the probability of making a Type II error. It is therefore 0.85 in this example.
0
0
f (X )
rejection region rejection regionacceptance region(5% test)
m0 Xm
0- 1.96s.d. m
0+1.96s.d. m
1
true distribution
false distribution
0.15 0.85
9
TYPE II ERROR AND THE POWER OF A TEST
The power depends on the distance between the value of m under the false null hypothesis and its actual value. The closer that the actual value is to m0, the harder it is to demonstrate that H0: m = m0 is false.
0
0
f (X )
rejection region rejection regionacceptance region(5% test)
m0 Xm
0- 1.96s.d. m
0+1.96s.d. m
1
true distribution
false distribution
0.15 0.85
10
TYPE II ERROR AND THE POWER OF A TEST
This is illustrated in the figure. m0 is the same as in the previous figure, and so the acceptance region and rejection regions for the test of H0: m = m0 are the same as in the previous figure.
0
0
rejection region rejection regionacceptance region(5% test)
f (X )
m0- 1.96s.d. m
0+1.96s.d.m
0
m2
true distribution
X
false distribution
11
TYPE II ERROR AND THE POWER OF A TEST
As in the previous figure, H0 is false, but now the true value is m2, and m2 is closer to m0. As a consequence, the probability of X lying in the acceptance region for H0 is much greater, 0.68 instead of 0.15, and so the power of the test, 0.32, is much lower.
0
0
rejection region rejection regionacceptance region(5% test)
f (X )
m0- 1.96s.d. m
0+1.96s.d.m
0
m2
true distribution
X
false distribution
0.320.68
12
TYPE II ERROR AND THE POWER OF A TEST
The figure plots the power of a 5 percent significance test as a function of the distance separating the actual value of m and m0, measured in terms of the standard deviation of the distribution of X.
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
Distance between the actual value of m and m 0 (standard deviations)
Pro
bab
ilit
y o
f re
ject
ing
H0
13
TYPE II ERROR AND THE POWER OF A TEST
As is intuitively obvious, the greater is the discrepancy, the greater is the probability of H0: m = m0 being rejected.
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
Distance between the actual value of m and m 0 (standard deviations)
Pro
bab
ilit
y o
f re
ject
ing
H0
14
TYPE II ERROR AND THE POWER OF A TEST
We now return to the original value of m1 and again consider the case where H0: m = m0 is false and H1: m = m1 is true. What difference does it make if we perform a 1 percent test, instead of a 5 percent test?
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%
15
TYPE II ERROR AND THE POWER OF A TEST
The figure shows the acceptance region for the 1 percent test.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%
16
TYPE II ERROR AND THE POWER OF A TEST
The probability of X lying in this region, given that it is actually distributed with mean m1, is shown as the yellow shaded area. It is 0.34. The probability of making a Type II error is therefore 0.34.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%0.34
17
TYPE II ERROR AND THE POWER OF A TEST
We have seen that the probability of making a Type II error with a 5 percent test, given by the blue shaded area, was 0.15. This illustrates the trade-off between the risks of Type I and Type II error.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%0.15
18
TYPE II ERROR AND THE POWER OF A TEST
If we perform a 1 percent test instead of a 5 percent test, and H0 is true, the risk of mistakenly rejecting it (and therefore committing a Type I error) is only 1 percent instead of 5 percent.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%0.15
19
TYPE II ERROR AND THE POWER OF A TEST
However, if H0 happens to be false, the probability of not rejecting it (and therefore committing a Type II error) is larger.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%0.15
20
TYPE II ERROR AND THE POWER OF A TEST
How much larger? This is not fixed. It depends on the distance between m0 and m1, measured in terms of standard deviations. In this particular case, it has increased from 0.15 to 0.34, so it has about doubled.
0
0
f (X )
acceptance region(1% test)
rejection region rejection region
m0
m0- 2.58s.d. m
0+2.58s.d.
m1
X
0.5% 0.5%0.15
21
TYPE II ERROR AND THE POWER OF A TEST
To generalize, we plot the power functions for the 5 percent and 1 percent tests.
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5
Distance between the actual value of m and m 0 (standard deviations)
Pro
bab
ilit
y o
f re
ject
ing
H0
5% test 1% test
22
Copyright Christopher Dougherty 2011.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section R.10 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own and who feel that they might
benefit from participation in a formal course should consider the London School