STAT 462-Computational Data Analysisnasser/Teaching/STAT462_862/STAT_462-Ch8P1.pdfSTAT 462-Computational Data Analysis Chapter 8- Part 1 Nasser Sadeghkhani a.sadeghkhani@queensu.ca

STAT 462-Computational DataAnalysis

Chapter 8- Part 1

Nasser Sadeghkhani

a.sadeghkhani@queensu.ca

October 2017

1 / 21

Outline

Background and Concepts

I Bayes vs. Classical Statistics

2 / 21

The basic tool of Bayesian statistics is Bayes theorem. It is namedafter Reverend Thomas Bayes, a nonconformist minister who livedin England in the first half of the eighteenth century. The theoremwas published posthumously in 1763 in ”An essay towards solvinga problem in the doctrine of chances”.

Figure: T. Bayes, 1702–1761

3 / 21

Bayesian vs. Frequentist

The basic philosophical difference between the frequentist andBayesian paradigms is that Bayesians treat unknown parameters asrandom and use probability to quantify their uncertainty about it.In contrast, frequentists treat unknown parameters fixed.

4 / 21

Bayes’ Theorem

5 / 21

Continuous version in terms of a parameter.

6 / 21

Historical Example

7 / 21

8 / 21

9 / 21

More generally

10 / 21

11 / 21

12 / 21

Normal model– Uniform Prior

13 / 21

14 / 21

Normal model–Normal prior

15 / 21

Conjugate PriorsDefinition: Let π(θ) be a class of prior distributions. Then it isconjugate for the model P (y|θ) whenever π(θ|y) belongs to thesame class (of prior).

16 / 21

How to choose the prior distribution?

17 / 21

Jeffrey’s Prior

18 / 21

Example of Jeffrey’s prior

19 / 21

Nuisance parameters

20 / 21

21 / 21