Making sense of randomness. E Pluribus Unum (Out of many, one)

Making sense of randomness

E Pluribus Unum (Out of many, one)

Fair coin?

• How do you determine whether a coin is fair?

• Rephrase: how many times do you need to toss a coin in order to be “confident” that it is fair?

Classical definition of probability• Pierre Simon Laplace. “Théorie

analytique des probabilités”

• The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

Frequentist view• … defines an event's probability as

the limit of its relative frequency in a large number of trials.

• frequentist account was motivated by problems and paradoxes of the classical interpretation.

• Frequentists: Venn, Fisher, von Mises

• Frequentists: probabilities pertain only to well-defined random experiments

• The set of all possible outcomes of a random experiment is called the sample space of the experiment.

• An event is defined as a particular subset of the sample space that you want to consider.

• For any event only one of two possibilities can happen: it occurs or it does not occur.

• The relative frequency of occurrence of an event, in a number of repetitions of the experiment, is a measure of the probability of that event.

Probabilities for events that are not mutually

independent• If two events A and B are

independent then • If the events are not independent

then

Hypothetical example

• The Cubs are a streaky baseball team.

• In 2015, 59 of their 97 wins were followed by another win, 38 wins were followed by losses.

• What is the probability that the Cubs win the next game in 2015 given that they win the current game?

• What is the probability that they win two consecutive games?

Analysis• The probability of the Cubs winning

any game is 97/162=0.59876543209• The probability of winning the next

game given a win in the current game is 59/97=0.60824742268

• Therefore the probability of winning game A followed by game B is

• 97/162 x 59/97 = 59/162=0.36419753086*

• Note: If the events were independent the probability of two consecutive wins would be 97/162 x 97/162= 0.35852004267**

Monte Hall Problem

• Monte's dilemma• “Suppose you’re on a game

show, and you’re given a choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say number 1, and the host, who knows what’s behind the doors, opens another door, say number. 3, which has a goat. He says to you, ‘Do you want to pick door number 2?’ Is it to your advantage to switch your choice of doors?”

Monte Hall – the Movie

• Monte Hall - the answer

Clicker question:

• In the Monte Hall problem, you should always switch

• A) True• B) False

Bayesian probability• Bayesian approach treats

“probability” as 'a measure of a state of knowledge' -- not as a frequency.

• objectivist school: rules of Bayesian statistics … extension of Aristotelian logic.

• subjectivist school: the state of knowledge corresponds to a 'personal belief’.

• “Machine learning” methods are based on objectivist Bayesian principles.

• A probability can be assigned to a hypothesis, -- not possible under the frequentist view in which a hypothesis can only be accepted or rejected.

• Aristotelian logic : every statement is either true or false

• Bayesian reasoning: incorporates uncertainty.

• Different from fuzzy logic in which truth is a value between 0 and 1.

Bayes’ rule

Bayes rule follows from formula for joint

probabilities

Cox’s axioms: Cox, 1961: The algebra of probable inference• Cox sought system to satisfy the following

conditions:

• 1. Divisibility and comparability - The plausibility of a statement is a real number and is dependent on known information related to the statement.

• 2. Common sense - Plausibilities should vary sensibly with the assessment of plausibilities in the model.

• 3. Consistency - If the plausibility of a statement can be derived in many ways, all the results must be equal.

• P(A) is the prior (or marginal) probability of A. Does not account for any information about B.

• P(A|B) is the conditional probability of A, given B. It is also called the posterior probability—it depends upon the specified value of B.

• P(B|A) is the conditional probability of B given A.

• P(B) is the prior or marginal probability of B, and acts as a normalizing constant.

• Bayesian probability interprets the concept of probability as 'a measure of a state of knowledge

• The frequentist view of probability overshadowed the Bayesian view during the first half of the 20th century

• The word Bayesian appeared in the 1950s, and by the 1960s it became the term preferred by people who sought to escape the limitations and inconsistencies of the frequentist approach to probability theory

Card version

• Suppose that there are three cards, one black on both sides, one white on both sides and one black on one side and white on the other.

• One of the three cards is pulled randomly out of a hat and the side showing is black.

• What is the probability that the other side is also black?

Frequentist interpretation

• The card is not the white card because the side showing is black.

• It is either the black card or the card that has black on one side and white on the other.

• The probability of either case is ½• Therefore, the probability that the

other side is black equals ½

Bayesian approach:

• A: the black card was drawn; P(A)=1/3

• B: the side showing of the card drawn is black. P(B)=1/2

• P(B|A): probability that the side showing is black if the black card is drawn (=1)

• P(A|B): probability that the black card is drawn if the side showing is black

Bertrand's paradox

Clicker Question

• What is the probability that a randomly chosen chord of the circle has length larger than the side of the equilateral triangle?

• A) 1/2• B) 1/3• C) ¼• D) It depends

Questions

• What is the main difference between the classical definition vs frequentist definition of probability?

• What is the main difference between frequentist probability and Bayesian probability?