Bayesian inference is a probabilistic inferential method. In the last two decades, it has become more popular than ever due to affordable computing power and recent advances in Markov chain Monte Carlo (MCMC) methods for approximating high dimensional integrals.
Bayesian inference can be traced back to Thomas Bayes (1764), who derived the inverse probability of the success probability ฮธ in a sequence of independent Bernoulli trials, where ฮธ was taken from the uniform distribution on the unit interval (0, 1) but treated as unobserved. For later reference, we describe his experiment using familiar modern terminology as follows.
Example 1.1 The Bernoulli (or Binomial) Model With Known Prior
Suppose that ฮธ โผ Unif(0, 1), the uniform distribution over the unit interval (0, 1), and that x 1 , . . . , x n is a sample from Bernoulli(ฮธ), which has the sample space X = {0, 1} and probability mass function (pmf) Pr (X = 1|ฮธ) = ฮธ and Pr (X = 0|ฮธ) = 1ฮธ,
(1.1)
where X denotes the Bernoulli random variable (r.v.) with X = 1 for success and X = 0 for failure. Write N = n i=1 x i , the observed number of successes in the n Bernoulli trials. Then N|ฮธ โผ Binomial(n, ฮธ), the Binomial distribution with parameters size n and probability of success ฮธ.