Bayesian Inference - nakaly’s notes

Bayesian Inference is a method for estimating a true probability distribution from samples of the distribution.

When I read the article in the wikipedia, I didn’t get the point.

But I found a better article that gives me more intuitive understanding.

My understanding of Bayesian Inference consists of the following steps:

Come up with a hypothesis of a target distribution, which is called a prior probability distribution. Use uniform distribution as a prior probability distribution if you have no hypotheses.
Update the hypothesis based on samples (which are called evidence) from the target distribution. The updated hypothesis is called a posterior probability.

When updating the hypothesis, we use Bayes' theorem.

$\displaystyle P(H\mid E) = \frac{P(E\mid H) \cdot P(H)}{P(E)} = \frac{P(E\mid H)}{P(E)} \cdot P(H)$

$\displaystyle P(H)$ : the prior probability

$\displaystyle P(H\mid E)$ : the posterior probability

We can get the posterior probability $\displaystyle P(H\mid E)$ by multiplying the prior probability $\displaystyle P(H)$ by the value $\displaystyle \frac{P(E\mid H)}{P(E)}$

The difficult part of Bayesian Inference is calculating $\displaystyle P(E)$ .

It can be calculated as follows:

$\displaystyle P(E) = \sum_{H}P(E|H) P(H)$ when the target distribution is discrete.

$\displaystyle P(E) = \int_{H} p(\mathbf{X} \mid H) p(H)$ when the target distribution is continuous.

MCMC (Markov Chain Monte Carlo) is sometimes used to calculate it.