Reversible Markov Chain - nakaly’s notes

In the previous post, I introduced some of important Markov chain properties.

I would like to write about a Markov chain property that is used in Markov Chain Monte Carlo(mcmc), which is reversibility.

Let $X_n$ is a variable of a Markov chain. If there is a probability distribution $\pi$ such that:

$\displaystyle \pi_i P(X_{n+1} = j \mid X_{n} = i) = \pi_j P(X_{n+1} = i \mid X_{n} = j)$

Then the Markov chain is said to be reversible.

One of trivial examples of reversible Markov chain is as follows:

f:id:nakaly:20170221030823p:plain:w300

When defining $p_{ij}$ as follows:

$\displaystyle p_{ij} = P(X_{n+1} = i \mid X_{n} = j)$

The equation of revesibility can be written as follows:

$\displaystyle \pi_i p_{ij} = \pi_j p_{ji}$ for all states $i,j$ of a Markov chain

Since the equation is true for all $i$ , when we sum up the left- and right-hand sides of the equation.

We get follows:

$\displaystyle \sum_i \pi_i p_{ij} = \sum_i \pi_j p_{ji} = \pi_j \sum_i p_{ji} = \pi_j$

It means that we get the following equation.

$\displaystyle \pi_j = \sum_i \pi_i p_{ij}$

This is the definition of a stationary distribution $\pi$ .