Long-Run Behavior of Markov Chains

As the time index $m$ approaches infinity, a Markov chain may settle down and exhibit steady-state behavior. If the following limit exists:

$$\lim_{m \rightarrow \infty} p^{(m)}_{ij} = \pi_{j}$$

for all values of $i$, then the $\{\pi_{j}\}$ are the limiting or steady-state probabilities.

Looking at the state probability as $m$ approaches infinity, we see that:

$$\lim_{m \rightarrow \infty} \pi_{j}(m) = \lim_{m \rightarrow \infty} \sum_{i} \pi_{i}(0)p_{ij}(m)$$ (1)

$$= \sum_{i} \pi_{i}(0) \lim_{m \rightarrow \infty} p_{ij}(m)$$

$$= \sum_{i} \pi_{i}(0) \pi_{j}$$

$$= \pi_{j} \sum_{i} \pi_{i}(0)$$

$$= \pi_{j}$$

When the limiting probabilities exist, then can be found using the following equations:

$$\Pi = \Pi P$$

and

$$\sum_{i} \pi_{i} = 1$$

where

$$\Pi = \left[ \begin{array}{cccc} \pi_{0} & \pi_{1} & \pi_{2} & \cdots \end{array} \right]$$