Markov Chain

马尔科夫链是一个具备马尔科夫性的随机过程。马尔科夫性是指：系统在下一步所处状态的条件概率仅与系统当前的状态相关，与系统以前的状态无关。

A Markov chain is a sequence of random variables X₁, X₂, X₃, ... with the Markov property, namely that, given the present state, the future and past states are independent. Formally,

The possible values of X_i form a countable set S called the state space of the chain. 

 一个典型的马尔科夫例子：

Another example is the dietary habits of a creature who eats only grapes, cheese or lettuce, and whose dietary habits conform to the following rules:

• It eats exactly once a day.
• If it ate cheese today, tomorrow it will eat lettuce or grapes with equal probability.
• If it ate grapes today, tomorrow it will eat grapes with probability 1/10, cheese with probability 4/10 and lettuce with probability 5/10.
• If it ate lettuce today, it will not eat lettuce again tomorrow but will eat grapes with probability 4/10 or cheese with probability 6/10.

 其实这与小时候玩的大富翁游戏很相似。

 to be continue...