Law of large numbers and Central limit theorem

大数定律 Law of large numbers (LLN)

虽然名字是 Law，但其实是严格证明过的 Theorem

• weak law of large number (Khinchin's law)

The weak law of large numbers: the sample average converges in probability to the expected value

$ar{X_n}=frac{1}{n}(X_1+ cdots +X_n) overset{p}{ o} E{X} $

• strong law of large number (proved by Kolmogorov in 1930)

The strong law of large numbers: the sample average converges almost surely to the expected value

$ar{X_n}=frac{1}{n}(X_1+ cdots +X_n) overset{a.s.}{ o} E{X} $

https://en.wikipedia.org/wiki/Law_of_large_numbers

https://terrytao.wordpress.com/2008/06/18/the-strong-law-of-large-numbers/

中心极限定理 Central Limit Theorem (CLT)

 https://en.wikipedia.org/wiki/Central_limit_theorem

切比雪夫不等式 (Chebyshev's Inequality)

Let $X$ be a random variable with finite expected value $mu$ and finit non-zero variance $sigma^2$, then for any real number $k>0$, 

$ mathrm{Pr} left( left|X-mu ight| geq k ight) leq frac{sigma^2}{k^2}$ 

马尔科夫不等式 (Markov's inequality)

If X is a nonnegative random variable and a > 0, then the probability that X is at least a is at most the expectation of X divided by a

$ mathrm{Pr} left( X geq a ight) leq frac{mu}{a} $

切尔诺夫限 (Chernoff bound)

The generic Chernoff bound for a random variable X is attained by applying Markov's inequality to e^tX. For every t > 0:

$ mathrm{Pr} left( X geq a ight)=mathrm{Pr} left( e^{tX} geq e^{ta} ight) leq frac{E[e^{tX}]}{e^{ta}} $