基本概率分布Basic Concept of Probability Distributions 8: Normal Distribution

PDF version

PDF & CDF

The probability density function is $$f(x; mu, sigma) = {1oversqrt{2pi}sigma}e^{-{1over2}{(x-mu)^2oversigma^2}}$$ The cumulative distribution function is defined by $$F(x; mu, sigma) = Phileft({x-muoversigma} ight)$$ where $$Phi(z) = {1oversqrt{2pi}} int_{-infty}^{z}e^{-{1over2}x^2} dx$$

Proof:

$$ egin{align*} int_{-infty}^{infty}f(x; mu, sigma) &= int_{-infty}^{infty}{1oversqrt{2pi}sigma}e^{-{1over2}{(x-mu)^2oversigma^2}} dx\ &= {1oversqrt{2pi}sigma}int_{-infty}^{infty}e^{-{1over2}{(x-mu)^2oversigma^2}} dx\ &= {1oversqrt{2pi}}int_{-infty}^{infty}e^{-{1over2}y^2} dyquadquadquadquadquad(mbox{setting} y={x-muoversigma} Rightarrow dx = sigma dy)\ end{align*} $$ Let $I = int_{-infty}^{infty}e^{-{1over2}y^2} dy$, then $$ egin{eqnarray*} I^2 &=& int_{-infty}^{infty}e^{-{1over2}y^2} dyint_{-infty}^{infty}e^{-{1over2}x^2} dx\ &=& int_{-infty}^{infty}int_{-infty}^{infty}e^{-{1over2}(y^2+x^2)} dydxquadquadquadquad(mbox{setting} x=rcos heta, y=rsin heta)\ &=& int_{0}^{infty}int_{0}^{2pi}e^{-{1over2}r^2} rd heta dr \ & & (mbox{double integral} iintlimits_{D}f(x, y) dxdy = iintlimits_{D^*}f(rcos heta, rsin heta)r drd heta) \ &=& 2piint_{0}^{infty}re^{-{1over2}r^2} dr\ &=& -2pi e^{-{1over2}r^2}Big|_{0}^{infty}\ &=& 2pi end{eqnarray*} $$ Hence $$int_{-infty}^{infty}f(x; mu, sigma) = {1oversqrt{2pi}} cdotsqrt{2pi} = 1$$

Standard Normal Distribution

If $X$ is normally distributed with parameters $mu$ and $sigma^2$, then $$Z = {X-muoversigma}$$ is normally distributed with parameters 0 and 1.

Proof:

An important conclusion is that if $X$ is normally distributed with parameters $mu$ and $sigma^2$, then $Y = aX + b$ is normally distributed with parameters $amu + b$ and $a^2sigma^2$. Denote $F_{Y}$ as the cumulative distribution function of $Y$: $$ egin{align*} F_{Y}(x) &= P(Y leq x)\ &= P(aX + b leq x)\ &= P(X leq {x-bover a})\ &= F_{X}left({x-bover a} ight) end{align*} $$ where $F_{X}(x)$ is the cumulative distribution function of $X$. By differentiation, the probability density function of $Y$ is $$ egin{align*} f_{Y}(x) &= {1over a}f_{X}left({x-bover a} ight)\ &= {1oversqrt{2pi}asigma}e^{-{1over2}{({x-bover a} - mu)^2over sigma^2}}\ &= {1oversqrt{2pi}(asigma)}e^{-{1over2}{(x-b - amu)^2over a^2sigma^2}}\ &= {1oversqrt{2pi}(asigma)}e^{-{1over2}{(x-(b + amu))^2over (asigma)^2}} end{align*} $$ which shows that $Y$ is normally distributed with parameters $amu + b$ and $a^2sigma^2$. According to the above result, we can easily deduce that $Z = {X-muoversigma}$ follows the normally distributed with parameters 0 and 1.

Mean

The expected value is $$E[X] = mu$$

Proof:

$$ egin{align*} E[Z] &= int_{-infty}^{infty}xf_{Z}(x) dxquadquadquad quadquad quadquad (mbox{setting} Z={X-muoversigma})\ &= {1oversqrt{2pi}}int_{-infty}^{infty}xe^{-{1over2}x^2} dx\ &= -{1oversqrt{2pi}}int_{-infty}^{infty}e^{-{1over2}x^2} dleft(-{1over2}x^2 ight)\ &= -{1oversqrt{2pi}}e^{-{1over2}x^2}Big|_{-infty}^{infty}\ &= 0 end{align*} $$ Hence $$ egin{align*} E[X] &= Eleft[sigma Z+mu ight]\ &= sigma E[Z] + mu\ &= mu end{align*} $$

Variance

The variance is $$mbox{Var}(X) = sigma^2$$

Proof:

$$ egin{align*} Eleft[Z^2 ight] &= {1oversqrt{2pi}}int_{-infty}^{infty}x^2e^{-{1over2}x^2} dxquadquadquad quadquad quadquadquadquadquad (mbox{setting} Z={X-muoversigma})\ &= {1oversqrt{2pi}}left(-xe^{-{1over2}x^2}Big|_{-infty}^{infty} +int_{-infty}^{infty}e^{-{1over2}x^2} dx ight)quadquadquad(mbox{integrating by parts})\ &= {1oversqrt{2pi}}int_{-infty}^{infty}e^{-{1over2}x^2} dx quadquadquadquadquadquadquad(mbox{standard normal distribution})\ &= 1 end{align*} $$ the integral by parts: $$u= x, dv = xe^{-{1over2}x^2} dx$$ $$implies du = dx, v = int xe^{-{1over2}x^2} dx = -e^{-{1over2}x^2}$$ $$implies int x^2e^{-{1over2}x^2} dx =-xe^{-{1over2}x^2} +int e^{-{1over2}x^2} dx$$ Hence $$mbox{Var}(X) = mbox{Var}(sigma Z + mu)= sigma^2mbox{Var}(Z) = sigma^2$$

Examples

1. If $X$ is a normal random variable with parameters $mu = 3$ and $sigma^2 = 9$, find (a) $P(2 < X <5 data-blogger-escaped-b=""> 0)$; (c) $P(|X - 3| > 6)$.

Solution:

(a) $$ egin{align*} P(2 < X < 5) &= Pleft({2-3over3} < {X - 3over 3} < {5-3over 3} ight)\ &= Pleft(-{1over3} < Z < {2over3} ight)\ &= Phileft({2over3} ight) - Phileft(-{1over3} ight) = 0.3780661 end{align*} $$ R code:

pnorm(2/3) - pnorm(-1/3)
# [1] 0.3780661 

(b) $$ egin{align*} P(X > 0) &= Pleft({X-3over3} > {0-3over3} ight)\ &= Pleft(Z > -1 ight)\ &= 1 - Phi(-1) = 0.8413447 end{align*} $$ R code:

1 - pnorm(-1)
# [1] 0.8413447

(c) $$ egin{align*} P(|X - 3| > 6) &= P(X > 9) + P(X < -3)\ &= Pleft({X-3over3} > {9-3over3} ight) + Pleft({X-3over3} < {-3-3over3} ight)\ &= P(Z > 2) + P(Z < -2)\ &= 1-Phi(2) + Phi(-2) = 0.04550026 end{align*} $$ R code:

1 - pnorm(2) + pnorm(-2)
# [1] 0.04550026 

2. Let $X$ be normally distributed with standard deviation $sigma$. Determine $Pleft(|X-mu| geq 2sigma ight)$. Compare with Chebyshev's Inequality.

Solution:

$$ egin{align*} Pleft(|X-mu| geq 2sigma ight) &= Pleft({X-muoversigma} geq 2 ight) + Pleft({X-muoversigma} leq -2 ight)\ &=2cdot Pleft({X-muoversigma} leq -2 ight) = 2Phi(-2) end{align*} $$ R code:

2 * pnorm(-2)
# [1] 0.04550026

By Chebyshev's Inequality, the probability is $$Pleft(|X-mu| geq 2sigma ight) leq {1over2^2}=0.25$$ which is a weaker estimation.

3. Let $X$ be a normally distributed random variable with expected value $mu=5$. Assume $P(X leq 0) = 0.1$. What is the variance of $X$?

Solution:

$$ egin{align*} P(X leq 0) &= Pleft({X - 5oversigma} leq {0-5oversigma} ight)\ &= Pleft(Z leq -{5oversigma} ight) = 0.1 end{align*} $$ Hence by using R:

z = qnorm(0.1)
var = (-5/z)^2
# [1] -1.281552
# [1] 15.22186 

$$-{5oversigma} = -1.281552Rightarrow sigma^2 = 15.22186$$

4. A normally distributed random variable $X$ satisfies $P(X leq 0) = 0.4$ and $P(X geq 10) = 0.1$. What is the expected value $mu$ and the standard deviation $sigma$?

Solution:

$$P(X leq 0) = 0.4Rightarrow Phileft({-muoversigma} ight) = 0.4$$ and $$P(X geq 10) = 0.1RightarrowPhileft({10-muover sigma} ight) = 0.9$$ Thus $$egin{cases}{-muoversigma}=-0.2533471\ {10-muover sigma}=1.281552 end{cases}Rightarrow egin{cases}mu = 1.650579\ sigma= 6.515088 end{cases}$$ R code:

z1 = qnorm(0.4); z2 = qnorm(0.9)
s = 10 / (z2 - z1)
mu = -s * z1
z1; z2
# [1] -0.2533471
# [1] 1.281552
mu; s
# [1] 1.650579
# [1] 6.515088 

5. Consider independent random variables $Xsim N(1, 3)$ and $Ysim N(2, 4)$. What is $P(X + Y leq 5)$?

Solution:

$X +Y$ is still normally distributed with parameters $$mu = mu_1 + mu_2 = 3$$ and $$sigma^2 = sigma_1^2 + sigma_2^2 = 7$$ Hence $$ egin{align*} P(X + Y leq 5) &= Pleft(Z leq {5-3 oversqrt{7}} ight)\ &= Phileft({2 oversqrt{7}} ight) = 0.7751541 end{align*} $$ R code:

pnorm(2 / sqrt(7))
# [1] 0.7751541

Reference

1. Ross, S. (2010). A First Course in Probability (8th Edition). Chapter 5. Pearson. ISBN: 978-0-13-603313-4.
2. Brink, D. (2010). Essentials of Statistics: Exercises. Chapter 5 & 15. ISBN: 978-87-7681-409-0.