【10】极大似然估计量的性质

【10】极大似然估计量的性质

(定理1) 设(X_{(i)}(i=1,...,n)sim N_p(mu,Sigma)，(n>p)),则（(mu,Sigma)）'s MLE is:

[hat{mu}=overline{X}=frac1nsum_{i=1}^nX_{(i)}\ hatSigma=frac1nA=frac1nsum_{i=1}^n(X_{(i)}-overline{X})(X_{(i)}-overline{X})' ]

(定理2) 若(overline{X},A)分别为(p)元正态总体(N_p(mu,Sigma))的样本均值向量，和样本离差阵，则：

1. (overline{X}sim N_p(mu,frac1nSigma));
2. (A=^dsum_{t=1}^{n-1}Z_tZ_t'),其中，(Z_1,...,Z_{n-1})独立同(N_p(0,Sigma))分布；
3. (overline{X},A)相互独立；
4. (P{A>0}=1Leftrightarrow n>p).

讨论何时(A)为一个正定矩阵；

• 一元的一些结论：

if (X_1,...X_n) iid (N(mu,sigma^2))：

1. (overline{X}sim N(mu,frac{sigma^2}{n}));
2. (frac{sum_{i-1}^n(X_i-overline{X})^2}{sigma^2}simchi^2(n-1));
3. (overline{X})与(s^2=frac1{n-1}sum_{i=1}^n(X-overline{X})^2)独立；

证明：

设 (Gamma) 为 (n) 阶正交阵（不同行内积为 0 , 同行内积为 1），具有以下形式：

[Gamma= left[ egin{array}{ccc} gamma_{11}&dots&gamma_{1n}\ vdots&&vdots\ gamma_{(n-1),1}&dots&gamma_{(n-1),n}\ frac1{sqrt{n}}&dots&frac1{sqrt{n}} end{array} ight]=(gamma_{ij})_{n imes n} ]

给 (X) 做线性变换 (Z=Gamma X) 即：

[Z=left[ egin{array}{c} Z_1'\ vdots\ Z_n' end{array} ight]=Gamma left[ egin{array}{c} X_{(1)}'\ vdots\ X_{(n)}' end{array} ight]=Gamma X ]

则：

[egin{align} Z'=&X'Gamma'\\ (Z_1,...,Z_n)=&(X_{(1)},...,X_{(n)})Gamma'\ Z_t=(X_{(1)},...,X_{(n)})&left[ egin{array}{c} gamma_{t1}\ vdots\ gamma_{tn} end{array} ight],(t=1,...,n) end{align} ]

(Z_t)为(p)维随机向量，而且是(p)维正态 r.v. (X_{(1)}',...,X_{(n)}')的线性组合，故(Z_t)也是(p)维正态随机向量。且：

[E(Z_t)=E(sum_{i=1}^nr_{ti}X_{(i)})=sum_{i=1}^nr_{ti}E(X_{(i)})=musum_{i=1}^nr_{ti}= egin{cases} mucdot(frac1{sqrt{n}}sum_{i=1}^nr_{ti})cdotsqrt{n}&=0&当\,t eq n\,时，\ mucdot nfrac1{sqrt{n}}&=sqrt{n}mu&当\,t=n\,时\ end{cases} ]

[egin{align} Cov(Z_{alpha},Z_{eta})=&E[(Z_{alpha}-E(Z_alpha))(Z_{eta}-E(Z_{eta}))']=sum_{i=1}^n(r_{alpha i}r_{eta i})Sigma= egin{cases} O&alpha eqeta,\ Sigma&alpha=eta end{cases} end{align} ]

• (overline{X}sim N_p(mu,frac1nSigma));

(Z_n=frac1{sqrt{n}}sum_{alpha=1}^nX_{(alpha)}=sqrt{n}overline{X}sim N_p(musqrt{n},Sigma))

• (A=^dsum_{t=1}^{n-1}Z_tZ_t'),其中，(Z_1,...,Z_{n-1})独立同(N_p(0,Sigma))分布；

(sum_{alpha=1}^nZ_alpha Z_alpha'=(Z_1,...,Z_n)left(egin{array}{c}Z_1'\vdots\Z_{n}'end{array} ight)=Z'Z=X'Gamma'cdotGamma X=sum_{alpha=1}^nX_alpha X_alpha')

于是：

[sum_{alpha=1}^{n-1}Z_alpha Z_alpha'=sum_{alpha=1}^nX_alpha X_alpha'-Z_{n}Z_{n}'=sum_{alpha=1}^nX_alpha X_alpha'-noverline{X}overline{X}'=A ]

• (overline{X},A)相互独立；

[A=g(sum_{t=1}^{n-1}Z_tZ_t')\ overline{X}=f(Z_n) ]

而(Z_i,Z_j)相互独立（(i eq j)）时，则(A、overline{X})也相互独立。

• (P{A>0}=1Leftrightarrow n>p).

(B=(Z_1,...,Z_{n-1}))，记(A=BB')，因为(A=BB')，(p imes(n-1))矩阵，显然(rank(A)=rank(B)),当(A)为正定矩阵时，(rank(A)=p),因此(rank(B)=p)，故 (（n-1）geq p),即(n>p).

(无偏性)

[E(overline{X})=left(egin{array}{c} frac1nsum_{i=1}^nE(x_{i1})\ vdots\ frac1nsum_{i=1}^nE(x_{ip})\ end{array} ight)= left(egin{array}{c} mu_1\vdots\mu_p end{array} ight)=mu ]

[E(A)=Eleft(sum_{i=1}^{n-1}Z_iZ_i' ight)=sum_{i=1}^{n-1}E(Z_iZ_i')=sum_{i=1}^{n-1}D(Z_i)=(n-1)Sigma eqSigma ]

故(hat{Sigma}=frac1nA)不是(Sigma)的无偏估计，应修正为：(S=frac1{n-1}A).