博客部分公式有格式问题:请前往语雀 https://www.yuque.com/leesamoyed/bvsayi/pd6nte#V4xDr
Linear Gaussian Model(线性高斯模型)
Kalman Fillter(卡诺曼滤波)
高斯分布非常重要!!!
一、极大似然估计:
$X:(x_1,x_2,...,x_n)^T
egin{gathered}
egin{pmatrix}
x_1x_2...x_n
end{pmatrix}
end{gathered}(
)x_i epsilon R^p quad x_i mathop{sim}^{iid} N(mu,Sigma) quad heta = (mu,Sigma)(
其中,iid:独立同分布
)MLE: heta_{MLE} = argmaxlimits_{ heta} P(X| heta)(
令)p=1, heta=(mu,sigma^2)(
()p=1(方便计算)
一维:
)p(x) = frac {1}{sqrt{2pi}sigma}exp(-frac{(x-mu)2}{2sigma2})(
多维(假设是)p(维):
)p(x) = frac {1}{(2pi){frac{p}{2}}|Sigma|{frac{1}{2}}}exp(-frac{1}{2}(x-mu)TSigma{-1}(x-mu))$
那么我们可以写出他的似然函数:
$log P(X| heta)
log mathop Pi limits_{i=1} limits^{N} p(x_i| heta)
mathop Pi limits_{i=1} limits^{N} log p(x_i| heta)
=mathop Sigma limits_{i=1} limits{N}logfrac{1}{sqrt{2pi}sigma}exp(-frac{(x-mu)2}{2sigma^2})
=mathop Sigma limits_{i=1} limits{N}[logfrac{1}{sqrt{2pi}}+log{frac{1}{sigma}}-frac{(x_i-mu)2}{2sigma^2}
](
那么)mu(估计:
)mu_{MLE} = argmaxlimits_{mu}log p(X| heta)=argmax mathop Sigma limits_{i=1}limits{N}-frac{(x_i-mu)2}{2sigma^2}=argminlimits_{mu} mathop Sigma limits_{i=1}limits{N}(x_i-mu)2(
对)mu(进行求导:
)frac{partial}{partialmu}Sigma(x_i-mu)^2 = mathop Sigma limits_{i-1}limits^{N} 2cdot(x_i-mu) cdot (-1)=0\mathop Sigma limits_{i=1}limits^{N}(x_i-mu)=0\mathop Sigma limits_{i=1}limits^{N}x_i -mathop Sigma limits_{i=1}limits^{N} mu =0
mu_{MLE} = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i(
无偏估计)E[mu_{MLE}]
= frac{1}{N}mathop Sigma limits_{i=1}limits^{N}E[x_i]
= frac{1}{N}mathop Sigma limits_{i=1}limits^{N}mu
= frac{1}{N}Ncdotmu
=mu$
那么(sigma^2)估计:
(sigma^2_{MLE} = argmaxlimits_{sigma} p(X|sigma)
\
=argmax Sigma
egin{matrix}
underbrace{ (-logsigma - frac{1}{2sigma^2}(x_i-mu)^2)
}
\ alpha
end{matrix})
求导:
(frac{partialalpha}{partialsigma} = mathop Sigma limits_{i=1}limits^{N}[-frac{1}{sigma}+frac{1}{2}(x_i-mu)^2 cdot (+2)sigma^{-3}] = 0\mathop Sigma limits_{i=1}limits^{N}[-frac{1}{sigma}+ (x_i-mu)^2 cdot sigma^{-3}] = 0\mathop Sigma limits_{i=1}limits^{N}[-sigma^2 + (x_i-mu)^2] = 0\-mathop Sigma limits_{i=1}limits^{N}sigma^2 + mathop Sigmalimits_{i-1}limits^{N}(x_i-mu)^2 = 0\mathop Sigma limits_{i=1}limits^{N}sigma^2 = mathop Sigma limits_{i-1}limits^{N}(x_i-mu)^2\sigma^2_{MLE} = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}(x_i-mu)^2)
(sigma^2_{MLE} = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}(x_i-mu_{MLE})^2)
有偏估计
实际上:(E[sigma^2_{MLE}] = frac{N-1}{N}sigma^2)
然而真正的无偏是这样的:(widehat{sigma} = frac{1}{N-1}mathop Sigma limits_{i=1}limits^{N}(x_i-mu_{MLE})^2)
二、无偏&有偏:
无偏估计:(mu_{MLE} = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i)
有偏估计:(sigma^2_{MLE} = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}(x_i-mu)^2)
其中:(x_i sim N(mu,sigma^2))
有偏估计和无偏估计的判断:(E[T(x)] = T(x))例如:(E[widehat{mu}] = mu quad E[widehat{sigma}] = sigma)相等则无偏,不相等则有偏
(E[mu_{MLE}] = E[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i] = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}E[x_i] = frac{1}{N}mathop Sigma limits_{i=1}limits^{N}mu = mu)所以无偏差
那么有偏差其实是看:
(E[sigma^2_{MLE}] mathop = limits^? sigma^2)
(sigma^2_{MLE} = frac{1}{N}mathop Sigma limits_{i-1}limits^{N}(x_i-mu_{MLE})^2= frac{1}{N}mathop Sigma limits_{i-1}limits^{N}(x_i^2 - 2x_imu_{MLE} + mu^2_{MLE}) =frac{1}{N}mathop Sigma limits_{i-1}limits^{N}x_i^2 - egin{matrix}underbrace{frac{1}{N}mathop Sigma limits_{i-1}limits^{N}2x_imu_{MLE}} \ 2cdotmu_{MLE}^2end{matrix}- egin{matrix}underbrace{frac{1}{N}mathop Sigma limits_{i-1}limits^{N}mu^2_{MLE}} \ mu^2_{MLE}end{matrix}\=frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i^2 -mu^2_{MLE})
(E[sigma^2_{MLE}] = E[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i^2 -mu^2_{MLE}]= E[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i^2 -mu^2 - (mu^2_{MLE}- mu^2)] \egin{matrix}underbrace{=E[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i^2 -mu^2]} \ underbrace{E[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}(x_i^2 -mu^2)]}\underbrace{frac{1}{N}mathop Sigma limits_{i=1}limits^{N}E(x_i^2 -mu^2)}\underbrace{frac{1}{N}mathop Sigma limits_{i=1}limits^{N}(E(x_i^2) -mu^2)}\ underbrace{frac{1}{N}mathop Sigma limits_{i=1}limits^{N} Var(x_i)}\underbrace{frac{1}{N}mathop Sigma limits_{i=1}limits^{N} sigma^2}\sigma^2end{matrix}egin{matrix}underbrace{-E[(mu^2_{MLE}- mu^2)]}\ underbrace{E[mu^2_{MLE}]- E[mu^2]}\ underbrace{E[mu^2_{MLE}]- mu^2}\ underbrace{E[mu^2_{MLE}]- mu_{MLE}^2}\ underbrace{Var(mu_{MLE})}\ frac{1}{N}sigma^2end{matrix}=frac{N-1}{N}sigma^2)
利用极大似然估计对于高斯分布:方差估计小了;
其中对于(Var(mu_{MLE})):
有:(Var[mu_{MLE}] = Var[frac{1}{N}mathop Sigma limits_{i=1}limits^{N}x_i] = frac{1}{N^2}mathop Sigma limits_{i=1}limits^{N}Var(x_i) = frac{1}{N^2}mathop Sigma limits_{i=1}limits^N sigma^2 = frac{1}{N^2}cdot N cdot sigma^2 = frac{sigma}{N})
三、从概率密度(PDF)角度观察:
(x sim (mu,Sigma) =
p(x) = frac {1}{(2pi)^{frac{p}{2}}|Sigma|^{frac{1}{2}}}exp(-frac{1}{2}(x-mu)^TSigma^{-1}(x-mu)))
其中(一般(Sigma)是半正定,在这里假设是正定的):
(x_i epsilon R^p quad r.v)
$x:(x_1,x_2,...,x_n)^T
egin{gathered}
egin{pmatrix}
x_1x_2...x_p
end{pmatrix}
end{gathered}$ (mu=egin{gathered}
egin{pmatrix}
mu_1\mu_2\...\mu_p
end{pmatrix}
end{gathered}) (Sigma=egin{gathered}
egin{pmatrix}
sigma_{11} quad sigma_{12} quad ... quad sigma_{1p}
\
sigma_{21} quad sigma_{22} quad ... quad sigma_{2p}
\
... quad ... quad ... quad ...
\
sigma_{p1} quad sigma_{p2} quad ... quad sigma_{pp}
end{pmatrix}
end{gathered}_{p×p})
主要研究式子中与(x)有关的部分即:((x-mu)^TSigma^{-1}(x-mu)) 可以看作是马氏距离((x)与(mu)之间)
当然最后得到的:((1×p)×(p×p)×(p×1)=1×1) 其实就是一个数(毕竟是概率密度)
((x-mu)^TSigma^{-1}(x-mu)) 可以看作是马氏距离((x)与(mu)之间)(当(Sigma)是单位矩阵,马氏距离等于欧式距离)
(Sigma = U wedge U^T)(特征分解), (UU^T=U^TU=I) ,(wedge = diag(lambda_i) quad i=1,...,p quad U=(U_1,U_2,...,U_p)_{p×p})
(=(u_1 quad u_2 quad ... quad u_p)
egin{gathered}
egin{pmatrix}
lambda_1 quad ... quad ... quad ...
\ ... quad lambda_2 quad ... quad ...
\ ... quad ... quad ... quad ...
\ ... quad ... quad ... quad lambda_p
end{pmatrix}
end{gathered})
(=(u1lambda_1 quad u2lambda_2 quad ... quad uplambda_p)
egin{gathered}
egin{pmatrix}
u_1^T
\u_2^T
\...
\u_p^T
end{pmatrix}
end{gathered})
(=)(mathop Sigma limits_{i=1} limits^{p}u_ilambda_iu_i^T)
(Sigma^{-1}=(U^Twedge U)^{-1}=(U^T)^{-1}wedge^{-1}U^{-1}=Uwedge^{-1}U^T)其中:(wedge^{-1}=diag(frac{1}{lambda_i})),(i=1,...,p)
(=mathop Sigma limits_{i=1} limits^{p}u_ifrac{1}{lambda_i}u_i^T)
((x-mu)^T Sigma^{-1}(x-mu)quad = quad (x-mu)^Tmathop Sigma limits_{i=1} limits^{p}u_ifrac{1}{lambda_i}u_i^T(x-mu)quad =quad mathop Sigma limits_{i=1} limits^{p}(x-mu)^Tu_ifrac{1}{lambda_i}u_i^T(x-mu))
令:(y_i=
egin{gathered}
egin{pmatrix}
y_1\
y_2\
...\
y_p
end{pmatrix}
end{gathered}) (y_i=(x-mu)^Tu_i quad = quad mathop Sigma limits_{i=1} limits^{p}y_ifrac{1}{lambda_i}y_i^T quad = quad mathop Sigma limits_{i=1} limits^{p}frac{y_i^2}{lambda_i})
令(p=2),(Delta=(x-mu)^T Sigma^{-1}(x-mu))
(Delta=frac{y_1^2}{lambda_1} + frac{y_2^2}{lambda_2}=1)
(P.S 这一章节理解起来巨麻烦)
四、局限性:
- 参数个数:(Sigma_{p×p}
ightarrow frac{p^2-p}{2}+p=frac{p^2+p}{2}=O(p^2)) 参数太多计算会太复杂
(Sigma ightarrow)对角矩阵 - 用一个高斯分布没法表达模型(无法确切表达模型)(多个高斯混合模型);
五、求边缘概率及条件概率:
高维高斯分布推到最复杂的地方章节!!!(要敏感)
已知:(x=egin{gathered}egin{pmatrix}x_a\x_bend{pmatrix}end{gathered}egin{gathered}egin{matrix}
ightarrow m\
ightarrow nend{matrix}end{gathered}) (m+n=p) (u=egin{gathered}egin{pmatrix}u_a\u_bend{pmatrix}end{gathered})
(Sigma=egin{gathered}egin{pmatrix}Sigma_{aa} quad Sigma_{ab}\Sigma_{ba} quad Sigma_{bb}end{pmatrix}end{gathered})
求:(P(x_a)),(P(x_b|x_a))((P(x_b)),(P(x_a|x_b))用对称性):
通用方法:配方法;
记住一个定理:
已知: (X sim N(mu,Sigma)) (y=AX+B)
结论: (y sim N(Amu+B,ASigma A^T) quadegin{gathered}egin{matrix}E[y]=E[Ax+B]\ =AE[x]+B\ =Amu+B end{matrix}end{gathered}quadegin{gathered}egin{matrix}Var[y]=Var[Ax+B]\=Var[Ax]+Var[B]\=Acdot Sigma cdot A^Tend{matrix}end{gathered})
关于(A)和(A^T)先后顺序,用矩阵维度去推理就可以
现在定义:(x_a=egin{gathered}egin{matrix}underbrace{egin{pmatrix}I_m&0_nend{pmatrix}}\ Aend{matrix}end{gathered}egin{gathered}egin{matrix}underbrace{egin{pmatrix}x_a\ x_bend{pmatrix}}\ xend{matrix}end{gathered}+0)egin{gathered}egin{pmatrix}I_m&0end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}mu_a\mu_bend{pmatrix}end{gathered}egin{gathered}egin{pmatrix}I_m & 0end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{aa} & Sigma_{ab} Sigma_{ba} & Sigma_{bb}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{aa} & Sigma{ab}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}I_m 0end{pmatrix}end{gathered}对于(x_b|x_a),我们可以先定义:(x_{b cdot a}=x_b - Sigma_{ba} Sigma_{aa}^{-1}x_a)
(大量实践)(mu_{b cdot a}=mu_b - Sigma_{ba} Sigma_{aa}^{-1}mu_a)(schur complement)egin{gathered}egin{matrix}underbrace{egin{pmatrix}-Sigma_{ba} Sigma_{aa}^{-1} & I end{pmatrix}} A end{matrix}end{gathered}egin{gathered}egin{matrix}underbrace{egin{pmatrix}x_a x_bend{pmatrix}} x end{matrix}end{gathered}(E[x_{b cdot a}] =egin{gathered}egin{pmatrix}-Sigma_{ba} Sigma_{aa}^{-1} & I end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}x_a \ x_bend{pmatrix}end{gathered}=mu_aSigma_{ba}Sigma_{aa}^{-1}mu_a=mu_{b cdot a})egin{gathered}egin{pmatrix}-Sigma_{ba} Sigma_{aa}^{-1} & Iend{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{aa} & Sigma_{ab} Sigma_{ba} & Sigma_{bb}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{aa} &Sigma_{ab} Sigma_{ba} & Sigma_{bb}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{aa}^{-1} & Sigma_{ba}^{-1} Iend{pmatrix}end{gathered}egin{gathered}egin{pmatrix}Sigma_{ba}Sigma_{aa}^{-1}Sigma_{aa}+Sigma_{ba} & Sigma_{ba}Sigma_{aa}{-1}Sigma_{ab}+Sigma_{bb}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}-Sigma_{aa}{-1} & Sigma_{ba}^{-1} Iend{pmatrix}end{gathered}egin{gathered}egin{pmatrix}0 & Sigma_{bb}-Sigma_{ba}Sigma_{aa}{-1}Sigma_{ab}end{pmatrix}end{gathered}egin{gathered}egin{pmatrix}-Sigma_{aa}{-1} &Sigma_{ba}^{-1} Iend{pmatrix}end{gathered}所以:(x_{b cdot a} sim N(mu_{b cdot a},Sigma_{b cdot b cdot a}))
求:(x_b|x_a)
其中:(x_b = x_{b cdot a}+ Sigma_{ba}Sigma_{aa}^{-1}x_a)
所以:(E[x_b|x_a] = mu_{b cdot a}+Sigma_{ba}Sigma_{aa}^{-1}x_a) (Var[x_b|x_a] = Var[x_{b cdot a}]=Sigma_{b cdot b cdot a})
最后得到:(x_b|x_a sim N(mu_{b cdot a}+Sigma_{ba}Sigma_{aa}^{-1}x_a,Sigma_{b cdot b cdot a})) $x_a|x_b $把其中a换b,b换a就行
**
**
六、求联合概率分布:
已知:(p(x)=N(x|mu,wedge^{-1}) quad p(y|x)=N(y|Ax+b,L^{-1}))
求:(p(y),p(x|y)) 已知:解:(y=Ax+b+varepsilon quad varepsilon sim N(0,L^{-1}))
①第一步:
%22%20aria-hidden%3D%22true%22%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-45%22%20x%3D%220%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5B%22%20x%3D%22764%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-79%22%20x%3D%221043%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5D%22%20x%3D%221540%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-3D%22%20x%3D%222096%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-45%22%20x%3D%223153%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5B%22%20x%3D%223917%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-41%22%20x%3D%224196%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-78%22%20x%3D%224946%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-2B%22%20x%3D%225741%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-62%22%20x%3D%226742%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-2B%22%20x%3D%227393%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-3B5%22%20x%3D%228394%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5D%22%20x%3D%228860%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-3D%22%20x%3D%229417%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-45%22%20x%3D%2210473%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5B%22%20x%3D%2211238%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-41%22%20x%3D%2211516%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-78%22%20x%3D%2212267%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-2B%22%20x%3D%2213061%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-62%22%20x%3D%2214062%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5D%22%20x%3D%2214491%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-2B%22%20x%3D%2214992%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-45%22%20x%3D%2215993%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5B%22%20x%3D%2216757%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-3B5%22%20x%3D%2217036%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-5D%22%20x%3D%2217502%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-3D%22%20x%3D%2218059%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-41%22%20x%3D%2219115%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-3BC%22%20x%3D%2219865%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMAIN-2B%22%20x%3D%2220691%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%20%3Cuse%20xlink%3Ahref%3D%22%23E1-MJMATHI-62%22%20x%3D%2221692%22%20y%3D%220%22%3E%3C%2Fuse%3E%0A%3C%2Fg%3E%0A%3C%2Fsvg%3E)
(
)y sim N(Amu+b,Acdot wedge^{-1} cdot A^T +L^{-1})$
②第二步:
(Z=egin{gathered}egin{pmatrix} x\yend{pmatrix}end{gathered}
sim
N(
egin{gathered}egin{matrix}underbrace{egin{bmatrix} mu \ Amu+bend{bmatrix}}\E[z] end{matrix}end{gathered},
egin{gathered}egin{bmatrix} wedge^{-1} & O \O & L^{-1}+Awedge^{-1}A^T end{bmatrix}end{gathered}
)),我们缺少(O)(圆圈)的部分,实质上两个圆圈缺的内容是一样的(本质一个圆圈内容是另一个圆圈内容转置,但是他们互为转置),我们定义圆圈的内容为(Delta),(Delta=Cov(x,y)\=E[(x-E[x]) cdot (y-E[y])]^T\=E[(x-mu)(y-Amu-b)]^T\=E[(x-
mu)(Ax+b+varepsilon-A/mu-b)^T]\=E[(x-mu)(Ax-Amu+varepsilon)^T]\=E[(x-mu)(Ax-A/mu)^T+(x-mu)varepsilon^T]\=E[(x-mu)(Ax-A/mu)^T]+egin{gathered}egin{matrix}underbrace{E[(x-mu)varepsilon^T]}\xotvarepsilon\x-muotvarepsilon\ E[(x-mu)]E[varepsilon] =0end{matrix}end{gathered}
\=E[(x-u)(Ax-Amu^T)]\=E[(x-mu)(x-mu)^Tcdot A^T]\=E[(x-u)(Ax-Amu^T)]\=E[(x-mu)(x-mu)^T]cdot A^T\=Var[x] /cdot A^T
\=wedge^{-1}A^T)
最后一步中,(mu)就是(E[x]),所以那个事方差公式
代入(O)(圆圈)
(p(x|y) sim N(,))最后答案套公式