这是我的letex学习笔记，由于时间有限，只能讲源码和结果贴出：
这里面是广义线性模型的推导过程：
documentclass{article}
usepackage{paralist}
egin{document}
itle{Generate Linear Model Estimation Note}
author{Xue Zoushi }
date{April 28, 2016}
maketitle
The general procedures:
egin{compactenum}
item General exponential family format
egin{equation}
f(y| heta) = exp left ( frac{y heta + b( heta)}{a(phi)} + c(y,phi)
ight)
end{equation}
[
ell( heta|y) =log[f(y| heta)]= frac{y heta + b( heta)}{a(phi)} + c(y,phi)
]
item Some important attributes of log-likelihood
[ E(Y) = mu = frac{partial b( heta)} {partial heta} ]
[ E[S( heta)]= 0 ]
[ E[frac{partial S}{partial heta}] = -E[S( heta)]^2 ]
[ I( heta)= Var(S( heta))= E[S( heta)]^2 - {E[S( heta)]}^2 ]
[ Var(Y) = a(phi)[frac{partial^2 b( heta)}{partial heta ^ 2}] ]
item Newton-Raphson and Fisher-scoring
The scalar form of Taylor series
[ ell( heta) equiv ell( ilde{ heta}) + ( heta - ilde{ heta})
left. frac{partial ell ( heta)}{partial heta}
ight |_{ heta = ilde{ heta}} +
frac{1}{2} ( heta - ilde{ heta})^2 left. frac{partial ell^2 ( heta)}{partial heta^2}
ight |_{ heta = ilde{ heta}}]
Set ( partial ell ( heta) / partial heta = 0 ) and rearranging terms yields:
[ heta equiv ilde{ heta} -
left [ left . frac{partial ^2 ell ( heta)}{partial heta ^2}
ight |_{ heta= ilde{ heta}}
ight ]^{-1}
left [ left . frac{partial ell ( heta)}{partial heta}
ight |_{ heta= ilde{ heta}}
ight ] ]
The basic matrix form of Newton-Raphson algorithm:
egin{equation}
heta equiv ilde{ heta} - [H( ilde{ heta})]^{-1} S( ilde{ heta})
end{equation}
Replace hession matrix with the information matrix (i.e. ( E(H( heta))= -Var[S( heta)]= -I( heta) )),
we get Fisher scoring algorithm:
egin{equation}
heta equiv ilde{ heta} - [I( ilde{ heta})]^{-1} S( ilde{ heta})
end{equation}
item Estimate the coefficient ( eta ).
Scalar form
egin{equation}
frac{partial ell (eta)}{partial eta} =
frac{partial ell ( heta) }{ partial heta} frac{partial heta }{ partial mu }
frac{partial mu }{ partial eta } frac{partial eta }{ partial eta }
end{equation}
Some results:
egin{itemize}
item
[ frac{partial ell ( heta)}{partial heta} = frac{y-mu}{a(phi)} ]
item
[frac{partial heta}{partialmu}=left(frac{partialmu}{partial heta}
ight)^{-1}=frac{1}{V(mu)}]
item
[ frac{partial eta}{partial eta} = frac{partial X eta}{eta}]
item
[ frac{partial ell(eta)}{partial eta} =
(y-mu)left( frac{1}{V(y)}
ight)left(frac{partial mu}{partial eta}
ight) X ]
end{itemize}
Matrix form
egin{equation}
frac{partialell( heta)}{partial eta} = X^{'} D^{-1} V^{-1}(y-mu)
end{equation}
where $y$ is the $n imes1$ vector of observations, $ell( heta)$ is the $n imes 1$ vector of log-likelihood
values associated with observations, $V = diag[Var(y_{i})]$ is the $n imes n$ variance matrix of the
observations, $D=diag[partial eta_{i} / partial mu_{i}]$ is the $n imes n$ matrix of derivatives, and $mu$
is the $n imes 1$ mean vector.\
Let $W=(DVD)^{-1}$, we can get:
[ S(eta) = frac{partial ell ( heta)}{partial eta}
= X^{'} D^{-1}V^{-1}(D^{-1}D)(y-mu) = X^{'}WD(y-mu) ]
[ Var[S(eta)] =X^{'}WD[Var(y-mu)]DWX =X^{'}WDVDWX=X^{'}WX ]
item Pseudo-Likelihood for GLM \
Using Fisher scoring equation yields
$eta = ilde{eta} +(X^{'} ilde{W}X)^{-1}X^{'} ilde{W} ilde{D}(y- ilde{mu})$,
where $ ilde{W}, ilde{D}$, and $mu$ evaluated at $ ilde{eta}$. So GLM estimating equations:
egin{equation}
X^{'} ilde{W}Xeta = X^{'} ilde{W}y^{*}
end{equation}
where $y^{*} = X ilde{eta} + ilde{D}(y- ilde{mu}) = ilde{eta} + ilde{D}(y- ilde{mu})$, and
$y^{*}$ is called the pseudo-variable.
[ E(y^{*}) =E[X ilde{eta} + ilde{D}(y - ilde{mu})] = Xeta ]
[ Var(y^{*}) = E[X ilde{eta} + ilde{D}(y - ilde{mu})] = ilde{D} ilde{V} ilde{D}= ilde{W}^{-1} ]
egin{equation}
X^{'}[Var(y^{*})]^{-1}Xeta = X^{'}[Var(y^{*})]^{-1} Rightarrow X^{'}WXeta = X^{'}Wy^{*}
end{equation}
end{compactenum}
end{document}
使用emacs编辑，然后使用命令 pdfletex -glm_estimation.tex生成，生成文件在博客园的文件附件中。
下面是生成的pdf文件截图：
