神经网络梯度下降的推导

https://blog.csdn.net/u012328159/article/details/80081962

https://mp.weixin.qq.com/s?__biz=MzUxMDg4ODg0OQ==&mid=2247484013&idx=1&sn=2f1ec616d9521b801ef318308aa66e57&chksm=f97d5c93ce0ad585343a415be0b346fe18c960a41f45bfe69d0db4128b95b97d76d3c23ed293&mpshare=1&scene=23&srcid=&sharer_sharetime=1591403700835&sharer_shareid=9ed15fc26b568c844598f8638f4c17a4#rd

公式细节推导

Ag课程的总结（单层神经网络）

Ag课程的总结（深层神经网络）

已知 (AL) 和 (J)，先求出 (dAL)：

(dAL = -(np.divide(Y, AL) - np.divide(1-Y, 1-AL)) ​)

---> (dZL = dAL * sigmod'(Z^{[L]}) = dAL*s*(1-s) ​)

---> (dWL=frac{1}{m}dZL·A^{[L-1]T}​)

---> (dbL=frac{1}{m}np.sum(dZL, axis=1, keepdims=True) ​)

---> (dA^{[L-1]} = W^{[L]T}·dZ^{[L]}​)

===>

(dZ^{[l]} = dA^{[l]} * g'(Z^{[l]})) （(l in [L-1 , 1]) ，(relu'(Z^{[l]}) = np.int64(A^{[l]} > 0)) ）

---> (dW^{[l]} = frac{1}{m}dZ^{[l]}·A^{[l-1]T})

---> (db^{[l]}=frac{1}{m}np.sum(dZ^{[l]}, axis=1, keepdims=True) ​)

---> (dA^{[l-1]} = dZ^{[l]}·W^{[l]} = W^{[l]T}·dZ^{[l]})

……

……