吴恩达机器学习第5周Neural Networks（Cost Function and Backpropagation）

5.1 Cost Function

假设训练样本为：{(x¹⁾,y⁽¹⁾),(x⁽²⁾,y⁽²⁾),...(x^(m),y^(m))}

L  = total no.of layers in network

s_L= no,of units(not counting bias unit) in layer L

K = number of output units/classes

如图所示的神经网络，L = 4，s1 = 3，s2 = 5，s3 = 5, s4 = 4

逻辑回归的代价函数：

            

神经网络的代价函数：

　　　

 5.2 反向传播算法 Backpropagation

 关于反向传播算法的一篇通俗的解释http://blog.csdn.net/shijing_0214/article/details/51923547

 

 5.3 Training a neural network

 

隐藏层的单元数一般一样，隐藏层一般越多越好，但计算量会较大。

Training a Neural Network

1. Randomly initialize the weights
2. Implement forward propagation to get hΘ(x(i)) for any x(i)
3. Implement the cost function
4. Implement backpropagation to compute partial derivatives
5. Use gradient checking to confirm that your backpropagation works. Then disable gradient checking.
6. Use gradient descent or a built-in optimization function to minimize the cost function with the weights in theta.