多变量梯度下降

Hypothesis: [{h_ heta }left( x ight) = { heta ^T}x = { heta _0} + { heta _1}{x_1} + { heta _2}{x_2} + ... + { heta _n}{x_n}]

参数（Parameters）: [{ heta _1},{ heta _2},{ heta _3},...,{ heta _n}]

可以用Θ向量表示上面的一系列值 

损失函数（Cost function）: [Jleft( {{ heta _1},{ heta _2},{ heta _3},...,{ heta _n}} ight) = frac{1}{{2m}}sumlimits_{i = 1}^m {{{left( {{h_ heta }left( {{x^{left( i ight)}}} ight) - {y^{left( i ight)}}} ight)}^2}} ]

当用Θ表示时，损失函数：[Jleft( Theta  ight) = frac{1}{{2m}}sumlimits_{i = 1}^m {{{left( {{h_ heta }left( {{x^{left( i ight)}}} ight) - {y^{left( i ight)}}} ight)}^2}} ]

---

梯度下降算法表示为：

重复（repeat）{

[{ heta _j}: = { heta _j} - alpha frac{partial }{{partial { heta _j}}}Jleft( {{ heta _1},{ heta _2},{ heta _3},...,{ heta _n}} ight)]  (simultaneously update for every j = 0,...,n)

如果用Θ表示 [{ heta _j}: = { heta _j} - alpha frac{partial }{{partial { heta _j}}}Jleft( Theta  ight)] (对于 j = 0,...,n，同时更新)

}

---

现在看以下部分怎么算 [frac{partial }{{partial { heta _j}}}Jleft( {{ heta _1},{ heta _2},{ heta _3},...,{ heta _n}} ight)] 

当n=1时

算法为：

repeat {

[{ heta _0}: = { heta _0} - alpha underbrace {frac{1}{m}sumlimits_{i = 1}^m {left( {{h_ heta }left( {{x^{left( i ight)}}} ight) - {y^{left( i ight)}}} ight)} }_{frac{partial }{{partial { heta _0}}}Jleft( heta  ight)}]

[{ heta _1}: = { heta _1} - alpha frac{1}{m}sumlimits_{i = 1}^m {left( {{h_ heta }left( {{x^{left( i ight)}}} ight) - {y^{left( i ight)}}} ight)} {x^{left( i ight)}}]

(simultaneously update θ₀, θ₁)

}

当n>=1时

算法为：

repeat {

[{ heta _j}: = { heta _j} - alpha frac{1}{m}sumlimits_{i = 1}^m {left( {{h_ heta }left( {{x^{left( i ight)}}} ight) - {y^{left( i ight)}}} ight)} x_j^{left( i ight)}]

(simultaneously update θ_j for j = 0,..., n)

}