1预备知识
1.1 什么叫二次函数
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193010883-4866117.png)
1.2 什么叫二阶收敛
如果在有限步内找到二次函数的最优解,则该算法就称为二阶收敛。
1.3 什么叫共轭方向
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193257141-143027340.jpg)
2 共轭梯度法
2.1 引入
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193409245-643227944.jpg)
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193450229-171609056.jpg)
2.2 特点
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193616986-986053878.jpg)
举个例子体会:
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193809421-1930648325.jpg)
2.3 Fletcher-Reeves 算法
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505193926355-1610930905.jpg)
举个例子体会:
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194059193-1515911470.jpg)
这个例子是想说明:按照前边的理论来说,对于二次函数,最多迭代n次(维数),必然达到最优点,而此例题是因为中间有计算的误差,所以没有达到最优点。解决办法是:执行步骤3。
2.4 Powell 算法
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194443975-869597546.jpg)
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194513780-1620182379.jpg)
举个例子体会:
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194607227-1911618402.jpg)
这个例子想说明:有计算误差,所以最后的结果不为0。
3 变尺度算法
3.1 引入
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194825620-2034455666.jpg)
3.2 Fletcher-Powell 变尺度算法
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505194941656-91501574.jpg)
举个例子体会:
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505195026003-1077127745.jpg)
![](https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190505195159397-114703614.png)