最优化算法【共轭梯度法】

特点：具有超线性收敛速度，只需要计算梯度，避免计算二阶导数

算法步骤

(step0:)
给定初始值(x_0),容许误差(epsilon)

(step1:)
计算梯度(g_k= abla f(x_k)),if (norm(g_k)<=epsilon)， (break;)
输出当前值(x_k)
else (to step2)

(step2:)

[egin{cases} d_k=-g_k, & ext {$k$=0} \ d_k=-g_k+eta_{k-1}d_{k-1}, & ext {$k$>=1} end{cases} ]

[eta_{k-1}=frac{g_k^Tg_k}{g_{k-1}^Tg_{k-1}} ]

(step3:)
利用线搜索技术确定(alpha_k)

[x_{k+1}=x_k+alpha_kd_k ]

(k=k+1),to step 1;

matlab code

function [x,val,fun_t] = conjugate_gradient(fun,gfun,x0,max_ite)
%myFun - Description
%
% Syntax: [x,val,fun_t] = myFun(fun,gfun,x0)
%
% conjugate gradient algorithm
    maxk=max_ite;
    rho=0.6;Sigma=0.4;
    k=0;epsilon=1e-4;
    n=length(x0);
    fun_t=zeros(1,max_ite);

    while k<maxk

        g=gfun(x0);
        itern=k-(n+1)*floor(k/(n+1));
        itern=itern+1;
        if(itern==1)
            d=-g;
        else
            Beta=(g'*g)/(g0'*g0);
            d=-g+Beta*d0;
            gd=g'*d;
    
            if gd>=0.0
                d=-g;
            end
        end
        if norm(g)<epsilon
            break;
        end
        m=0;mk=0;
        while m<20
            if fun(x0+rho^m*d)<fun(x0)+Sigma*rho^m*g'*d
                mk=m;
                break;
            end
            m=m+1;
        end
        x0=x0+rho^mk*d;
        g0=g;d0=d;
        
        k=k+1;
        fun_t(1,k)=fun(x0);
    end

    x=x0;
    val=fun(x0);
end

main code

%%%%%%%%conjugate  gradient algorithm
clc;
close all;
fun=@(x) 100*(x(1)^2-x(2))^2+(x(1)-1)^2;
gfun=@(x) [400*(x(1)^2-x(2))*x(1)+2*(x(1)-1);-200*(x(1)^2-x(2))];
x0=[0;0];
max_ite=200; %%number of iterations

[x,val,fun_t] = conjugate_gradient(fun,gfun,x0,max_ite);

disp(x);
disp(val);
figure(1);
plot(1:max_ite,fun_t);
set(get(gca, 'XLabel'), 'String', 'number of iterations');
set(get(gca, 'YLabel'), 'String', 'function value');

result

conclusion

共轭梯度算法介于梯度下降和牛顿法之间，快于线性收敛，只需要梯度，不用计算二阶导数；

reference

《最优化方法及其matlab程序设计》