zoukankan      html  css  js  c++  java
  • Boosting

    Boosting is a greedy alogrithm. The alogrithm works by applying the weak learner sequentially to weighted version of the data, where more weight is given to examples that were misclassified by earlier rounds. Breiman( 1998) showed that boosting can be interperted as a form of gradient descent in function space. This view was then extended in (Friedman et al. 2000), who showed how boosting could be extended to handle a variety of loss functions , including for regression, robust regression, Poission regression, etc. 

    1. Forward stagewise additive modeling:

       The goal of boosting is to solve the following optimization problem:

      (min_{f} sum_{i=1}^N L(y_i, f(x_i)))

    and (L(y,hat{y})) is some loss function, and f is assumed to be an (adaptive basis function model) ABM.

    the picture above portries some possible loss function and their corresponding algrithm names.

    2. The procedures of forward stagewise algorithm:

    Input: training data: ( T = {(x_1,y_1),(x_2,y_2),...,(x_N,y_N)}); Loss function (L(y,f(x))); basis function set: (b{x;r}).

    Output: addative model: f(x):

    (1)  Initialize (f_0(x)).

    (2) for m in  1,2,...,M:

      (a): minimize loss function:

        ((eta_m,r_m) = argmin_{eta,r} sum_{i = 1}^{N}L(y_i,f_{m-1}(x_i) + eta b(x_i;r)));

         then we got the parameters: (eta_m,r_m).

      (b): Update:

        (f_m(x) = f_{m-1} (x) = eta_m b_(x;r_m))

    (3) additive model:

      (f(x) = f_M(x) = sum_{m =1}^N eta_m b(x;r_m))

    Reference:

    1. Machine learning a probabilistic perspective 553-563.

    2. The elements of statistical learning

    3. http://blog.csdn.net/dark_scope/article/details/24863289

  • 相关阅读:
    iOS 静态、全局变量、常量
    原子性atomic/nonatomic
    iOS数组遍历
    iOS开发过程中易犯的小错误
    mac开启Airdrop的硬件要求
    Activity Monitor 闪退 & 无法进入睡眠
    在Linux中连接android设备
    网格布局(GridLayout) 行数与列数
    $符号报not defing 报错
    eclipse鼠标变成十字符号
  • 原文地址:https://www.cnblogs.com/vpegasus/p/6884671.html
Copyright © 2011-2022 走看看