zoukankan      html  css  js  c++  java
  • Various Optimization Algorithms For Training Neural Network

    from

    https://towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6

     

    Optimizers help to get results faster

    Gradient Descent

    Stochastic Gradient Descent

    Mini-Batch Gradient Descent

    Momentum

    Nesterov Accelerated Gradient

     

    NAG vs momentum at local minima

    Adagrad

     

    A derivative of loss function for given parameters at a given time t.

     

    Update parameters for given input i and at time/iteration t

    AdaDelta

     

    Update the parameters

    Adam

     

    First and second order of momentum

    Update the parameters

    Comparison between various optimizers

     

    Comparison 1

     

    comparison 2

    Conclusions

    出处:http://www.cnblogs.com/lightsong/ 本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接。
  • 相关阅读:
    Bolero and Data Mining
    2007年3月15日 网站论坛出现以下错误/forum/inc/Dv_ClsMain.asp,行 1344
    A Probabilistic Model for Retrospective News Event
    信息抽取的资料文档
    Textual Data Mining and WEBSOM
    DockPanel Suite更新到2.6了 武胜
    Use Custom Events from your WCF ServiceHost http://www.codeproject.com/Tips/150702/UseCustomEventsfromyourWCFServiceHost 武胜
    Unable to convert MySQL date/time value to System.DateTime 解决方案 转 武胜
    XML 转义字符 武胜
    Using Nini .NET Configuration Library 武胜
  • 原文地址:https://www.cnblogs.com/lightsong/p/14643083.html
Copyright © 2011-2022 走看看