zoukankan      html  css  js  c++  java
  • What's Dropout?

    Dropout is one of usual regularization methods in Machine Learning especially in Deep Learning. It could prevent model from over-fitting during train period.

    Let's consider a condition that out data has some noisy. For example, a picture is added some noisy randomly, which results some areas becomes black from white. For people, that slightly change could be ignored and we would still realize its shape. But for the model which is trained from data, a bit change would impact its inference.

    Naturally, we want our model have the ability to ignore noisy and do inference correctly. To get this ability, we could introduce some noisy to our model when it's training. It means that in our training period, we can add some noise to the data. In the other hand, we hope that distribution of the data would be changeless, which means its expectation value would be same.

    [E(x_i^{'}) = x_i ]

    For example, we could change the value with probability.

    [egin{equation*} x_i^{prime}=left{egin{array}{ll} 0 & ext { 概率为 } p \ frac{x_i}{1-p} & ext { 其他情况 } end{array} ight. end{equation*} ]

    As above equation displayed, we change (x_i) to 0 with probability p, and to (frac{x_i}{1-p}) with probability 1- p. That's what Dropout does in training.

  • 相关阅读:
    第37天新版动画系统和有限状态机
    第36天旧版动画系统
    第35天2D游戏相关
    第34天协同程序和异步加载
    第33天力、射线检测、球形检测和延迟函数
    第32天Line渲染器,物理系统和力
    第31天Camera组件和灯光组件
    第29天动态加载、对象池
    第28天3D数学
    第27天3D数学
  • 原文地址:https://www.cnblogs.com/shayue/p/14676830.html
Copyright © 2011-2022 走看看