zoukankan      html  css  js  c++  java
  • What's Dropout?

    Dropout is one of usual regularization methods in Machine Learning especially in Deep Learning. It could prevent model from over-fitting during train period.

    Let's consider a condition that out data has some noisy. For example, a picture is added some noisy randomly, which results some areas becomes black from white. For people, that slightly change could be ignored and we would still realize its shape. But for the model which is trained from data, a bit change would impact its inference.

    Naturally, we want our model have the ability to ignore noisy and do inference correctly. To get this ability, we could introduce some noisy to our model when it's training. It means that in our training period, we can add some noise to the data. In the other hand, we hope that distribution of the data would be changeless, which means its expectation value would be same.

    [E(x_i^{'}) = x_i ]

    For example, we could change the value with probability.

    [egin{equation*} x_i^{prime}=left{egin{array}{ll} 0 & ext { 概率为 } p \ frac{x_i}{1-p} & ext { 其他情况 } end{array} ight. end{equation*} ]

    As above equation displayed, we change (x_i) to 0 with probability p, and to (frac{x_i}{1-p}) with probability 1- p. That's what Dropout does in training.

  • 相关阅读:
    Python之 continue继续循环
    Python之 break退出循环
    Python之 while循环
    Python之 for循环
    java中collection、map、set、list简介 (转)
    CloudSim介绍和使用
    智能指针和动态内存
    boost signal2
    boost 信号 :
    boost库 线程使用
  • 原文地址:https://www.cnblogs.com/shayue/p/14676830.html
Copyright © 2011-2022 走看看