zoukankan      html  css  js  c++  java
  • How to determine the correct number of epoch during neural network training? 如何确定Epoch

    The number of epochs is not that significant. More important is the the validation and training error. As long as it keeps dropping training should continue. For instance, if the validation error starts increasing that might be a indication of overfitting. You should set the number of epochs as high as possible and terminate training based on the error rates.

    Just to be clear, an epoch is one learning cycle where the learner sees the whole training data set. If you have two batches, the learner needs to go through two iterations for one epoch.

    If you have enough data, you can try Early Stopping method: divide data in three data sets, training, validation and evaluation. Train each network along a sufficient number of epochs to see the training Mean Squared Error to be stuck in a minimum. The training process uses training data-set and must be executed epoch by epoch, in order to calculate the Mean Squared Error of the network in each epoch for the validation set. The network for the epoch with the minimum validation MSE is selected for the evaluation process.

    This approach can be applied also with regularization methods and k-fold cross validation.
     
     
    You can follow the "elbow method". The basic idea is that, let's say your priority is the loss function. Then, make a line plot of your loss function vs the number of epochs. As soon as the slope of your line drops dramatically (to almost zero) that point is called an elbow point. That is the point increasing the number of epoch make only slight difference in the loss function. Consider in the picture below the y-axis represents the loss value and the x-axis represents the number of epochs. Then, clearly n=3 epoch is an elbow point.
     
     
     

    Well, the correct answer is the number of epochs is not that significant. more important is the validation and training error. As long as these two error keeps dropping, training should continue.

    For instance, if the validation error starts increasing that might be an indication of overfitting.

    You should set the number of epochs as high as possible and terminate the training when validation error start increasing。

     
     

    REF

    https://www.researchgate.net/post/How_to_determine_the_correct_number_of_epoch_during_neural_network_training

    https://www.researchgate.net/post/How_does_one_choose_optimal_number_of_epochs

    https://medium.com/@upendravijay2/what-is-epoch-and-how-to-choose-the-correct-number-of-epoch-d170656adaaf

  • 相关阅读:
    apt-clone安装与使用
    利用异或求(整数数组中,有2K+1个数,其中有2k个相同,找出不相同的那个数)
    运行程序,填写结果
    throw与throws的区别
    牛客网多线程程序执行结果选择题
    一个继承了抽象类的普通类的执行顺序
    int i=0;i=i++
    HashMap浅入理解
    &&和&、||和|的区别
    System.out.println()
  • 原文地址:https://www.cnblogs.com/emanlee/p/14403265.html
Copyright © 2011-2022 走看看