zoukankan      html  css  js  c++  java
  • Pytorch lr_scheduler 中的 last_epoch 用法

    The last_epoch parameter is used when resuming training and you want to start the scheduler where it left off earlier. Its value is increased every time you call .step() of scheduler. The default value of -1 indicates that the scheduler is started from the beginning.

    From the docs:

    Since step() should be invoked after each batch instead of after each epoch, this number represents the total number of batches computed, not the total number of epochs computed. When last_epoch=-1, the schedule is started from the beginning.

    For example,

    >>> import torch
    >>> cc = torch.nn.Conv2d(10,10,3)
    >>> myoptimizer = torch.optim.Adam(cc.parameters(), lr=0.1)
    >>> myscheduler = torch.optim.lr_scheduler.StepLR(myoptimizer,step_size=1, gamma=0.1)
    >>> myscheduler.last_epoch, myscheduler.get_lr()
    (0, [0.1])
    >>> myscheduler.step()
    >>> myscheduler.last_epoch, myscheduler.get_lr()
    (1, [0.001])
    >>> myscheduler.step()
    >>> myscheduler.last_epoch, myscheduler.get_lr()
    (2, [0.0001])
    

    Now, if you decide to stop the training in the middle, then resume it, you can provide last_epoch parameter to schedular so that it start from where it was left off, not from the beginning again.

    >>> mynewscheduler = torch.optim.lr_scheduler.StepLR(myoptimizer,step_size=1, gamma=0.1, last_epoch=myscheduler.last_epoch)
    >>> mynewscheduler.last_epoch, mynewscheduler.get_lr()
    (3, [1.0000000000000004e-05])


    原文链接:https://stackoverflow.com/questions/62724824/what-is-the-param-last-epoch-on-pytorch-optimizers-schedulers-is-for



    如果这篇文章帮助到了你,你可以请作者喝一杯咖啡

  • 相关阅读:
    WP8.1 UI 编程 四、图形
    分治法 全排列问题的一个Java实现
    合并排序算法时间复杂度分析
    js如何实现复制粘贴功能
    关于<meta name="applicable-device"content="pc">
    ready
    css文字强制大写输入
    关于input,button标签在苹果手机上显示阴影解决办法
    【C#】中用if判断值是否是否为空
    视频格式MP4,需要转码
  • 原文地址:https://www.cnblogs.com/sddai/p/14627966.html
Copyright © 2011-2022 走看看