zoukankan      html  css  js  c++  java
  • tensorflow 中的L1和L2正则化

    import tensorflow as tf
    weights = tf.constant([[1.0, -2.0],[-3.0 , 4.0]])
    
    >>> sess.run(tf.contrib.layers.l1_regularizer(0.5)(weights))
    5.0
    >>> sess.run(tf.keras.regularizers.l1(0.5)(weights))
    5.0
    >>> sess.run(tf.keras.regularizers.l1()(weights))
    0.099999994
    >>> sess.run(tf.keras.regularizers.l1(1)(weights))
    10.0
    >>> sess.run(tf.nn.l2_loss(weights))
    15.0
    >>> sess.run(tf.keras.regularizers.l2(1)(weights))
    30.0
    >>> sess.run(tf.keras.regularizers.l2(0.5)(weights))
    15.0
    >>> sess.run(tf.contrib.layers.l1_regularizer(0.5)(weights))
    5.0
    >>> sess.run(tf.contrib.layers.l2_regularizer(0.5)(weights))
    7.5
    >>> sess.run(tf.contrib.layers.l2_regularizer(1.0)(weights))
    15.0

    在tensorflow中,tf.nn中只有tf.nn.l2_loss,却没有l1_loss,于是自己网上查阅资料,了解到tf.contrib.layers中有tf.contrib.layers.l1_regularizer(),但是tf.contrib目前新版本已经被弃用了,后来发现tf.keras.regularizers下面有l1和l2正则化器,但是该正则化器的l2有点不一样,从上面的结果可以看出,scale都为1时,它要多一倍。可以查看源代码,tf.nn.l2_loss和 tf.contrib.layers.l2_regularizer 中都统一除以了2.所以值要少一半。

    >>> sess.run(tf.nn.l2_loss(weights))
    15.0
    >>> sess.run(tf.keras.regularizers.l2(1)(weights))
    30.0
    >>> sess.run(tf.contrib.layers.l2_regularizer(1.0)(weights))
    15.0
    将scale设为0.5后,可以得到一样的值,因此,以后在损失函数中可以使用这样的方式来求l2损失和l1损失。
    >>> sess.run(tf.keras.regularizers.l2(0.5)(weights))
    15.0
    参考了  day-17 L1和L2正则化的tensorflow示例 - 派森蛙 - 博客园

    https://www.cnblogs.com/python-frog/p/9416970.html

    '''
    输入:
    x = [[1.0,2.0]]
    w = [[1.0,2.0],[3,0,4.0]]
    
    输出:
    y = x*w = [[7.0,10.0]]
    l1 = (1.0+2.0+3.0+4.0)*0.5 = 5.0
    l2 = (1.0**2 + 2.0**2 + 3.0**2 + 4.0**2) / 2)*0.5 = 7.5
    '''
    
    import tensorflow as tf
    from tensorflow.contrib.layers import *
    
    w = tf.constant([[1.0,2.0],[3.0,4.0]])
    x = tf.placeholder(dtype=tf.float32,shape=[None,2])
    y = tf.matmul(x,w)
    
    with tf.Session()  as sess:
        init = tf.global_variables_initializer()
        sess.run(init)
        print(sess.run(y,feed_dict={x:[[1.0,2.0]]}))
        print("=========================")
        print(sess.run(l1_regularizer(scale=0.5)(w)))
        #(1.0+2.0+3.0+4.0)*0.5 = 5.0
        print("=========================")
        print(sess.run(l2_regularizer(scale=0.5)(w)))
        #(1.0**2 + 2.0**2 + 3.0**2 + 4.0**2) / 2)*0.5 = 7.5
  • 相关阅读:
    [BZOJ 3282] Tree 【LCT】
    [BZOJ 2049] [Sdoi2008] Cave 洞穴勘测 【LCT】
    [BZOJ 1036] [ZJOI2008] 树的统计Count 【Link Cut Tree】
    [HDOJ
    Excel+DDT数据驱动实例
    jenkins+SVN配置
    [转]loadrunner:系统的平均并发用户数和并发数峰值如何估算
    loadrunner:Auto Correlate自动定位瓶颈
    loadrunner:判断是否服务器连接池瓶颈
    利用page_source抓取网页中的URL,进行链接测试
  • 原文地址:https://www.cnblogs.com/Jerry-home/p/12377929.html
Copyright © 2011-2022 走看看