zoukankan      html  css  js  c++  java
  • 线性回归 随机梯度下降SGD (Tensorflow 2.1)

     采用类的方式,参考链接

    import tensorflow as tf
    
    x_data = tf.Variable(tf.random.uniform((1,3), -1.0, 1.0))
    y_data = x_data * 0.1 + 0.3
    
    class Linear(tf.keras.Model):
        def __init__(self):
            super().__init__()
            self.dense = tf.keras.layers.Dense(
                units=1,
                activation=None,
                kernel_initializer=tf.zeros_initializer(),
                bias_initializer=tf.zeros_initializer()
            )
    
        def call(self, input):
            output = self.dense(input)
            return output
    
    model = Linear()
    optimizer = tf.keras.optimizers.SGD(learning_rate = 1e-2)
    
    for i in range(100):
        with tf.GradientTape() as tape:
            y_pred = model(x_data)
            loss = tf.reduce_mean(tf.square(y_pred - y_data))
        grads = tape.gradient(loss, model.variables)
        optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
        if i % 20 == 0:
            print(i,":loss == ", loss)

     --------------------

    在更新一波,修改了梯度的部分

    import tensorflow as tf
    import numpy as np
    
    # name: create data
    # function:
    #   np.random.rand()
    #       1.当函数括号内没有参数时,则返回一个浮点数;
    #       2.当函数括号内有一个参数时,则返回秩为1的数组,不能表示向量和矩阵;
    #       3.当函数括号内有两个及以上参数时,则返回对应维度的数组,能表示向量或矩阵;
    #       4.通过本函数可以返回一个或一组服从“0~1”均匀分布的随机样本值。随机样本取值范围是[0,1),不包括1.
    #   astype()
    #       1.转化数据类型
    x_data = np.random.rand(100).astype(np.float32)
    y_data = x_data * 0.1 + 0.3
    
    # name: create tensorflow structure
    # function:
    #   tf.Variable()
    #       1.tf.Variable(initializer,name),参数initializer是初始化参数,name是可自定义的变量名称
    #   tf.random.uniform()
    #       tf.random.uniform(shape, minval=0, maxval=None, dtype=tf.dtypes.float32, seed=None, name=None)
    #       shap是矩阵维数
    #   tf.zeros()
    #       tf.zeros(shape, dtype=tf.dtypes.float32, name=None)
    Weights = tf.Variable(tf.random.uniform((1,), -1.0, 1.0))
    biases = tf.Variable(tf.zeros((1,)))
    
    variables = [Weights, biases]
    
    num_epoch = 1000
    optimizer = tf.keras.optimizers.SGD(learning_rate = 1e-3)
    for e in range(num_epoch):
        with tf.GradientTape() as tape:
            y_pre = Weights * x_data + biases
            loss = 0.5 * tf.reduce_sum(tf.square(y_pre - y_data))
        grads = tape.gradient(loss, variables)
        optimizer.apply_gradients(grads_and_vars=zip(grads, variables))
        if e % 20 == 0:
            print(e, ": loss == ", loss)

    -------------------------

    import tensorflow as tf
    import numpy as np
    
    # name: create data
    # function:
    #   np.random.rand()
    #       1.当函数括号内没有参数时,则返回一个浮点数;
    #       2.当函数括号内有一个参数时,则返回秩为1的数组,不能表示向量和矩阵;
    #       3.当函数括号内有两个及以上参数时,则返回对应维度的数组,能表示向量或矩阵;
    #       4.通过本函数可以返回一个或一组服从“0~1”均匀分布的随机样本值。随机样本取值范围是[0,1),不包括1.
    #   astype()
    #       1.转化数据类型
    x_data = np.random.rand(100).astype(np.float32)
    y_data = x_data * 0.1 + 0.3
    
    # name: create tensorflow structure
    # function:
    #   tf.Variable()
    #       1.tf.Variable(initializer,name),参数initializer是初始化参数,name是可自定义的变量名称
    #   tf.random.uniform()
    #       tf.random.uniform(shape, minval=0, maxval=None, dtype=tf.dtypes.float32, seed=None, name=None)
    #       shap是矩阵维数
    #   tf.zeros()
    #       tf.zeros(shape, dtype=tf.dtypes.float32, name=None)
    Weights = tf.Variable(tf.random.uniform((1,), -1.0, 1.0))
    biases = tf.Variable(tf.zeros((1,)))
    
    #name: loss function
    #function:
    #   tf.keras.losses.MSE()
    #       tf.keras.losses.MSE(y_true, y_pred)
    #       y_true真实值, y_pred预测值
    #   tf.keras.optimizers.SGD() 随机梯度下降
    #       tf.keras.optimizers.SGD(learning_rate=0.01, momentum=0.0, nesterov=False, name='SGD', **kwargs)
    def loss():
        return tf.keras.losses.MSE(y_data, Weights * x_data + biases)
    optimizer = tf.keras.optimizers.SGD(learning_rate=0.5)
    
    # minimize()
    #其中minimize()包含两个步骤:
    #   1.计算loss对指定val_list的梯度(导数),返回元组列表[(gradient,variable),…]
    #   compute_gradients(loss,val_list)
    #   注:tf.gradients(loss, tf.variables)与compute_gradients(loss,val_list)作用类似,但是只返回梯度
    #   2.用计算得到的梯度来更新对应的变量(权重)
    #   optimizer.apply_gradients(grads_and_vars, global_step=global_step, name=None)将
    #   compute_gradients(loss,val_list)的返回值作为输入对variable更新
    #所以一下写法可能隐藏梯度爆炸和梯度消失
    #参考链接https://blog.csdn.net/sinat_37386947/article/details/88849519
    for step in range(201):
        optimizer.minimize(loss, var_list=[Weights, biases])
        if step % 20 == 0:
            print("{} step, weights = {}, biases = {}".format(step, Weights.read_value(), biases.read_value()))  # read_value函数可用numpy替换
  • 相关阅读:
    HDU 2544 最短路
    HDU 3367 Pseudoforest
    USACO 2001 OPEN
    HDU 3371 Connect the Cities
    HDU 1301 Jungle Roads
    HDU 1879 继续畅通工程
    HDU 1233 还是畅通工程
    HDU 1162 Eddy's picture
    HDU 5745 La Vie en rose
    HDU 5744 Keep On Movin
  • 原文地址:https://www.cnblogs.com/lalalatianlalu/p/12433043.html
Copyright © 2011-2022 走看看