tensorflow学习之softmax regression

电脑配置：win10 + Anaconda3 + pyton3.5 + vs2013 + tensorflow + Gpu980 + matlab2016b

softmax regression的详细介绍，请参考黄文坚的《tensorflow实战》的第3.2节。

原书pdf下载地址： 链接：https://pan.baidu.com/s/1sk8Qm4X 密码：28jk

原书code下载地址：链接：https://pan.baidu.com/s/1eR1LepW 密码：kmiz

我这里的贡献，主要将代码改写为能够直接调用我们matlab的数据集，比如COIL20数据集

其中读取数据在matlab，训练和识别在python

数据集读写代码如下：

function output = data_imread_MSE(name,sele_num)
% 用于 tensorflow下的 3.2节 softmax regression的数据读取
% 数据存储为细胞组形式，4个元祖分别为 训练矩阵，训练标签，测试矩阵，测试标签
% 其中 训练矩阵和测试矩阵都是一行一个样本
% 测试标签为 MSE的one-hot矩阵 一行只有一个元素为1 一行为一个样本的类标
addpath('H:2015629房师兄代码data set');
load (name);
fea = double(fea);
nnClass = length(unique(gnd));  % The number of classes;
num_Class = [];
for i = 1:nnClass
    num_Class = [num_Class length(find(gnd==i))]; %The number of samples of each class
end
%%------------------select training samples and test samples--------------%% 
Train_Ma  = [];
Train_Lab = [];
Test_Ma   = [];
Test_Lab  = [];
for j = 1:nnClass    
    idx = find(gnd==j);
    randIdx  = randperm(num_Class(j));
    Train_Ma = [Train_Ma; fea(idx(randIdx(1:sele_num)),:)];            % select select_num samples per class for training
    Train_Lab= [Train_Lab;gnd(idx(randIdx(1:sele_num)))];
    Test_Ma  = [Test_Ma;fea(idx(randIdx(sele_num+1:num_Class(j))),:)];  % select remaining samples per class for test
    Test_Lab = [Test_Lab;gnd(idx(randIdx(sele_num+1:num_Class(j))))];
end
Train_Ma = Train_Ma';                       % transform to a sample per column
Train_Ma = Train_Ma./repmat(sqrt(sum(Train_Ma.^2)),[size(Train_Ma,1) 1]);
Test_Ma  = Test_Ma';
Test_Ma  = Test_Ma./repmat(sqrt(sum(Test_Ma.^2)),[size(Test_Ma,1) 1]);  % -------------

label = unique(Train_Lab);
Train_Lab = bsxfun(@eq, Train_Lab, label');

label = unique(Test_Lab);
Test_Lab = bsxfun(@eq, Test_Lab, label');

output = cell(1,4);
output{1} = Train_Ma';
output{2} = Train_Lab;
output{3} = Test_Ma';
output{4} = Test_Lab;
end

其中softmax regression主函数如下：

# -*- coding: utf-8 -*-
"""
Created on Wed Dec 13 20:25:47 2017

@author: Administrator
"""

#%%
# Copyright 2015 The TensorFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
# 用matlab读取数据
data_name = 'COIL20.mat'
sele_num  = 4
import matlab.engine
eng = matlab.engine.start_matlab()
t = eng.data_imread_MSE(data_name,sele_num)
eng.quit()
#t = np.array(t)
Train_Ma  = np.array(t[0]).astype(np.float32)
Train_Lab = np.array(t[1]).astype(np.int8)
Test_Ma   = np.array(t[2]).astype(np.float32)
Test_Lab  = np.array(t[3]).astype(np.int8)
Num_fea   = Train_Ma.shape[1]
Num_Class = Train_Lab.shape[1]

import tensorflow as tf
sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, [None, Num_fea])

W = tf.Variable(tf.zeros([Num_fea, Num_Class]))
b = tf.Variable(tf.zeros([Num_Class]))

y = tf.nn.softmax(tf.matmul(x, W) + b)

y_ = tf.placeholder(tf.float32, [None, Num_Class])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))


train_step = tf.train.GradientDescentOptimizer(0.2).minimize(cross_entropy)

tf.global_variables_initializer().run()

for i in range(500):
    batch_xs = Train_Ma
    batch_ys = Train_Lab
    train_step.run({x: batch_xs, y_: batch_ys})

correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print(accuracy.eval({x: Test_Ma, y_: Test_Lab}))

识别结果如下

针对COIL20数据集，随机选取每类4个样本作为训练样本，余下为测试样本

当迭代次数为500时，选取不同的learning_rate时的对比

learnng_rate=0.2	0.5	0.8	1	5	10
69.71	72.43	73.38	73.60	74.85	75.15

当learning_rate为10时，选取　　

iter_num=100	500	1000	2000	3000
74.34	75.15	75.15	75.44	75.37

选取10个样本时的识别率大约为84.11，这与LRLR等传统方法的结果是差不多的。

本文代码下载链接如下：

链接：https://pan.baidu.com/s/1dFvXInB 密码：z2s6

当然，咱也可以用传统回归的损失函数：min |Y-WX| + lambda*|W|

regu = 0.01
cross_entropy = tf.reduce_sum(tf.multiply(y_-y,y_-y))+tf.reduce_sum(tf.multiply(W,W))*regu

当lambda=0.01,learning_rate=0.2,迭代次数为100时，也能得到82.02的识别率