多分类例题 - 手写数字识别
提供的数据集包括5000张手写数字0~9图片及对应的正确数字值。其中，每一张图片已被预处理成20 * 20 像素的灰白图片，并转化成灰度存入到矩阵中。要求利用OVA算法进行手写数字识别。
绘制训练集
本题的图片绘制涉及灰度的一些内容，我并不了解。这里使用coursera的代码进行绘制（因此也没必要展示了），简单观察一下各个图片。
设计思路
- 由于输出有0～9十种状态，因此需要拟合10个目标函数，分别估计给定数字为0～9的可能性。最终取可能性最大的数字作为最终预测答案。
- 由于每一个数据都是20*20像素的灰白图片，因此可以将其转化1*400的向量。即每一个输入值有400个属性。
- 由于matlab下标从1开始，因此在for循环中求0的目标函数不太方便。因此coursera提供的数据集中所有的y=0均用y=10代替。（其实没必要这么做，绕来绕去更加麻烦。还不如直接mod10）
训练
首先写出代价函数的计算函数，方便后期调用
function [J, grad] = lrCostFunction(theta, X, y, lambda)
%LRCOSTFUNCTION Compute cost and gradient for logistic regression with
%regularization
% J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using
% theta as the parameter for regularized logistic regression and the
% gradient of the cost w.r.t. to the parameters.
% Initialize some useful values
m = length(y); % number of training examples
% You need to return the following variables correctly
J = 0;
grad = zeros(size(theta));
% ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
% You should set J to the cost.
% Compute the partial derivatives and set grad to the partial
% derivatives of the cost w.r.t. each parameter in theta
%
% Hint: The computation of the cost function and gradients can be
% efficiently vectorized. For example, consider the computation
%
% sigmoid(X * theta)
%
% Each row of the resulting matrix will contain the value of the
% prediction for that example. You can make use of this to vectorize
% the cost function and gradient computations.
%
% Hint: When computing the gradient of the regularized cost function,
% there're many possible vectorized solutions, but one solution
% looks like:
% grad = (unregularized gradient for logistic regression)
% temp = theta;
% temp(1) = 0; % because we don't add anything for j = 0
% grad = grad + YOUR_CODE_HERE (using the temp variable)
%
J = -1/m * sum(y .* log(sigmoid(X*theta))+(1-y) .* log(1-sigmoid(X*theta))) + lambda / (2*m) * sum(theta(2:end,:) .* theta(2:end,:));
temp = theta;
temp(1) = 0;
grad = 1/m * X' * (sigmoid(X*theta)-y) + lambda/m * temp;
% =============================================================
grad = grad(:);
end
然后在开始利用OVA训练10个目标函数。利用for循环分别对1至10（还记得么，数字0用10表示）求解目标函数。
function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta
%corresponds to the classifier for label i
% [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
% logistic regression classifiers and returns each of these classifiers
% in a matrix all_theta, where the i-th row of all_theta corresponds
% to the classifier for label i
% Some useful variables
m = size(X, 1);
n = size(X, 2);
% You need to return the following variables correctly
all_theta = zeros(num_labels, n + 1);
% Add ones to the X data matrix
X = [ones(m, 1) X];
% ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
% logistic regression classifiers with regularization
% parameter lambda.
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 1's and 0's that tell you
% whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
% function. It is okay to use a for-loop (for c = 1:num_labels) to
% loop over the different classes.
%
% fmincg works similarly to fminunc, but is more efficient when we
% are dealing with large number of parameters.
%
% Example Code for fmincg:
%
% % Set Initial theta
% initial_theta = zeros(n + 1, 1);
%
% % Set options for fminunc
% options = optimset('GradObj', 'on', 'MaxIter', 50);
%
% % Run fmincg to obtain the optimal theta
% % This function will return theta and the cost
% [theta] = ...
% fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
% initial_theta, options);
%
for idx=1:10
Init_theta = zeros(size(X,2),1);
options = optimset('GradObj','on','MaxIter',50);
[theta,~,~]= fmincg(@(t)lrCostFunction(t,X,(y==idx),lambda),Init_theta,options);
theta = theta';
all_theta(idx,:) = theta;
end
% =========================================================================
end
调用方法：
fprintf('
Training One-vs-All Logistic Regression...
')
lambda = 0.1;
[all_theta] = oneVsAll(X, y, num_labels, lambda);
fprintf('Program paused. Press enter to continue.
');
pause;
结果判定
下面这段代码来自coursera，它只能判断经验误差，而不能判断泛化误差。
pred = predictOneVsAll(all_theta, X);
fprintf('
Trraining Set Accuracy: %f
', mean(double(pred == y)) * 100);