Logistic Regression 用于预测马是否生病

1.利用Logistic regression 进行分类的主要思想

根据现有数据对分类边界线建立回归公式，即寻找最佳拟合参数集，然后进行分类。

2.利用梯度下降找出最佳拟合参数

3.代码实现

# -*- coding: utf-8 -*-
"""
Created on Tue Mar 28 21:35:25 2017

@author: MyHome
"""
import numpy as np
from random import uniform
'''定义sigmoid函数'''
def sigmoid(inX):
    return 1.0 /(1.0 +np.exp(-inX))

'''使用随机梯度下降更新权重，并返回最终值'''
def StocGradientDescent(dataMatrix,classLabels,numIter = 600):
    m,n = dataMatrix.shape
    #print m,n
    weights = np.ones(n)
    for j in xrange(numIter):
        dataIndex = range(m)

        for i in xrange(m):

            alpha = 4 / (1.0+j+i) + 0.01
            randIndex = int(uniform(0,len(dataIndex)))
            h = sigmoid(sum(dataMatrix[randIndex]*weights))
            gradient = (h - classLabels[randIndex])*dataMatrix[randIndex]
            weights = weights - alpha*gradient
            del(dataIndex[randIndex])

    return weights


'''创建分类器'''
def classifyVector(inX,weights):
    prob = sigmoid(sum(inX*weights))
    if prob > 0.5:
        return 1.0
    else:
        return 0.0

'''测试'''
def Test():

    frTrain = open("horseColicTraining.txt")
    frTest = open("horseColicTest.txt")
    trainingSet = []
    trainingLabel = []
    for line in frTrain.readlines():
        currLine = line.strip().split("	")
        lineArr = []
        for i in range(21):
            lineArr.append(float(currLine[i]))
        trainingSet.append(lineArr)
        trainingLabel.append(float(currLine[21]))
    trainWeights = StocGradientDescent(np.array(trainingSet),trainingLabel)
    errorCount = 0.0
    numTestVec = 0.0
    for line in frTest.readlines():
        numTestVec += 1.0
        currLine = line.strip().split("	")
        lineArr = []
        for i in range(21):
            lineArr.append(float(currLine[i]))
        if int(classifyVector(np.array(lineArr),trainWeights)) != int(currLine[21]):
            errorCount += 1
    errorRate = (float(errorCount)/numTestVec)
    print "the error rate of this test is:%f"%errorRate
    return errorRate

'''调用Test()10次求平均值'''
def multiTest():
    numTest = 10
    errorSum = 0.0
    for k in range(numTest):
        errorSum += Test()
    print "after %d iterations the average errror rate is:
        %f"%(numTest,errorSum/float(numTest))

if __name__ == "__main__":
    multiTest()

结果：

the error rate of this test is:0.522388
the error rate of this test is:0.328358
the error rate of this test is:0.313433
the error rate of this test is:0.358209
the error rate of this test is:0.298507
the error rate of this test is:0.343284
the error rate of this test is:0.283582
the error rate of this test is:0.313433
the error rate of this test is:0.343284
the error rate of this test is:0.358209
after 10 iterations the average errror rate is:        0.346269

4.总结

Logistic regression is finding best-fit parameters to a nonlinear function called the sigmoid.
Methods of optimization can be used to find the best-fit parameters. Among the
optimization algorithms, one of the most common algorithms is gradient descent. Gradient
desent can be simplified with stochastic gradient descent.
Stochastic gradient descent can do as well as gradient descent using far fewer computing
resources. In addition, stochastic gradient descent is an online algorithm; it can
update what it has learned as new data comes in rather than reloading all of the data
as in batch processing.
One major problem in machine learning is how to deal with missing values in the
data. There’s no blanket answer to this question. It really depends on what you’re
doing with the data. There are a number of solutions, and each solution has its own
advantages and disadvantages.