zoukankan      html  css  js  c++  java
  • K-近邻算法(KNN)

    原理:

    存在一个样数据集合,样本集中每个数据都存在标签,输入没有标签的新数据之后,将新数据的每个特征与样本数据的对应特征进行比较,算法提取出样本集中特征最相似的k个数据,然后这k个数据中出现次数最多的分类作为新数据的分类。

    k越大,决策边界越平滑。实际中选择k,cross validation!

    优缺点:

    精度高,对异常值不敏感。

    缺点:计算复杂度高,空间复杂的高。

    kNN基本代码,数字识别代码:

    # _*_ coding:UTF8 _*_
    # 测试demo 约会网站 数字识别
    '''
    Created on Sep 16, 2010
    kNN: k Nearest Neighbors
    
    Input:      inX: vector to compare to existing dataset (1xN)
                dataSet: size m data set of known vectors (NxM)
                labels: data set labels (1xM vector)
                k: number of neighbors to use for comparison (should be an odd number)
                
    Output:     the most popular class label
    
    @author: pbharrin
    '''
    from numpy import *
    import operator
    from os import listdir
    
    def classify0(inX, dataSet, labels, k):
        dataSetSize = dataSet.shape[0]
        diffMat = tile(inX, (dataSetSize,1)) - dataSet #tile函数复制
        sqDiffMat = diffMat**2
        print type(sqDiffMat)
        sqDistances = sqDiffMat.sum(axis=1) #按照行相加
        distances = sqDistances**0.5
        sortedDistIndicies = distances.argsort() #  存储的是下标
        classCount={}          
        for i in range(k):
            voteIlabel = labels[sortedDistIndicies[i]]
            classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
        sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True)
        return sortedClassCount[0][0]
    
    def createDataSet():
        group = array([[1.0,1.1],[1.0,1.0],[0,0],[0,0.1]])
        labels = ['A','A','B','B']
        return group, labels
    
    def file2matrix(filename):
        fr = open(filename)
        numberOfLines = len(fr.readlines())         #get the number of lines in the file
        returnMat = zeros((numberOfLines,3))        #prepare matrix to return
        classLabelVector = []                       #prepare labels return   
        fr = open(filename)
        index = 0
        for line in fr.readlines():
            line = line.strip()
            listFromLine = line.split('	')
            returnMat[index,:] = listFromLine[0:3]
            classLabelVector.append(int(listFromLine[-1]))
            index += 1
        return returnMat,classLabelVector
        
    def autoNorm(dataSet):
        minVals = dataSet.min(0)
        maxVals = dataSet.max(0)
        ranges = maxVals - minVals
        normDataSet = zeros(shape(dataSet))
        m = dataSet.shape[0]
        normDataSet = dataSet - tile(minVals, (m,1))
        normDataSet = normDataSet/tile(ranges, (m,1))   #element wise divide
        return normDataSet, ranges, minVals
       
    def datingClassTest():
        hoRatio = 0.50      #hold out 10%
        datingDataMat,datingLabels = file2matrix('datingTestSet2.txt')       #load data setfrom file
        normMat, ranges, minVals = autoNorm(datingDataMat)
        m = normMat.shape[0]
        numTestVecs = int(m*hoRatio)
        errorCount = 0.0
        for i in range(numTestVecs):
            classifierResult = classify0(normMat[i,:],normMat[numTestVecs:m,:],datingLabels[numTestVecs:m],3)
            print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, datingLabels[i])
            if (classifierResult != datingLabels[i]): errorCount += 1.0
        print "the total error rate is: %f" % (errorCount/float(numTestVecs))
        print errorCount
        
    def img2vector(filename):
        returnVect = zeros((1,1024))
        fr = open(filename)
        for i in range(32):
            lineStr = fr.readline()
            for j in range(32):
                returnVect[0,32*i+j] = int(lineStr[j])
        return returnVect
    
    def handwritingClassTest():
        hwLabels = []
        trainingFileList = listdir('digits/trainingDigits')           #load the training set
        m = len(trainingFileList)
        trainingMat = zeros((m,1024))
        for i in range(m):
            fileNameStr = trainingFileList[i]
            fileStr = fileNameStr.split('.')[0]     #take off .txt
            if fileStr == '' :
                continue
            classNumStr = int(fileStr.split('_')[0])
            hwLabels.append(classNumStr)
            trainingMat[i,:] = img2vector('digits/trainingDigits/%s' % fileNameStr)
        testFileList = listdir('digits/testDigits')        #iterate through the test set
        errorCount = 0.0
        mTest = len(testFileList)
        for i in range(mTest):
            fileNameStr = testFileList[i]
            fileStr = fileNameStr.split('.')[0]     #take off .txt
            if fileStr == '':
                continue
            classNumStr = int(fileStr.split('_')[0])
            vectorUnderTest = img2vector('digits/testDigits/%s' % fileNameStr)
            classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3)
            print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr)
            if (classifierResult != classNumStr): errorCount += 1.0
        print "
    the total number of errors is: %d" % errorCount
        print "
    the total error rate is: %f" % (errorCount/float(mTest))
    View Code

     Kaggle 数字识别kNN代码

    def kaggleHandWriting():
        data = loadData('Kaggle/train.csv')
        m = len(data)
        n = len(data[0]) - 1
        hwLabels = []
        trainData = zeros((m, n))
        for i in range(m):
            hwLabels.append(int(data[i][0]))
            for j in range(n):
                trainData[i][j] = int(data[i][j + 1])
        testData = loadData('Kaggle/test.csv')
        mTest = len(testData)
        predictLabels = []
        for i in range(mTest):
            curTest = []
            for j in range(n):
                curTest.append(int(testData[i][j]))
            label = classify0(curTest, trainData, hwLabels, 5)
            predictLabels.append(label)
            print '第 %d 个数为:%d'%(i,label)
        saveResult(predictLabels)
    
    def loadData(filename):
        data = []
        f = file(filename, 'rb')
        lines = csv.reader(f)
        for line in lines:
            print ','.join(line)
            data.append(line)
        del(data[0])
        f.close()
        return data
    def saveResult(result):
        f = file('Kaggle/sample_submission.csv','wb')
        myWriter = csv.writer(f)
        myWriter.writerow(['ImageId','Label'])
        m = len(result)
        for i in range(m):
            myWriter.writerow([i,result[i]])
        f.close()
    View Code
  • 相关阅读:
    02安卓用户界面优化之(三)如何使用菜单
    07-业务敏捷:帮助DevOps快速落地的源动力
    转型之路:企业实施DevOps的常见路径和问题
    价值流分析:关于DevOps转型,我们应该从何处入手
    DevOps的衡量:你是否找到了DevOps的实施路线图
    DevOps的实施:到底是工具先行还是文化先行
    DevOps的价值:数字化转型时代,DevOps是必选项
    DevOps的“定义”:DevOps究竟要解决什么问题
    Jenkins产品经理是如何设计产品的
    关于DevOps组织和文化的那些趣事儿.
  • 原文地址:https://www.cnblogs.com/futurehau/p/6389014.html
Copyright © 2011-2022 走看看