zoukankan      html  css  js  c++  java
  • Perceptron Algorithm for Classification

    In classification tasks, we are given a training set T={X,t}, where X={xi} is the set of training data vectors and t={ti} is the set of vector labels.
        
        Linear classifiers use linear functions to model the mapping from input vector to output labels, that is, the classifier is defined as a function of linear combination of the input vector
        
        t=f(\sum_i x_i \theta_i) = f(x*\theta)
        
        In binary classification problems, $f$ is usually a sign function where
        
        f(y) = { +1, y >= 0;
                 {  -1, y < 0.

        Therefore

        x * \theta = 0

        defined a decision plane, theta is the normal vector of this plane. So the problem turns into finding the optimal decision plane, i.e. optimal theta to separate the training points according to their labels.
        
        We can use the number of classification mistakes as the evaluation metric for all the planes. Other metrics can also be used here. It is easy to observe that when we make a mistake t * f(x * \theta) < 0 which is equivalent to t x * \theta. So we hope to increase t x * \theta for all points that are mistakenly classified. Therefore, we maximize the loss function

            L = \sum_i t_i x_i * \theta

        
        The algorithm is to adapt theta for each mistakenly classified test point to increase the above target function. That is
        
        1. Repeat until convergence
        2.   For each training point (x, t)
        3.     If tx * theta < 0 (i.e. there is a mistake)
        4.       \theta <-- \theta + tx
        
        After each adaptation tx * theta will increase because

        tx*(\theta + tx) = tx * \theta + ||x||^2
                              >  tx * \theta
        
        This is the perceptron algorithm.
        
        After each adaptation, the \theta may be adjusted to perform better on the mistakenly classified training point, but other new mistakes may also appear. It is proved that this perceptron algorithm will converge when the points are linearly separable. Otherwise this algorithm may never converge.
  • 相关阅读:
    django集成django-xadmin
    Django设置 DEBUG=False后静态文件无法加载解决
    Django ORM必会的查询方法
    django admin-过滤器
    Django settings.py 中设置访问 MySQL 数据库【一种是直接在 settings.py 文件中直接写数据库信息,另一种是读文件获取数据库信息】
    django-admin之ModelAdmin最全解释
    SPL(Standard PHP Library 标准PHP类库)
    rsync 数据同步
    PHP 安装memcache.so 和memcached.so
    linux 安装memcached
  • 原文地址:https://www.cnblogs.com/alexdeblog/p/3119734.html
Copyright © 2011-2022 走看看