zoukankan      html  css  js  c++  java
  • 5.机器学习第二周(2)

    Computing Parameters Analytically

    1 Normal Equation(正规方程)

    Gradient descent gives one way of minimizing J. Let’s discuss a second wayof doing so, this time performing the minimization explicitly and withoutresorting to an iterative algorithm. In the "Normal Equation" method, we will minimize J byexplicitly taking its derivatives with respect to the θj’s, and setting them tozero. This allows us to find the optimum theta without iteration. The normal equation formula is given below:

    $$ heta = (X^T X){-1}XT y$$

    img

    There is no need to do feature scaling with the normal equation.

    The following is a comparison of gradient descent and the normal equation:

    Gradient Descent Normal Equation
    Need to choose alpha No need to choose alpha
    Needs many iterations No need to iterate
    O ($$kn^2$$) O ($$n^2$$), need to calculate inverse of $$X^TX$$
    Works well when n is large Slow if n is very large

    With the normal equation, computing the inversion has complexity O($$n^3$$). So if we have a very large number of features, the normal equation will be slow. In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.

    2. Normal Equation Noninvertibility

    When implementing the normal equation in octave we want to use the 'pinv' function rather than 'inv.' The 'pinv' function will give you a value of θ even if $$X^TX$$ is not invertible.

    If $$X^TX$$ is noninvertible, the common causes might be having :

    • Redundant features, where two features are very closely related (i.e. they are linearly dependent)
    • Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).

    Solutions to the above problems include deleting a feature that is linearly dependent with another or deleting one or more features when there are too many features.

  • 相关阅读:
    1009 说反话 (20 分)
    1007 素数对猜想 (20 分)
    Visual Studio2017下载方法
    百词斩和扇贝打卡测试与评估
    创建者模式->工厂模式
    STM32F427|----------IO讲解与应用
    MySQL的索引
    STM32F103RCt6 与 MG996R
    2020物联网实验室的考核题目
    初学数据库-MySQL&IDEA&Navicat
  • 原文地址:https://www.cnblogs.com/flysevenwu/p/6146841.html
Copyright © 2011-2022 走看看