zoukankan      html  css  js  c++  java
  • 5.机器学习第二周(2)

    Computing Parameters Analytically

    1 Normal Equation(正规方程)

    Gradient descent gives one way of minimizing J. Let’s discuss a second wayof doing so, this time performing the minimization explicitly and withoutresorting to an iterative algorithm. In the "Normal Equation" method, we will minimize J byexplicitly taking its derivatives with respect to the θj’s, and setting them tozero. This allows us to find the optimum theta without iteration. The normal equation formula is given below:

    $$ heta = (X^T X){-1}XT y$$

    img

    There is no need to do feature scaling with the normal equation.

    The following is a comparison of gradient descent and the normal equation:

    Gradient Descent Normal Equation
    Need to choose alpha No need to choose alpha
    Needs many iterations No need to iterate
    O ($$kn^2$$) O ($$n^2$$), need to calculate inverse of $$X^TX$$
    Works well when n is large Slow if n is very large

    With the normal equation, computing the inversion has complexity O($$n^3$$). So if we have a very large number of features, the normal equation will be slow. In practice, when n exceeds 10,000 it might be a good time to go from a normal solution to an iterative process.

    2. Normal Equation Noninvertibility

    When implementing the normal equation in octave we want to use the 'pinv' function rather than 'inv.' The 'pinv' function will give you a value of θ even if $$X^TX$$ is not invertible.

    If $$X^TX$$ is noninvertible, the common causes might be having :

    • Redundant features, where two features are very closely related (i.e. they are linearly dependent)
    • Too many features (e.g. m ≤ n). In this case, delete some features or use "regularization" (to be explained in a later lesson).

    Solutions to the above problems include deleting a feature that is linearly dependent with another or deleting one or more features when there are too many features.

  • 相关阅读:
    APP性能之终端兼容优化分享
    Java反射机制
    语音编码的WAVE文件头格式剖析
    【原创】ASP.NET MVC3开发中遇到问题以及解决方法
    linux常用命令(基础)
    vue中粘贴板clipboard的使用方法
    解决部署zabbix中zabbixagent的状态为灰色现象
    IAR Embedded Workbench for ARM: Porting code from V4 to V5 ( for stm32)
    MSDN帮助文档 "无法显示该网页" 的问题解决方案(转)
    二叉排序树求每个结点平衡因子程序
  • 原文地址:https://www.cnblogs.com/flysevenwu/p/6146841.html
Copyright © 2011-2022 走看看