zoukankan      html  css  js  c++  java
  • 核范数求次梯度

    Start with the SVD decomposition of $x$:

    $$x=USigma V^T$$

    Then $$|x|_*=tr(sqrt{x^Tx})=tr(sqrt{(USigma V^T)^T(USigma V^T)})$$

    $$Rightarrow |x|_*=tr(sqrt{VSigma U^T USigma V^T})=tr(sqrt{VSigma^2V^T})$$

    By circularity of trace:

    $$Rightarrow |x|_*=tr(sqrt{V^TVSigma^2})=tr(sqrt{V^TVSigma^2})=tr(sqrt{Sigma^2})=tr(Sigma)$$

    Since the elements of $Sigma$ are non-negative.

    Therefore nuclear norm can be also defined as the sum of the absolute values of the singular value decomposition of the input matrix.

    Now, note that the absolute value function is not differentiable on every point in its domain, but you can find a subgradient.


    $$frac{partial |x|_*}{partial x}=frac{partial tr(Sigma)}{partial x}=frac{ tr(partialSigma)}{partial x}$$

    You should find $partialSigma$. Since $Sigma$ is diagonal, the subdifferential set of $Sigma$ is: $partialSigma=SigmaSigma^{-1}partialSigma$, now we have:

    $$frac{partial |x|_*}{partial x}=frac{ tr(SigmaSigma^{-1}partialSigma)}{partial x}$$ (I)

    So we should find $partialSigma$.

    $x=USigma V^T$, therefore:
    $$partial x=partial USigma V^T+UpartialSigma V^T+USigmapartial V^T$$

    Therefore:

    $$UpartialSigma V^T=partial x-partial USigma V^T-USigmapartial V^T$$

    $$Rightarrow U^TUpartialSigma V^TV=U^Tpartial xV-U^Tpartial USigma V^TV-U^TUSigmapartial V^TV$$


    $$Rightarrow partialSigma =U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV$$

    egin{align}
    Rightarrow\
    tr(partialSigma) &=& tr(U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV)\
    &=& tr(U^Tpartial xV)+tr(-U^Tpartial USigma - Sigmapartial V^TV)
    end{align}


    You can show that $tr(-U^Tpartial USigma - Sigmapartial V^TV)=0$ (Hint: diagonal and antisymmetric matrices, proof in the comments.), therefore:

    $$tr(partialSigma) = tr(U^Tpartial xV)$$

    By substitution into (I):

    $$frac{partial |x|_*}{partial x}= frac{ tr(partialSigma)}{partial x} =frac{ tr(U^Tpartial xV)}{partial x}=frac{ tr(VU^Tpartial x)}{partial x}=(VU^T)^T$$

    Therefore you can use $U V^T$ as the subgradient.

     参考:这里

  • 相关阅读:
    12 Source Code Profilers for C & C++
    HttpWebRequest的使用方法
    MSDN Windows 下载
    Qt 4.7 在VS2010环境下的编译
    [转].NET Logging Tools and Libraries
    硬盘崩溃之后
    .net core 下使用 logdashboard 日志面板
    工具收藏 年终工作总结必备工具之ppt利器
    Dapper 的应用和Dapper.Contrib 的方法封装(一)
    Dapper 的应用和Dapper.Contrib 的方法封装(二)
  • 原文地址:https://www.cnblogs.com/Eufisky/p/12820447.html
Copyright © 2011-2022 走看看