zoukankan      html  css  js  c++  java
  • 核范数求次梯度

    Start with the SVD decomposition of $x$:

    $$x=USigma V^T$$

    Then $$|x|_*=tr(sqrt{x^Tx})=tr(sqrt{(USigma V^T)^T(USigma V^T)})$$

    $$Rightarrow |x|_*=tr(sqrt{VSigma U^T USigma V^T})=tr(sqrt{VSigma^2V^T})$$

    By circularity of trace:

    $$Rightarrow |x|_*=tr(sqrt{V^TVSigma^2})=tr(sqrt{V^TVSigma^2})=tr(sqrt{Sigma^2})=tr(Sigma)$$

    Since the elements of $Sigma$ are non-negative.

    Therefore nuclear norm can be also defined as the sum of the absolute values of the singular value decomposition of the input matrix.

    Now, note that the absolute value function is not differentiable on every point in its domain, but you can find a subgradient.


    $$frac{partial |x|_*}{partial x}=frac{partial tr(Sigma)}{partial x}=frac{ tr(partialSigma)}{partial x}$$

    You should find $partialSigma$. Since $Sigma$ is diagonal, the subdifferential set of $Sigma$ is: $partialSigma=SigmaSigma^{-1}partialSigma$, now we have:

    $$frac{partial |x|_*}{partial x}=frac{ tr(SigmaSigma^{-1}partialSigma)}{partial x}$$ (I)

    So we should find $partialSigma$.

    $x=USigma V^T$, therefore:
    $$partial x=partial USigma V^T+UpartialSigma V^T+USigmapartial V^T$$

    Therefore:

    $$UpartialSigma V^T=partial x-partial USigma V^T-USigmapartial V^T$$

    $$Rightarrow U^TUpartialSigma V^TV=U^Tpartial xV-U^Tpartial USigma V^TV-U^TUSigmapartial V^TV$$


    $$Rightarrow partialSigma =U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV$$

    egin{align}
    Rightarrow\
    tr(partialSigma) &=& tr(U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV)\
    &=& tr(U^Tpartial xV)+tr(-U^Tpartial USigma - Sigmapartial V^TV)
    end{align}


    You can show that $tr(-U^Tpartial USigma - Sigmapartial V^TV)=0$ (Hint: diagonal and antisymmetric matrices, proof in the comments.), therefore:

    $$tr(partialSigma) = tr(U^Tpartial xV)$$

    By substitution into (I):

    $$frac{partial |x|_*}{partial x}= frac{ tr(partialSigma)}{partial x} =frac{ tr(U^Tpartial xV)}{partial x}=frac{ tr(VU^Tpartial x)}{partial x}=(VU^T)^T$$

    Therefore you can use $U V^T$ as the subgradient.

     参考:这里

  • 相关阅读:
    字符串中包含最多的字符
    循环左移操作 (左旋转字符串)
    翻转字符串
    和为s的连续正数序列
    Unity之使用技巧记录
    Unity资源
    各种IDE的使用
    Unity3d之MonoBehavior自带方法的执行顺序
    c#之new关键词——隐藏基类方法
    算法——各种类型对象通用的二分法插入排序
  • 原文地址:https://www.cnblogs.com/Eufisky/p/12820447.html
Copyright © 2011-2022 走看看