zoukankan      html  css  js  c++  java
  • 核范数求次梯度

    Start with the SVD decomposition of $x$:

    $$x=USigma V^T$$

    Then $$|x|_*=tr(sqrt{x^Tx})=tr(sqrt{(USigma V^T)^T(USigma V^T)})$$

    $$Rightarrow |x|_*=tr(sqrt{VSigma U^T USigma V^T})=tr(sqrt{VSigma^2V^T})$$

    By circularity of trace:

    $$Rightarrow |x|_*=tr(sqrt{V^TVSigma^2})=tr(sqrt{V^TVSigma^2})=tr(sqrt{Sigma^2})=tr(Sigma)$$

    Since the elements of $Sigma$ are non-negative.

    Therefore nuclear norm can be also defined as the sum of the absolute values of the singular value decomposition of the input matrix.

    Now, note that the absolute value function is not differentiable on every point in its domain, but you can find a subgradient.


    $$frac{partial |x|_*}{partial x}=frac{partial tr(Sigma)}{partial x}=frac{ tr(partialSigma)}{partial x}$$

    You should find $partialSigma$. Since $Sigma$ is diagonal, the subdifferential set of $Sigma$ is: $partialSigma=SigmaSigma^{-1}partialSigma$, now we have:

    $$frac{partial |x|_*}{partial x}=frac{ tr(SigmaSigma^{-1}partialSigma)}{partial x}$$ (I)

    So we should find $partialSigma$.

    $x=USigma V^T$, therefore:
    $$partial x=partial USigma V^T+UpartialSigma V^T+USigmapartial V^T$$

    Therefore:

    $$UpartialSigma V^T=partial x-partial USigma V^T-USigmapartial V^T$$

    $$Rightarrow U^TUpartialSigma V^TV=U^Tpartial xV-U^Tpartial USigma V^TV-U^TUSigmapartial V^TV$$


    $$Rightarrow partialSigma =U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV$$

    egin{align}
    Rightarrow\
    tr(partialSigma) &=& tr(U^Tpartial xV-U^Tpartial USigma - Sigmapartial V^TV)\
    &=& tr(U^Tpartial xV)+tr(-U^Tpartial USigma - Sigmapartial V^TV)
    end{align}


    You can show that $tr(-U^Tpartial USigma - Sigmapartial V^TV)=0$ (Hint: diagonal and antisymmetric matrices, proof in the comments.), therefore:

    $$tr(partialSigma) = tr(U^Tpartial xV)$$

    By substitution into (I):

    $$frac{partial |x|_*}{partial x}= frac{ tr(partialSigma)}{partial x} =frac{ tr(U^Tpartial xV)}{partial x}=frac{ tr(VU^Tpartial x)}{partial x}=(VU^T)^T$$

    Therefore you can use $U V^T$ as the subgradient.

     参考:这里

  • 相关阅读:
    封装/继承
    模板
    常用模块-re模块1
    包常用模块
    模块和软件开发的目录规范
    Hadoop 综合大作业
    hive基本操作与应用
    用mapreduce 处理气象数据集
    熟悉常用的HBase操作,编写MapReduce作业
    爬虫大作业
  • 原文地址:https://www.cnblogs.com/Eufisky/p/12820447.html
Copyright © 2011-2022 走看看