最小二乘中优化顺序交换
[XQc o XCq
]
其中(Xin R^{n,p},\,Qin R^{p,k},\, cin R^{k,1},\, Cin R^{p, kp},\, qin R^{kp, 1})，(n)表示样本数，(p)表示特征维度，(c)表示pattern的个数。
推导过程如下：
[egin{align*}(XQc)_{r, 1}&=sum_l(XQ)_{r,l}\,c_{l,1}\&=sum_l sum_m X_{r,m}Q_{m,l}\,c_{l,1}\&=sum_msum_l X_{r, m} \,c_{l,1}Q_{m,l}end{align*}
]
令(m)先固定，如(m=1)，(l={1, cdots ,k})上式为
[X_{r,1}c_{(1,cdots,k),1}Q_{1, (1,cdots,k)}
]
可以看到(c)对(Q)每一行都做了矩阵乘法，而(m)是变动的，不难可以想象出(Iotimes c.T)，(Q)要拉成已为向量。代码如下：
import numpy as np
np.random.seed(10)
X = np.random.randn(5, 10)
Q = np.random.randn(10, 3)
c = np.random.randn(3, 1)
# original
result = X.dot(Q.dot(c))
# transformed
I = np.eye(10)
q = Q.reshape(-1, 1)
#
C = np.outer(I, c.T).reshape(10, -1)
# C = np.multiply.outer(I, c.T).reshape(10, -1)
# C = np.einsum('ab, cd->abcd', I, c.T).reshape(10, -1)
new_result = X.dot(C.dot(q))
print("变换是否等价"， np.allclose(result, new_result, rtol=1e-4))
MMD
[Vert frac{1}{n_s}sumlimits_{i=1}^{n_s}A^Tx_i-frac{1}{n_t}sumlimits_{i=n_s}^{n_s+n_t}A^Tx_iVert_2^2
]
例子来源于[1]，具体的过程再赘述。
[egin{align*}Vert frac{1}{n_s}sumlimits_{i=1}^{n_s}A^Tx_i-frac{1}{n_t}sumlimits_{i=n_s}^{n_s+n_t}A^Tx_iVert_2^2&=Vert frac{1}{n_s}A^T X_s {mathbf 1}-frac{1}{n_t}A^T X_t{mathbf 1}Vert_2^2\&=trleft(A^Tleft(egin{bmatrix}X_s & X_tend{bmatrix}egin{bmatrix}frac{1}{n_s}^2 & -frac{1}{n_sn_t}\-frac{1}{n_sn_t}& frac{1}{n_t}^2end{bmatrix}egin{bmatrix}X_s \ X_tend{bmatrix}
ight)A
ight)\&=tr(A^TXMX^TA)end{align*}
]
Scatter Matrix
[AHA^T
]
Scatter matrix可以度量矩阵的方差[2]，(H=I-frac{1}{n}{mathbf 1}{mathbf 1}^T)方差的定义是
[egin{align*}var&=sum_{i=1}^n (x_i-m)(x_i-m)^T\&=sum_{i=1}^nx_ix_i^T-nar{x}ar{x}^Tend{align*}
]
其中(ar{x}=frac{1}{n}sum x_i)。令(X=egin{bmatrix} x_1& cdots&x_nend{bmatrix})，则上式可以表达为
[egin{align*}var &= XIX^T-X(frac{1}{n}{mathbf 1}{mathbf 1}^T)X^Tend{align*}
]