【转载请注明出处】http://www.cnblogs.com/mashiqi
(居然有朋友说内容不接地气，那么我就再加一段嘛，请喜欢读笑话的同学直接看第二段)假设这里有一组向量$left{ x_i ight}_{i=1}^n$和一个待投影的向量$u$。假设$u$和每个$x_i$的内积都为正数，也就是说$u$和每个$x_i$的夹角都小于90度。那么当我们把$u$投影到$left{ x_i ight}_{i=1}^n$上时，理所应当地每个$x_i$的系数$eta_i$也都应该大于零：$$u = x_1eta_1+cdots+x_neta_n,eta_igeq0$$不知道读者们的空间直觉怎么样，反正我最开始就是这么天真的认为的。最近看了Efron的“Least Angle Regression”后，才明白原来不是这样的，自己以前too young了。有些时候系数会变成负的。
(大家好我是第二段)举个栗子，设$$x_1=(0.1793,0.1216,0.9762)^T$$$$x_2=(-0.5947,0.5150,0.6173)^T$$$$x_3=(-0.2013,0.1782,-0.9632)^T$$并令$$u=(3.7952,6.9738,-0.5415)^T$$那么我们可以计算出$$(u,x_1)=(u,x_2)=(u,x_3)=1 > 0$$也就是说$x_1,x_2,x_3$与$u$的夹角都小于90度。但是通过解矩阵方程$Xeta=u$我们可以得到$$eta = (36.5949,-6.6537,33.3891)^T$$也就是说：$$u = 36.5949x_1 -6.6537x_2 + 33.3891x_3$$看呐！$x_2$君的系数居然是负的！$x_1$和$x_3$一样，$x_2$君也是同样思念着$u$君的，还思念的同样深(内积都是$1$ = =)。可是$x_2$君为了集体的利益甘愿牺牲自己成全整个$left{ x_i ight}_{i=1}^3$集合，这是多么伟大的奉献精神，此处应有掌声，啪啪啪，啪啪啪。。。
下面贴一个小MATLAB代码，自己去体会吧！
1 %{ 2 % This small matlab program show you a unexpected result in 3 % high-dimensional geometry: for any set of n-dimensional vectors 4 % {x_1,...,x_n}, if these vectors are indepentent, then you can always find 5 % an equiangular vector u, so that the inner product (x_i,u)=1 for all i. 6 % BUT, if we project u into {x_i}, some coefficients may be 'negative'! 7 %} 8 9 %% 10 clear; 11 close all; 12 %% You can change the following two variable's value 13 dimension = 3; 14 vectors = 3; 15 16 %% In case of unpredicted problems, dont change the following code. 17 if vectors > dimension 18 disp('Please set vectors <= dimension.'); 19 end 20 X = randn(dimension,vectors); % every column is a vector 21 X = X./repmat(sqrt(sum(X.^2,1)),dimension,1); % standardize 22 if rank(X) ~= size(X,2) 23 disp('These vectors are not independent. Run again.'); 24 end 25 w = (X'*X)ones(vectors,1); 26 u = X*w; 27 w % the coefficient 28 u % the equiangular vector 29 X'*u % the correlation value 30 if dimension == 3 % '3' is the upper bound of dimensions of humans where we can draw. 31 quiver3(0,0,0,X(1,1),X(2,1),X(3,1),'r'),hold on; 32 quiver3(0,0,0,X(1,2),X(2,2),X(3,2),'r'),hold on; 33 quiver3(0,0,0,X(1,3),X(2,3),X(3,3),'r'),hold on; 34 quiver3(0,0,0,u(1),u(2),u(3),'b'),hold off; 35 end
总结一句话：高维空间有危险，忽久留= =||