#Stereo Rectification
##[This is an original blog, not citation]
[Visit My GitHub](http://www.github.com/hiroki-kyoto/)
For stereo matching, rectification is required, however, it will be confucious to complete this task using opencv.
Calibration and rectification are implemented in opencv by a wierd manner: the projection matrix of $3 imes4$ elements are not used fully, only $3 imes3$ of the top-left submatrix is used to rectify the views. While, we can give it an explanation but the author does not give even a hint in the code, not to mention an explanation in the documents. Actually, only rotation matrix can be used in rectification, despite the fact that the translation vector exists between two views.
A translation vector contains offset in axis $Z$, which is strongly related to the exact depth of point in real world. This is going beyond our control with only pixels on hand. We do not know the depth for each pixel with certainty, therefore, the projection matrix should not include any non-zero element on $Z$ axis. Opencv adopted an approach to avoiding using Translation vector in its absolute scale, converting a single translation transform into two rotation matrices, one for each view.
This is how it is done:
- Retrieve (R) and (vec{t}), satisfying (P_2equiv Rcdot P_1+vec{t});
- Set the first rotation matrix for both view (R_1=R^{-frac{1}{2}}), (R_2=R^{frac{1}{2}}) to put two cameras parallel but not in the same plane;
- Set the second rotation matrix for both view (R_3=R_ycdot R_z) to put tow cameras co-plannar, in which, (R_y) and (R_z) are respectively the corresponding rotation matrix along axis (Y) and (Z) to rotate the transformed vector of (vec{t'}=R_1cdotvec{t}) to new axis (X) of left camera, the angles are easy to solve as: (cos heta_y = vec{t'}cdotvec{e_y}) and (cos heta_z = vec{t'}cdotvec{e_z});
- Apply (R_1cdot R_3) to left camera, apply (R_2cdot R_3) to the right.
##
3-Dimension Rotation represented in exponential manner
$$
R_i=e^{[ heta_i]_×},[ heta_i]_×=egin{bmatrix}
0&- heta_{i3}& heta_{i2}\ heta_{i3}&0&- heta_{i1}\- heta_{i2}& heta_{i1}&0\end{bmatrix}
ag 1
$$