A.Kaw矩阵代数初步学习笔记 7. LU Decomposition

“矩阵代数初步”(Introduction to MATRIX ALGEBRA)课程由Prof. A.K.Kaw(University of South Florida)设计并讲授。
PDF格式学习笔记下载（Academia.edu）
第7章课程讲义下载（PDF）

Summary

• For a nonsingular matrix $[A]$ on which one can always write it as $$[A] = [L][U]$$ where $[L]$ is a lower triangular matrix, $[U]$ is a upper triangular matrix.
• Note that not all matrices have LU decomposition, such as $egin{bmatrix}0& 2\ 2& 0end{bmatrix}$. $$egin{bmatrix}0& 2\ 2& 0end{bmatrix}=egin{bmatrix}1& 0\ a& 1end{bmatrix} egin{bmatrix}b& c\ 0& dend{bmatrix} Rightarrow egin{cases} b=0\ ab=2end{cases}$$ which is contradiction.
• If one is solving a set of equations $$[A][X]=[B]$$ then $$LUX=B$$ $$Rightarrow L^{-1}LUX=L^{-1}B$$ $$Rightarrow UX=L^{-1}B=Y$$ then we have $$egin{cases}LY=B\ UX=Yend{cases}$$ So we can solve the first equation for $[Y]$by using forward substitution and then use the second equation to calculate the solution vector $[X]$ by back substitution.
• For instance, solve the following set of equations: $$egin{bmatrix}1& 2& 3\ 2& 1& -4\ 1& 5& 2end{bmatrix}cdot egin{bmatrix} x\ y\ z end{bmatrix} = egin{bmatrix} 14\ -8\ 17end{bmatrix}$$ Applying LU decomposition on the coefficient matrix,
  • Firstly write down an identity matrix (the same size as the coefficient matrix) on the left and the coefficient matrix on the right. $$Lleftarrowegin{bmatrix}1& 0& 0\ 0& 1& 0\ 0& 0& 1 end{bmatrix} egin{bmatrix}1& 2& 3\ 2& 1& -4\ 1& 5& 2end{bmatrix} ightarrow U$$
  • Then applying elementary row operation on the right while simultaneously updating successive columns of the matrix on the left. For example, if we are doing $R_1 + m R_2$ on the right then we will do $C_2-mC_1$ on the left. That is, we will keep the equivalent of the product. $$egin{bmatrix}1& 0& 0\ 0& 1& 0\ 0& 0& 1 end{bmatrix} egin{bmatrix}1& 2& 3\ 2& 1& -4\ 1& 5& 2end{bmatrix}$$ $$Rightarrowegin{cases}R_2-2R_1 \ C_1+2C_2end{cases} egin{bmatrix}1& 0& 0\ 2& 1& 0\ 0& 0& 1 end{bmatrix} egin{bmatrix}1& 2& 3\ 0& -3& -10\ 1& 5& 2end{bmatrix}$$ $$Rightarrowegin{cases}R_3-R_1 \ C_1+C_3end{cases} egin{bmatrix}1& 0& 0\ 2& 1& 0\ 1& 0& 1 end{bmatrix} egin{bmatrix}1& 2& 3\ 0& -3& -10\ 0& 3& -1end{bmatrix}$$ $$Rightarrowegin{cases}R_3+R_2 \ C_2-C_3end{cases} egin{bmatrix}1& 0& 0\ 2& 1& 0\ 1& -1& 1 end{bmatrix} egin{bmatrix}1& 2& 3\ 0& -3& -10\ 0& 0& -11end{bmatrix}$$ Thus far, the right matrix is an upper triangular matrix (i.e. $U$) and the left one is a lower triangular matrix (i.e. $L$).
  • Solving $[L][Y]=[B]$, that is $$egin{bmatrix}1& 0& 0\ 2& 1& 0\ 1& -1& 1 end{bmatrix}cdot Y=egin{bmatrix} 14\ -8\ 17end{bmatrix}Rightarrow Y=egin{bmatrix}14\ -36\ -33end{bmatrix}$$
  • Solving $[U][X]=[Y]$, that is $$egin{bmatrix}1& 2& 3\ 0& -3& -10\ 0& 0& -11end{bmatrix}cdot egin{bmatrix} x\ y\ z end{bmatrix} = egin{bmatrix}14\ -36\ -33end{bmatrix}$$ $$ Rightarrowegin{cases}x=1\ y=2 \ z=3end{cases}$$

Selected Problems

1. Find the $[L]$ and $[U]$ matrices of the following matrix $$egin{bmatrix}25& 5& 4\ 75& 7& 16\ 12.5& 12& 22 end{bmatrix}$$

Solution: $$egin{bmatrix}1& 0& 0\ 0& 1& 0\ 0& 0& 1 end{bmatrix}egin{bmatrix}25& 5& 4\ 75& 7& 16\ 12.5& 12& 22 end{bmatrix}$$ $$Rightarrow egin{cases}R_2-3R_1\ R_3-{1over2}R_1\ C_1+3C_2\ C_1+{1over2}C_3end{cases} egin{bmatrix}1& 0& 0\ 3& 1& 0\ {1over2}& 0& 1 end{bmatrix} egin{bmatrix}25& 5& 4\ 0& -8& 4\ 0& 9.5& 20 end{bmatrix}$$ $$Rightarrow egin{cases}R_3+{19over16}R_2\C_2-{19over16}C_3end{cases} egin{bmatrix}1& 0& 0\ 3& 1& 0\ {1over2}& -{19over16}& 1 end{bmatrix} egin{bmatrix}25& 5& 4\ 0& -8& 4\ 0& 0& {99over4} end{bmatrix}$$ That is, $$L= egin{bmatrix}1& 0& 0\ 3& 1& 0\ {1over2}& -{19over16}& 1 end{bmatrix}, U = egin{bmatrix}25& 5& 4\ 0& -8& 4\ 0& 0& {99over4} end{bmatrix}.$$

2. Using LU decomposition to solve: $$egin{cases} 4x_1 + x_2 - x_3 = -2\ 5x_1+x_2+2x_3=4\ 6x_1+x_2+x_3=6 end{cases}$$

Solution: $$egin{bmatrix}1& 0& 0\ 0& 1& 0\ 0& 0& 1 end{bmatrix} egin{bmatrix}4& 1& -1\ 5& 1& 2\ 6& 1& 1end{bmatrix}$$ $$Rightarrow egin{cases}R_2-{5over4}R_1\ R_3-{3over2}R_1\ C_1+{5over4}C_2\ C_1+{3over2}C_3end{cases} egin{bmatrix}1& 0& 0\ {5over4}& 1& 0\ {3over2}& 0& 1 end{bmatrix} egin{bmatrix}4& 1& -1\ 0& -{1over4}& {13over4}\ 0& -{1over2}& {5over2}end{bmatrix}$$ $$Rightarrow egin{cases}R_3-2R_2\ C_2+2C_3end{cases} egin{bmatrix}1& 0& 0\ {5over4}& 1& 0\ {3over2}& 2& 1 end{bmatrix} egin{bmatrix}4& 1& -1\ 0& -{1over4}& {13over4}\ 0&0& -4end{bmatrix}$$ That is, $$L = egin{bmatrix}1& 0& 0\ {5over4}& 1& 0\ {3over2}& 2& 1 end{bmatrix}, U= egin{bmatrix}4& 1& -1\ 0& -{1over4}& {13over4}\ 0&0& -4end{bmatrix}.$$ Then we solve $[L][Y]=[B]$, $$egin{bmatrix}1& 0& 0\ {5over4}& 1& 0\ {3over2}& 2& 1 end{bmatrix}cdot Y=egin{bmatrix}-2\4\6end{bmatrix} Rightarrow Y=egin{bmatrix}-2\{13over2}\ -4end{bmatrix}$$ Finally, we solve $[U][X]=[Y]$, $$egin{bmatrix}4& 1& -1\ 0& -{1over4}& {13over4}\ 0&0& -4end{bmatrix}cdot X= egin{bmatrix}-2\{13over2}\ -4end{bmatrix}Rightarrow X=egin{bmatrix}3\-13\1end{bmatrix}$$ Thus the solution is $$egin{cases}x_1 = 3\ x_2 = -13\ x_3 = 1end{cases}$$

3. Find the inverse of $$[A]=egin{bmatrix}3& 4& 1\ 2& -7& -1\ 8& 1& 5end{bmatrix}$$

Solution:

To find the inverse of a matrix, actually it is to solve a set of equations: $$egin{cases}AX_1=[1, 0, 0]^{T}\ AX_2 = [0, 1, 0]^{T}\ AX_3 = [0, 0, 1]^{T} end{cases}$$ Firstly, we will find the $[L]$ and $[U]$. $$egin{bmatrix}1& 0& 0\ 0& 1& 0\ 0& 0& 1 end{bmatrix} egin{bmatrix}3& 4& 1\ 2& -7& -1\ 8& 1& 5end{bmatrix}$$ $$Rightarrow egin{cases}R_2-{2over3}R_1\ R_3-{8over3}R_1\ C_1+{2over3}C_2\ C_1+{8over3}C_3end{cases} egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 0& 1 end{bmatrix} egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0& -{29over3}& {7over3}end{bmatrix}$$ $$Rightarrow egin{cases}R_3-R_2\ C_2+C_3end{cases} egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 1& 1 end{bmatrix} egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0&0& 4end{bmatrix}$$ That is, $$L = egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 1& 1 end{bmatrix}, U= egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0&0& 4end{bmatrix}.$$ Then we solve $[L][Y]=[I]$, note that there are three columns of $[Y]$: $$LY_1 = egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 1& 1 end{bmatrix} cdot Y_1 = egin{bmatrix}1\0\0end{bmatrix} Rightarrow Y_1=left[1, -{2over3}, -2 ight]^{T}$$ $$LY_2 = egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 1& 1 end{bmatrix} cdot Y_2 = egin{bmatrix}0\1\0end{bmatrix} Rightarrow Y_2=left[0, 1, -1 ight]^{T}$$ $$LY_3 = egin{bmatrix}1& 0& 0\ {2over3}& 1& 0\ {8over3}& 1& 1 end{bmatrix} cdot Y_3 = egin{bmatrix}0\0\1end{bmatrix} Rightarrow Y_3=left[0, 0, 1 ight]^{T}$$ Finally we can solve $[X]$ by $[U][X]=[Y]$: $$UX_1=Y_1Rightarrow egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0&0& 4end{bmatrix} cdot X_1 = egin{bmatrix}1\ -{2over3}\ -2end{bmatrix}Rightarrow X_1= left[{17over58}, {9over58}, -{1over2} ight]^{T}$$ $$UX_2=Y_2Rightarrow egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0&0& 4end{bmatrix} cdot X_2 = egin{bmatrix}0\ 1\ -1end{bmatrix}Rightarrow X_2= left[{19over116}, -{7over116}, -{1over4} ight]^{T}$$ $$UX_3=Y_3Rightarrow egin{bmatrix}3& 4& 1\ 0& -{29over3}& -{5over3}\ 0&0& 4end{bmatrix} cdot X_3 = egin{bmatrix}0\ 0\ 1end{bmatrix}Rightarrow X_3= left[-{3over116}, -{5over116}, {1over4} ight]^{T}$$ Thus the inverse of the original matrix is $$[A]^{-1} = egin{bmatrix}{17over58} & {19over116} & -{3over116}\ {9over58} & -{7over116} & -{5over116}\ -{1over2} & -{1over4} & {1over4}end{bmatrix}$$