“矩阵代数初步”(Introduction to MATRIX ALGEBRA)课程由Prof. A.K.Kaw(University of South Florida)设计并讲授。
PDF格式学习笔记下载（Academia.edu）
第9章课程讲义下载（PDF）
Summary
- Ill-conditional system
- A system of equations is considered to be ill-conditioned if a small change in the coefficient matrix or a small change in the right hand side results in a large change in the solution vector.
- For example, the following system $$egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7.999end{bmatrix}$$ The solution is $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix} ^{-1}cdotegin{bmatrix}4\ 7.999end{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Make a small change in the right hand side vector of the equations $$egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4.001\ 7.998end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix} ^{-1}cdotegin{bmatrix}4.001\ 7.998end{bmatrix} = egin{bmatrix}-3.999\ 4.000end{bmatrix}$$ Make a small change in the coefficient matrix of the equations $$egin{bmatrix} 1.001& 2.001\ 2.001& 3.998 end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7.999end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1.001& 2.001\ 2.001& 3.998end{bmatrix} ^{-1}cdotegin{bmatrix}4\ 7.999end{bmatrix} = egin{bmatrix} 6.989016\ -1.497254end{bmatrix}$$ We can see that a small change in the coefficient matrix or the right hand side resulted in a large change in the solution vector.
- Well-conditional system
- A system of equations is considered to be well-conditioned if a small change in the coefficient matrix of a small change in the right hand side results in a small change in the solution vector.
- For example, the following system $$egin{bmatrix} 1& 2\ 2& 3end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ The solution is $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1& 2\ 2& 3end{bmatrix} ^{-1}cdotegin{bmatrix}4\ 7end{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Make a small change in the right hand side vector of the equations $$egin{bmatrix} 1& 2\ 2& 3end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4.001\ 7.001end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1& 2\ 2& 3end{bmatrix} ^{-1} cdot egin{bmatrix} 4.001\ 7.001end{bmatrix} = egin{bmatrix}1.999\ 1.001end{bmatrix}$$ Make a small change in the coefficient matrix of the equations $$egin{bmatrix} 1.001& 2.001\ 2.001& 3.001 end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix} 1.001& 2.001\ 2.001& 3.001end{bmatrix} ^{-1}cdotegin{bmatrix}4\ 7end{bmatrix} = egin{bmatrix} 2.003\ 0.997end{bmatrix}$$ We can see that a small change in the coefficient matrix or the right hand side resulted in a small change in the solution vector.
- Norm
- Just like the determinant, the norm of a matrix is a simple unique scalar number. For a $m imes n$ matrix $[A]$, the row sum norm of $[A]$ is defined as $$|A|_{infty}=max_{1leq ileq m}sum_{j=1}^{n}|a_{ij}|$$ that is, find the sum of the absolute value of the elements pf each row of the matrix $[A]$. The maximum out of the $m$ such values is the row sum norm if the matrix $[A]$.
- For example, we have the following matrix $$[A] = egin{bmatrix}10& -3& 5\ -7& 2.099& -1\ 0& 6& 5end{bmatrix}$$ The row sum norm of $[A]$ is $$|A|_{infty} = max_{1leq ileq3} sum_{j=1}^{3}|a_{ij}|$$ $$=max[(10+7+0), (3+2.099+6), (5,-1,5)]$$ $$=max[17, 11.099, 11] =17$$
- The relationship between the norm and the conditioning of the matrix
- Example of the ill-conditioned system. $$egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7.999end{bmatrix}$$ which has the solution $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Denoting the above system as $AX=B$, and hence we have $$|X|_{infty}=2$$ $$|B|_{infty}=7.999$$ Making a small change in the right hand side $$egin{bmatrix} 1& 2\ 2& 3.999end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4.001\ 7.998end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}-3.999\ 4.000end{bmatrix}$$ Denoting the above changed system as $AX'=B'$ and $$Delta X=X'-X=egin{bmatrix}-3.999\ 4.000end{bmatrix} - egin{bmatrix}2\ 1end{bmatrix} = egin{bmatrix}-5.999\ 3.000end{bmatrix}$$ $$Delta B=B'-B = egin{bmatrix}4.001\ 7.998end{bmatrix} - egin{bmatrix}4\ 7.999end{bmatrix} = egin{bmatrix}0.001\ -0.001end{bmatrix}$$ Then $$|Delta X|_{infty} = 5.999$$ $$|Delta B|_{infty} = 0.001$$ The relative change in the norm of the solution vector is $${|Delta X|_{infty}over |X|_{infty}} = {5.999over2}=2.9995$$ The relative change in the norm of the right hand side vector is $${|Delta B|_{infty}over |B|_{infty}} = {0.001over7.999}=1.25 imes10^{-4}$$ That is, the small relative change of $1.25 imes10^{-4}$ in the right hand side vector norm results in a large relative change in the solution vector norm of $2.9995$. We can see the ratio of this two norms is $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta B|_{infty} ig/ | B|_{infty}} = 23993$$
- Example of the well-conditioned system. $$egin{bmatrix} 1& 2\ 2& 3end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ which has the solution $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Denoting the above system as $AX=B$, and hence we have $$|X|_{infty}=2$$ $$|B|_{infty}=7$$ Making a small change in the right hand side $$egin{bmatrix} 1& 2\ 2& 3end{bmatrix}egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4.001\ 7.001end{bmatrix}$$ gives $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}1.999\ 1.001end{bmatrix}$$ Denoting the above changed system as $AX'=B'$ and $$Delta X=X'-X=egin{bmatrix}1.999\ 1.001end{bmatrix} - egin{bmatrix}2\ 1end{bmatrix} = egin{bmatrix}-0.001\ 0.001end{bmatrix}$$ $$Delta B=B'-B = egin{bmatrix}4.001\ 7.001end{bmatrix} - egin{bmatrix}4\ 7end{bmatrix} = egin{bmatrix}0.001\ 0.001end{bmatrix}$$ Then $$|Delta X|_{infty} = 0.001$$ $$|Delta B|_{infty} = 0.001$$ The relative change in the norm of the solution vector is $${|Delta X|_{infty}over |X|_{infty}} = {0.001over2}=5 imes10{-4}$$ The relative change in the norm of the right hand side vector is $${|Delta B|_{infty}over |B|_{infty}} = {0.001over7} = 1.429 imes 10^{-4}$$ That is, the small relative change of $1.429 imes10^{-4}$ in the right hand side vector norm results in a small relative change in the solution vector norm of $5 imes10^{-4}$. We can see the ratio of this two norms is $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta B|_{infty} ig/ | B|_{infty}} = 3.5$$
- Properties of Norms
- $|A| geq 0$
- $|kA| = |k||A|$ where $k$ is a scalar.
- $|A+B|leq |A| + |B|$
- $|AB| leq |A|cdot|B|$
- For a system $AX=B$, we have $${|Delta X|over |X|} leq |A||A^{-1}|{|Delta B|over |B|}$$ and $${|Delta X|over |X + Delta X|} leq |A||A^{-1}|{|Delta A|over |A|}$$ where $|A||A^{-1}|$ is called the extbf{condition number}, Cond$(A)$.
- Significant Digits
- The possible relative error in the solution vector norm is no more then Cond$(A) imesepsilon$, where $epsilon$ is the machine epsilon which is $2.220446 imes10^{-16}$ or $2^{-52}$ here (obtained by R code .Machine$double.eps on 64-bit PC, more details refer to link1 and link2). Hence Cond$(A) imes epsilon$ should give us the number of significant digits, $m$ that are at least correct in our solution by finding out the largest value of $m$ for which Cond$(A) imesepsilon$ is less than or equal to $0.5 imes 10^{-m}$.
- How many significant digits can I trust in the solution of the following system of equations? $$egin{bmatrix}1& 2 \ 2& 3end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ For $$A=egin{bmatrix}1& 2 \ 2& 3end{bmatrix}$$ and $$A^{-1}= egin{bmatrix}-3& 2 \ 2& -1end{bmatrix}$$ Then $$|A|_{infty}=5, |A^{-1}|_{infty}=5Rightarrow ext{Cond}(A)=|A|_{infty}|A^{-1}|_{infty} = 25$$ Thus $$ ext{Cond}(A) imesepsilon leq 0.5 imes10^{-m}$$ $$Rightarrow 25 imesepsilonleq0.5 imes10^{-m}$$ $$Rightarrow log(25 imesepsilon) leq log(0.5 imes10^{-m})$$ $$Rightarrow mleq13.95459 $$ That is, 13 digits are at least correct in the solution vector.
Selected Problems
1. What factors does the adequacy of the solution of simultaneous linear equations depend on?
Solution:
The product of condition number Cond$(A)=|A||A^{-1}|$ and machine epsilon $epsilon$.
2. If a system of equations $[A][X]=[B]$ is ill-conditioned, then
A. $det(A)=0$
B. Cond$(A)=1$
C. Cond$(A)$ is large
D. $|A|$ is large
Solution:
If the system is ill-conditioned, then the condition number Cond$(A)=|A||A^{-1}|$ is large. The correct answer is C.
3. If Cond$(A)=10^{4}$ and $epsilon=0.119 imes10^{-6}$, then in $[A][X]=[B]$, at least how many significant digits are correct in the solution?
Solution: $$ ext{Cond}(A) imesepsilon leq 0.5 imes10^{-m}$$ $$Rightarrow 10^{4} imes0.119 imes10^{-6} leq 0.5 imes10^{-m}$$ $$Rightarrow mleq {log(0.5)-log(0.119 imes10^{-2})over log(10)} = 2.623423$$ Thus at least 2 significant digits are correct in the solution.
4. Make a small change in the coefficient matrix of $$egin{bmatrix}1& 2 \ 2& 3.999end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7.999end{bmatrix}$$ and find $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta A|_{infty} ig/ | A|_{infty}}$$
Solution:
The solution of the original system is $$ egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Making a small change in the coefficient matrix as $$egin{bmatrix} 1.001& 2.001 \ 2.001& 4.000end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7.999end{bmatrix}$$ and the solution is $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}5999\ -2999end{bmatrix}$$ Hence the row sum norms are $$|X| = 2, |Delta X|=5997, |A|=5.999, |Delta A|=0.002$$ Thus the ratio is $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta A|_{infty} ig/ | A|_{infty}} = {5997 ig/ 2over 0.002 ig/ 5.999} = 8994001$$ It is a large number. Hence we can conclude that this system is ill-conditioned. On the other hand, we can calculate the condition number of the coefficient matrix, note that $A^{-1} = egin{bmatrix}-3999& 2000 \ 2000& -1000 end{bmatrix}$, and hence $$|A||A^{-1}|= 5.999 imes5999=35988 $$ which is also a large number.
5. Make a small change in the coefficient matrix of $$egin{bmatrix}1& 2 \ 2& 3end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ and find $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta A|_{infty} ig/ | A|_{infty}}$$
Solution:
The solution of the original system is $$ egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}2\ 1end{bmatrix}$$ Making a small change in the coefficient matrix as $$egin{bmatrix} 1.001& 2.001 \ 2.001& 3.001end{bmatrix} egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}4\ 7end{bmatrix}$$ and the solution is $$egin{bmatrix}x\ yend{bmatrix} = egin{bmatrix}2.003\ 0.997end{bmatrix}$$ Hence the row sum norms are $$|X| = 2, |Delta X|=0.003, |A|=5, |Delta A|=0.002$$ Thus the ratio is $${|Delta X|_{infty} ig/ |X|_{infty}over |Delta A|_{infty} ig/ | A|_{infty}} = {0.003 ig/ 2over 0.002 ig/ 5} = 3.75$$ It is a small number. Hence we can conclude that this system is well-conditioned. On the other hand, we can calculate the condition number of the coefficient matrix, note that $A^{-1} = egin{bmatrix}-3& 2 \ 2& -1end{bmatrix}$, and hence $$|A||A^{-1}|= 5 imes5=25$$ which is also a small number.
6. Prove $${|Delta X|over |X|} leq |A||A^{-1}|{|Delta B|over |B|}$$
Solution:
The key point is $|XY| leq |X||Y|$. Let $AX=B$, then if $B$ is changed to $B'$, the $X$ is changed to $X'$, such that $$AX'=B'$$ Hence we have $$AX=B, AX'=B'$$ $$Rightarrow Delta X=X'-X=A^ {-1}B'-A^{-1}B = A^{-1}Delta B$$ $$Rightarrow|Delta X|leq |A^{-1}||Delta B|$$ and $$AX=BRightarrow |B|=|AX| leq |A||X|$$ Multiply the above inequalities and obtain $$|Delta X||B| leq |A^{-1}||Delta B||A||X|$$ $$Rightarrow {|Delta X|over |X|} leq |A||A^{-1}|{|Delta B|over |B|}$$
7. Prove $${|Delta X|over |X + Delta X|} leq |A||A^{-1}|{|Delta A|over |A|}$$
Solution:
Similar to the previous question, we have $$AX=B, A'X'=B$$ $$Rightarrow AX=A'X'=(A+Delta A)(X+Delta X)=AX+ADelta X+Delta AX + Delta ADelta X$$ $$Rightarrow ADelta X+Delta AX + Delta ADelta X = [0]$$ $$Rightarrow Delta A(X+Delta X)=-ADelta X $$ $$Rightarrow Delta X= -A^{-1}Delta A(X+Delta X) leq |A^{-1}||Delta A||X+Delta X|$$ $$Rightarrow |A|Delta Xleq |A||A^{-1}||Delta A||X+Delta X|$$ $$Rightarrow {|Delta X|over |X + Delta X|} leq |A||A^{-1}|{|Delta A|over |A|}$$
8. Prove that Cond$(A) geq 1$.
Solution: $$ ext{Cond}(A) = |A||A^{-1}| geq |AA^{-1}| = |I|=1$$
9. For $$[A] = egin{bmatrix}10& -7& 0\ -3& 2.099& 6\ 5& -1& 5end{bmatrix}$$ gives $$[A]^{-1} = egin{bmatrix}-0.1099& -0.2333& 0.2799\ -0.2999& -0.3332& 0.3999\ 0.04995& 0.1666& 6.664 imes10^{-5}end{bmatrix}$$ (A) What is the condition number of $[A]$?
(B) How many significant digits can we at least trust in the solution of $[A][X] = B$ if $epsilon = 0.1192 imes10^{-6}$?
Solution:
(A) Cond$(A) = |A||A^{-1}| = 17 imes1.033 = 17.561$
(B) $$ ext{Cond}(A) imesepsilon leq 0.5 imes10^{-m}$$ $$Rightarrow 17.561 imes0.1192 imes10^{-6} leq 0.5 imes10^{-m}$$ $$Rightarrow m leq 5.378145$$ Hence 5 significant digits can be trusted in the solution.
10. Let $$[A] = egin{bmatrix}1& 2+delta\ 2-delta& 1end{bmatrix}$$ Based on the row sum norm and given that $delta ightarrow0$, $delta > 0$, what is the condition number of the matrix?
Solution:
Recall that the inverse of the matrix $[M]=egin{bmatrix}a &b\ c &d end{bmatrix}$ is $$egin{bmatrix}{doverdet(M)} & {-bover det(M)}\ {-coverdet(M)} &{aoverdet(M)} end{bmatrix}$$ where $det(M) = ad-bc$. Thus we have $$A^{-1} = egin{bmatrix}{1over -3+delta^2}& -{2+deltaover -3+delta^2}\{-2+deltaover -3+delta^2}& {1over -3+delta^2}end{bmatrix}$$ The row sum norms are $$|A| = max(3+delta, 3-delta) = 3+delta$$ and $$|A^{-1}| = maxleft({3+deltaover 3-delta^2}, {3-delta over 3-delta^2} ight) = {3+deltaover 3-delta^2}$$ Hence $$ ext{Cond}(A) = |A||A^{-1}| = {(3+delta)^2over3-delta^2}$$