\documentclass[11pt]{article}
\usepackage{amsfonts}
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{latexsym}
\usepackage{verbatim}
\usepackage[pdftex]{graphicx}
\usepackage{url}
\usepackage{cancel}
\usepackage[table]{xcolor}
\usepackage[all,arc]{xy}
\usepackage{color, colortbl}
\usepackage{mathrsfs}
\usepackage{hyperref}
\hypersetup{colorlinks=true}
\setlength{\oddsidemargin}{.25in}
\setlength{\evensidemargin}{.25in}
\setlength{\textwidth}{6in}
\setlength{\topmargin}{-0.4in}
\setlength{\textheight}{8.5in}

\input{preamble.tex}

\newcommand\overmat[2]{%
  \makebox[0pt][l]{$\smash{\color{white}\overbrace{\phantom{%
    \begin{matrix}#2\end{matrix}}}^{\text{\color{black}#1}}}$}#2}
\newcommand\bovermat[2]{%
  \makebox[0pt][l]{$\smash{\overbrace{\phantom{%
    \begin{matrix}#2\end{matrix}\hspace{15pt}}}^{\text{#1}}}$}#2}
\newcommand\partialphantom{\vphantom{\frac{\partial e_{P,M}}{\partial w_{1,1}}}}

\begin{document}

\lecture{5: The Schur Decomposition}{Week 5}


Repeatedly through the past three weeks, we have taken some matrix $A$ and written $A$ in the form
\begin{align*}
A = UBU^{-1},
\end{align*}
where $B$ was a diagonal matrix, and $U$ was a change-of-basis matrix.  

However, on HW $\#2$, we saw that this was not always possible: in particular, you proved in problem 4 that for the matrix $A = \begin{bmatrix} 1 & 1\\ 0 & 1\end{bmatrix}$, there was no possible basis under which $A$ would become a diagonal matrix: i.e. you proved that there was no diagonal matrix $D$ and basis $B = \{(b_{11}, b_{21}), (b_{12}, b_{22})\}$ such that
\begin{align*}
A = \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix} \cdot D \cdot  \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix}^{-1}.
\end{align*}

This is a bit of a shame, because diagonal matrices (for reasons discussed earlier) are pretty fantastic: they're easy to raise to large powers and calculate determinants of, and it would have been nice if every linear transformation was diagonal in some basis.  So: what now?  Do we simply assume that some matrices cannot be written in a ``nice'' form in any basis, and that we should assume that operations like matrix exponentiation and finding determinants is going to just be awful in many situations?

The answer, as you may have guessed by the fact that these notes have more pages after this one, is no!  In particular, while diagonalization\footnote{We say that a linear transformation is \textbf{diagonalizable} if it can be written as a diagonal matrix in some basis.} might not always be possible, there is something fairly close that is - the \textbf{Schur decomposition}.

Our goal for this week is to prove this, and study its applications.  To do this, we need one quick deus ex machina:
\begin{thm}
Suppose that $V$ is a $n$-dimensional vector space over $\mathbb{C}$, and $T$ is a linear transformation from $V \to V$.  Then $T$ has a complex-valued eigenvalue with corresponding nontrivial eigenvector: i.e.\ there is some vector $\vec{v} \neq \vec{0} \in V$ such that $T(\vec{v}) = \lambda \vec{v}$.
\end{thm}
If we find a basis $B$ for $V$, and write $T$ as a matrix over the basis $B$ and $\vec{v}$ as a vector in the base $B$, this is equivalent to the following theorem:
\begin{thm}
Suppose that $T$ is a complex-valued $n \times n$ matrix.  Then $T$ has a complex-valued eigenvalue with corresponding nontrivial eigenvector: i.e.\ there is some vector $\vec{v} \neq \vec{0} \in \mathbb{C}^n$ such that $T(\vec{v}) = \lambda \vec{v}$.
\end{thm}
We aren't going to prove this theorem in this course, because it essentially boils down to a very large theorem from complex analysis that is very far outside of the aim of this course: the \textbf{fundamental theorem of algebra}\footnote{\begin{thm}
(The Fundamental Theorem of Algebra)  If $f(x)$ is a nonconstant polynomial with coefficients from some field $F$, with $F = \mathbb{R}$ or $\mathbb{C}$, then it has a root in $\mathbb{C}$.  In other words, every nonconstant polynomial $f(x)$ has some corresponding value $r \in \mathbb{C}$ such that $f(r) = 0$.
\end{thm}}
  Talk to me if you want to know how this works!  

Instead, we're going to study applications of this result:
\section{The Schur Decomposition}

The Schur decomposition is the following result:
\begin{thm}
(Schur decomposition): For any $n \times n$ matrix $A$ with entries from $\mathbb{C}$, there is some orthonormal basis $B$ of $\mathbb{C}$ and some upper-triangular\footnote{A matrix is called \textbf{upper-triangular} if all of its entries below the main diagonal are 0.  For example, $\begin{bmatrix} 1 & 2 & 3\\ 0 & 3 & 2\\ 0 & 0 &1 \end{bmatrix} $ is upper-triangular.} matrix $R$ with entries in $\mathbb{C}$.
\begin{align*}
A =  \begin{bmatrix} \vdots & & \vdots \\ \vec{b_1} & \ldots & \vec{b_n} \\ \vdots & & \vdots \end{bmatrix} \cdot R \cdot  \begin{bmatrix} \vdots & & \vdots \\ \vec{b_1} & \ldots & \vec{b_n} \\ \vdots & & \vdots \end{bmatrix}^{-1}_.
\end{align*}

In other words, for any $n \times n$ complex-valued matrix  there is an orthonormal basis in which that matrix is upper-triangular!

\end{thm}

We prove this theorem here, provide an example of such a decomposition, and finally use this decomposition to calculate something that would otherwise be fairly difficult!  

First, the proof:
\begin{proof}

We proceed in four stages.
\begin{enumerate}
\item First, find an eigenvalue $\lambda_1$ of $A$.  We are guaranteed that some such $\lambda_1$ exists, by our earlier result.
\item Now, let $E_\lambda$ denote the set of all vectors $\vec{v}$ such that $A\vec{v} = \lambda \vec{v}$.  This is a subspace of $\mathbb{C}^n$, as linear combinations of vectors in this space are still in this space.  Therefore, it has an orthonormal basis!  Pick some orthonormal basis $\vec{b_{1\lambda_1}}, \ldots \vec{b_{k\lambda_1}}$ for this space.
\item Now, extend this basis to an orthonormal basis for all of $\mathbb{C}^n$!  This is easy to do: one by one, pick a vector not in the span of our basis, run Gram-Schmidt on that vector to make it orthogonal to everything in our basis, and add in this new orthogonal vector $\vec{c_i}$ to our basis.  Do this until we have $n$ vectors in our basis, at which point we have an orthonormal basis for $\mathbb{C}^n$.
\item Now, write our matrix $A$ in the orthonormal basis $\{\vec{b_{1\lambda_1}}, \ldots \vec{b_{k_1\lambda_1}}, \vec{c_1}, \ldots \vec{c_{n-k_1}}\}$.  What does this look like?  Well: we know that for each $\vec{b_{i\lambda_1}},$ by definition, we have $A\vec{b_{i\lambda_1}} = \lambda_1\vec{b_{i\lambda_1}},$ which in our orthonormal basis is the vector with $i$-th entry $\lambda_1$ and the rest 0.   In other words, we know the first $k$ columns of our matrix!  In particular, our matrix has the following form:\\
\begin{align*}A_B = 
\left[\begin{array}{ccc|ccc}
\bovermat{$k_1$ \textrm{ entries}}{\lambda_1 & &} & \bovermat{n-$k_1$ \textrm{ entries}}{& &}\\
 & \ddots &  & &A_{\textrm{rem}}&\\
& & \lambda_1 & & &\\\hline
& & & & & \\
&0& & &A_2& \\
& & & & & \\
\end{array}\right]\begin{aligned}
 & \left.\begin{matrix}
  \\
\\
\\
  \end{matrix} \right\} %
  k_1 \textrm{ entries}\\
&\left.\begin{matrix}
  \\
\\
\\
  \end{matrix} \right\} %
  n-k_1 \textrm{ entries}\\
 \end{aligned}
\end{align*}
To be specific: the above matrix consists of four distinct ``blocks:'' 
\begin{enumerate}
\item a $k_1 \times k_1$ diagonal matrix in the upper-left, with $\lambda$ on its diagonal and 0's elsewhere, 
\item a $n-k_1 \times k_1$ matrix in the lower-left made entirely of 0's, 
\item a $k_1 \times n-k_1$ matrix in the uppper-right corner, which we name $A_\textrm{rem}$, and
\item a $n-k_1 \times n-k_1$ matrix in the lower-right corner, which we name $A_2$.
\end{enumerate}
\item Consider the matrix $A_2$.  This is, by definition, a linear map that takes in vectors from the span of the set $\{\vec{c_1}, \ldots \vec{c_{n-k_1}}\}$ and outputs vectors within that same span!  (Think about why that is for a second.   Persuade yourself that it is true.)  In particular, because $A_2$ is now a square matrix over some vector space, we can take $A_2$ and put it back into steps 1-4 above!  We can do this whenever this matrix $A_2$ exists, which is whenever $n-k_1$ is nonzero.
\item What do we have now?  Well: we have an orthonormal basis 
\begin{align*}
\{\vec{b_{1\lambda_1}}, \ldots \vec{b_{k_1\lambda_1}}, \vec{b_{1\lambda_2}}, \ldots \vec{b_{k_2\lambda_2}}, \vec{c_1'}, \ldots \vec{c_{n-k_1-k_2}}\},
\end{align*}
and have also shown that the matrix $A$ written under the above basis has the form\\
\begin{align*}A_B = 
\left[\begin{array}{ccc|ccc|ccc}
\bovermat{$k_1$ \textrm{ entries}}{\lambda_1 & &} & \bovermat{$k_2$ \textrm{ entries}}{& & } &\bovermat{n-$k_1 - k_2$ \textrm{ entries}}{& & }~~\\
 & \ddots &  & &A_{\textrm{rem}_1}& & &A_{\textrm{rem}_2}&\\
& & \lambda_1 & & &\\\hline
& & & \lambda_2& & & & & \\
&0& & &\ddots& & &A_{\textrm{rem}_3}&\\
& & & & &\lambda_2& & & \\\hline
& & & & & & & & \\
&0& & &0& & &A_3&\\
& & & & && & & \\
\end{array}\right]\begin{aligned}
 & \left.\begin{matrix}
  \\
\\
\\
  \end{matrix} \right\} %
  k_1 \textrm{ entries}\\
& \left.\begin{matrix}
  \\
\\
\\
  \end{matrix} \right\} %
  k_2 \textrm{ entries}\\
&\left.\begin{matrix}
  \\
\\
\\
  \end{matrix} \right\} %
  n-k_1-k_2 \textrm{ entries}\\
 \end{aligned}
\end{align*}
\item Now, do it again to $A_3$!  In fact, keep repeating this process until we cannot continue. What does our matrix under our resuting look like then?  Well: we've filled its diagonal with these diagonal $\begin{bmatrix} \lambda_i && \\ & \ddots & \\ &&\lambda_i\end{bmatrix}$ matrices, which have only 0's below them.  In other words, our matrix is upper-triangular!
\end{enumerate}
So we're done!
\end{proof}

\section{The Schur Decomposition: Why We Care}

The above decomposition is incredibly useful in certain situations, like (as we often study) raising a matrix to some large power!  We study an example here:
\begin{example}
Consider the following matrix:
\begin{align*}
A = \begin{bmatrix}
13 & 8 & 8\\
-1 & 7 & -2\\
-1 & -2 & 7\\
\end{bmatrix}
\end{align*}

What is $A^{50}$?  
\end{example}
\begin{proof}
We could calculate this directly, given enough computing time and power.  However, that seems...bad.  How can we do this faster?  Well: we might hope that we could do the trick that we've used before, and write $A$ in the form $UDU^{-1}$, where $D$ is a diagonal matrix!  

That doesn't work.  In fact, there is no pair of matrices $U$, $D$, such that $U$ is invertible and $D$ is diagonal, and $A = UDU^{-1}$.  (In this sense, $A$ is like $\begin{bmatrix} 1 & 1\\ 0 & 1\end{bmatrix}$, which we studied on HW$\#2$.

However, we can do something almost as good: find A's Schur decomposition!  We first find this decomposition, and then talk about why it is useful for finding $A^{50}$.

First, we try to find an eigenvalue with corresponing nontrivial eigenvector for $A$.  We do this, as always, by brute-force: if we have a vector $(x,y,z)$ and constant $\lambda$ such that
\begin{align*}
A = \begin{bmatrix}
13 & 8 & 8\\
-1 & 7 & -2\\
-1 & -2 & 7\\
\end{bmatrix} \cdot \begin{bmatrix} x\\y\\z\end{bmatrix} = \begin{bmatrix}13x + 8y + 8z\\-x+7y -2z\\-x-2y+7z\end{bmatrix} = \lambda \begin{bmatrix} x\\y\\z\end{bmatrix}
\end{align*}
This gives us three linear equations, which we can solve:
\begin{align*}
13x + 8y + 8z&= \lambda x,\\
-x+7y -2z&= \lambda y,\\
-x-2y+7z &= \lambda z.
\end{align*}
In particular, adding four copies of the second to the first gives us
\begin{align*}
9x + 36y = \lambda x + 4\lambda y \Leftrightarrow (9-\lambda)x = (4\lambda - 36)y.
\end{align*}
Similarly, adding four copies of the third to the first gives us
\begin{align*}
9x + 36z = \lambda x + 4\lambda z \Leftrightarrow (9-\lambda)x = (4\lambda - 36)z.
\end{align*}
Finally, subtracting the third from the second gives
\begin{align*}
9y - 9z = \lambda y - \lambda z \Leftrightarrow (9-\lambda)y = (9-\lambda)z.
\end{align*}
So: um, these three equations seem to be basically shouting ``try setting $\lambda = 9$,'' because that satisfies \textbf{all} of our equations!  If we do this, then all three of our equations simplify to the same constraint, which is
\begin{align*}
x + 2y + 2z = 0.
\end{align*}
There are lots of solutions to this equation!  One that comes to mind is $(2,-2,1)$.  Checking shows that this is indeed an eigenvector for the eigenvalue 9:
\begin{align*}
A = \begin{bmatrix}
13 & 8 & 8\\
-1 & 7 & -2\\
-1 & -2 & 7\\
\end{bmatrix} \cdot \begin{bmatrix} 2\\-2\\1\end{bmatrix} = \begin{bmatrix}18\\-18\\9\end{bmatrix}
\end{align*}

So: we have an eigenvalue!  To find the Schur decomposition of $A$, we now need to find \textbf{all} of the eigenvectors corresponding to the eigenvalue 9.  As shown above, this is just the space
\begin{align*}
E_9 = \{(x,y,z) ~|~ x + 2y + 2z = 0\}.
\end{align*}
We want an orthonormal basis for this space.  To do so, we first find a basis, and then use Gram-Schmidt.  The basis is easy enough: we know that $(2,-2,1)$ is in our space, and we can also immediately see that $(0,1,-1)$ is also in our space, as it satisfies the constraint $x + 2y + 2z = 0$.  If we perform Gram-Schmidt on these two vectors, setting $\vec{u_1} = (2,-2,1),$ and
\begin{align*}
\vec{u_2}& = (0,1,-1) - \textrm{proj}((0,1,-1) \textrm{ onto }(2,-2,1))\\
& = (0,1,-1) - \frac{(0,1,-1) \cdot (2,-2,1)}{(2,-2,1)\cdot(2,-2,1)}(2,-2,1)\\
&= (0,1,-1) + \frac{3}{9}(2,-2,1)\\
&= \left(\frac23, \frac13, -\frac23\right).
\end{align*}
Finally, we scale these vectors by their length to get a basis for $E_9: \left\{ \left(\frac23, -\frac23, \frac13\right) , \left(\frac23, \frac13, -\frac23\right)\right\}$.  We now need to extend this to a basis for all of $\mathbb{C}^3$: to do this, we simply take some vector not in the span of these two vectors, like $(0,0,1)$ (check for yourself: why is this not in their span?), and perform Gram-Schmidt on this third vector:
\begin{align*}
\vec{u_3} &= (0,0,1) -  \textrm{proj}\left((0,0,1) \textrm{ onto }\left(\frac23, -\frac23, \frac13\right)\right) -  \textrm{proj}\left((0,0,1) \textrm{ onto }\left(\frac23, \frac13, -\frac23\right)\right) \\
&= (0,0,1) - \frac{1/3}{1}\left(\frac23, -\frac23, \frac13\right) + \frac{2/3}{1} \left(\frac23, \frac13, -\frac23\right)\\
&=\left(\frac29, \frac49, \frac49\right).
\end{align*}
Scaling this vector by its length, which is $\sqrt{36/81} = \frac23$, gives us $\left( \frac13, \frac23, \frac23\right)$.   So we have an orthonormal basis 
\begin{align*}
B = \left\{ \vec{b_{1,\lambda}} =  \left(\frac23, -\frac23, \frac13\right) , \vec{b_{2,\lambda}}\left(\frac23, \frac13, -\frac23\right), \vec{c} = \left( \frac13, \frac23, \frac23\right)\right\}
\end{align*}

Now we need to write our matrix $A$ in the basis $B$.  To do this, simply note that because $\vec{b_{1,\lambda}} , \vec{b_{2,\lambda}}$ are both eigenvectors for the eigenvalue $\lambda = 9$, we know exactly where $A$ sends these vectors: specifically, we have $A\vec{b_{1,\lambda}} = 9\vec{b_{1,\lambda}} = (9,0,0)_B$, and $A\vec{b_{2,\lambda}} = 9\vec{b_{2,\lambda}} = (0,9,0)_B$.

Finally, we can just calculate what $A$ does to $\vec{c}$:
\begin{align*}
\begin{bmatrix}
13 & 8 & 8\\
-1 & 7 & -2\\
-1 & -2 & 7\\
\end{bmatrix} \cdot \begin{bmatrix}1/3\\2/3\\2/3\end{bmatrix} = \begin{bmatrix}15\\3\\3\end{bmatrix} = 9(\vec{b_{1,\lambda}} + \vec{b_{2,\lambda}} + \vec{c}) = (9,9,9)_B.
\end{align*}

Therefore, our matrix $A$ under the basis $B$ is just the matrix with these three vectors as its columns: i.e.
\begin{align*}
A_B = \begin{bmatrix} 9 & 0 & 9\\ 0 & 9 & 9 \\ 0 & 0 & 9\end{bmatrix}_B.
\end{align*}
In other words, under the orthonormal basis we've found, $A$ is upper-triangular!  (We were lucky here and only had to go one step along the Schur process to get an upper-triangular matrix.  However, if the lower-right-hand block was bigger --- if we were looking at a $4 \times 4$ matrix, say ---we might have had to repeat this process  of finding an eigenvalue, extending it to a basis of the relevant space, etc. on the lower block.)

So: how can we use this to raise $A$ to a large power?  Well: notice that if we write
\begin{align*}
A_B^{50} = \left( \begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B + \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B \right)^{50},
\end{align*}
we can use the binomial theorem to write this as the following sum:
\begin{align*}
\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{50} &+ \binom{50}{1}\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{49} \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B^1 +  \binom{50}{2}\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{48} \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B^2\\
&+ \ldots  +\binom{50}{49}\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{1} \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B^{49} + \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_{B.}^{50}
\end{align*}


Cool observation: 
\begin{align*}
 \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B^{2} = \textrm{the all-zeroes matrix}!
\end{align*}


This lets us simplify  the above dramatically!  In particular, every term with a $ \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_B^{k}$ goes away for $k \geq 2$; so we have

\begin{align*}
A_B^{50} &= \begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{50} + \binom{50}{1}\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{49} \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_{B}\\
&=\begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{50} + 50 \begin{bmatrix} 9 & 0 & 0\\ 0 & 9 & 0 \\ 0 & 0 & 9\end{bmatrix}_B^{49}  \begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_{B}\\
&=\begin{bmatrix} 9^{50}& 0 & 0\\ 0 & 9^{50} & 0 \\ 0 & 0 & 9^{50}\end{bmatrix}_B + \begin{bmatrix} 50\cdot9^{49}& 0 & 0\\ 0 & 50\cdot9^{49} & 0 \\ 0 & 0 & 50\cdot9^{49}\end{bmatrix}_B\begin{bmatrix} 0 & 0 & 9\\ 0 & 0 & 9 \\ 0 & 0 & 0\end{bmatrix}_{B}\\
&=\begin{bmatrix} 9^{50}& 0 & 0\\ 0 & 9^{50} & 0 \\ 0 & 0 & 9^{50}\end{bmatrix}_B  +  \begin{bmatrix} 0& 0 & 50\cdot9^{50}\\ 0 & 0& 50\cdot9^{50} \\ 0 & 0 & 0\end{bmatrix}_B\\
&=\begin{bmatrix} 9^{50}& 0 & 50\cdot9^{50}\\ 0 & 9^{50} & 50\cdot9^{50} \\ 0 & 0 & 9^{50}\end{bmatrix}_{B.}
\end{align*}
If we convert back to the standard basis, we have
\begin{align*}
A &= \begin{bmatrix} \frac23 & \frac23 & \frac13 \\ -\frac23 & \frac13 & \frac23 \\ \frac13 & -\frac23 & \frac23 \end{bmatrix} \cdot \begin{bmatrix} 9^{50}& 0 & 50\cdot9^{50}\\ 0 & 9^{50} & 50\cdot9^{50} \\ 0 & 0 & 9^{50}\end{bmatrix}_{B} \cdot  \begin{bmatrix} \frac23 & -\frac23 & \frac13 \\ \frac23 & \frac13 & -\frac23 \\ \frac13 & \frac23 & \frac23 \end{bmatrix} \\
&= 9^{49} \cdot \begin{bmatrix} 209 & 400 & 400 \\ -50 & -91 & -100 \\ -50 & -100 & -91 \end{bmatrix}.
\end{align*}
Solution!  With relatively little work, as compared to calculating $A^{50}$ by hand!
\end{proof}



\section{The Schur Decomposition: How People Actually Find It}

It bears noting that in practice, people don't use repeated Gram-Schmidt to find this Schur decomposition!  Instead, they use the following absolutely beautiful application of the QR-decomposition to find the Schur decomposition:

\begin{thm}
(Francis 1961, Kublanovskaja 1962, Huang and Tam 2005): Take some $n \times n$ matrix $A$ with entries in $\mathbb{C}$.  Suppose that $A$ has $n$ distinct eigenvalues, and the Schur decomposition $Q_s^TR_sQ_s$.  Consider the following process:
\begin{enumerate}
\item Set $A_1 = A$.
\item Find a QR-decomposition $Q_1R_1$ of $A$.
\item Define $A_2 = R_1Q_1.$
\item Now, find a QR-decomposition $Q_2R_2$ of $A_2$.
\item Repeat this process!  I.e.\ if $A_k$ has the QR-decomposition $Q_kR_k$, use this to define $A_{k+1} = R_kQ_k$.
\end{enumerate}
Notice that because 
\begin{align*}
A_{k+1} = R_k Q_k = Q_k^T (Q_k R_k) Q_k = (Q_kR_k)_{\textrm{in the basis }Q_k^T},
\end{align*}
the eigenvalues of $A_{k}$ and $A_{k+1}$ don't change.  

Furthermore, the following very surprising fact holds: if we look at the sequence $A_1, A_2, A_3, \ldots$, the diagonals of these matrices converge to the eigenvalues of $A$!  In fact, in many situations we actually have something stronger: the sequence of the $A_i$'s will converge to $R_s$, the upper-triangular part of the Schur decomposition!  In other words, we often have the following result:
\begin{align*}
\lim_{n \to \infty} A_n = R_s.
\end{align*}
\end{thm}
A proof of this is beyond this course, but it's cool to know about this theorem nonetheless!









\end{document}