\documentclass[11pt]{article}
\usepackage{amsfonts}
\usepackage{amsthm}
\usepackage{amsmath}
\usepackage{latexsym}
\usepackage{verbatim}
\usepackage[pdftex]{graphicx}
\usepackage{url}
\usepackage{cancel}
\usepackage[table]{xcolor}
\usepackage[all,arc]{xy}
\usepackage{color, colortbl}
\usepackage{mathrsfs}
\usepackage{hyperref}
\hypersetup{colorlinks=true}
\setlength{\oddsidemargin}{.25in}
\setlength{\evensidemargin}{.25in}
\setlength{\textwidth}{6in}
\setlength{\topmargin}{-0.4in}
\setlength{\textheight}{8.5in}

\input{preamble.tex}

\newcommand\overmat[2]{%
  \makebox[0pt][l]{$\smash{\color{white}\overbrace{\phantom{%
    \begin{matrix}#2\end{matrix}}}^{\text{\color{black}#1}}}$}#2}
\newcommand\bovermat[2]{%
  \makebox[0pt][l]{$\smash{\overbrace{\phantom{%
    \begin{matrix}#2\end{matrix}\hspace{15pt}}}^{\text{#1}}}$}#2}
\newcommand\partialphantom{\vphantom{\frac{\partial e_{P,M}}{\partial w_{1,1}}}}

\begin{document}

\lecture{7: Similar/Elementary Matrices; Jordan Canonical Form}{Week 8}

Over the last two weeks of lecture, we proved the \textbf{spectral theorem}, which stated that any real-valued symmetric matrix $A$ can be written as the product $UDU^T$, for some real-valued diagonal matrix $D$ and orthogonal real-valued matrix $U$.  This was a remarkably strong result as contrasted with the Schur decomposition, which stated that any arbitrary matrix $A$ can be written in the form $URU^*$, for some upper-triangular $R$ and unitary $U$.

Over the next two weeks, we are going to attempt to study another decomposition result, namely the Jordan canonical form!  In next week's talks we will explicitly state what the Jordan canonical form is.  For now, however, consider the following question: suppose we take an arbitrary matrix $A$.  What kinds of matrices $B$ can we find such that $A = UBU^{-1}$, for some $U$?  We know that if we restrict $U$ to be unitary, the best we can get is upper-triangular (Schur!), but what if we weaken our restrictions on $U$?  Can we make $B$ into something nicer?  How much nicer?

With this desire in mind, we introduce our first topic of the week: similar matrices!

\section{Similar Matrices}
We have been working with the concept of \textbf{similarity} for essentially this entire course, so this definition should look fairly familiar:
\begin{defn}
Two matrices $A$, $B$ are called \textbf{similar} if there is some matrix $U$ such that $A = UBU^{-1}$.  If we want to specify what $U$ is, we can specifically state that $A$ and $B$ are similar \textbf{via} $U$.
\end{defn}

To illustrate this definition, we rephrase some of the results we've recently proven using this language of similarity:
\begin{thm}
(Spectral theorem.)  Any real-valued symmetric matrix is similar to a real-valued diagonal matrix via a real-valued orthogonal matrix.
\end{thm}
\begin{thm}
(Schur decomposition.)  Any $n \times n$ matrix $A$ is similar to an upper-triangular matrix via a unitary matrix.
\end{thm}
\begin{thm}
The matrix $\begin{bmatrix} 1 & 1\\ 0 & 1\end{bmatrix}$ is not similar to any diagonal matrix $D$.
\end{thm}

Notice that similarity is an \textbf{equivalence relation}: i.e. 
\begin{enumerate}
\item \textbf{Reflexivity}: any matrix $A$ is similar to itself, because $A = IAI^{-1}$.
\item \textbf{Symmetry}: If $A$ is similar to $B$ via a matrix $U$, then $B$ is similar to $A$ via the matrix $U^{-1}$; this is because $A = UBU^{-1} \Rightarrow B = U^{-1}AU$.
\item \textbf{Transitivity}: If $A$ is similar to $B$ via a matrix $U$, and $B$ is similar to $C$ via a matrix $V$, then $A$ is similar to $C$; this is because we can write $A = UBU^{-1} = UVCV^{-1}U^{-1} = (UV)C(UV)^{-1}$.
\end{enumerate}
This is a nice property!  In particular, to show that two matrices $A, B$ are similar we will often just show that they are both similar to some matrix $C$, and use symmetry and transitivity to conclude that $A$ and $B$ are similar to each other.  

In this language, the question we mentioned at the start of class is the following: given an arbitrary $n \times n$ matrix $A$, what ``nice'' kinds of matrices can we find that are similar to $A$?  The Schur decomposition tells us that $A$ is similar to an upper-triangular matrix: can we improve this to something nicer than upper-triangular?

Working with general matrices is a pain; they can do lots of awful things.  Instead, let's work with upper-triangular matrices!  Because any matrix is similar to an upper-triangular matrix, if we can find some ``nicer'' form that upper-triangular matrices are similar to, then (by transitivity) every matrix will be similar to that ``nicer'' form! 

This simplifies things somewhat.  However, we can simplify things more: instead of looking at similarities via arbitrary matrices, why not look at similarity via particularly nice and easy-to-understand matrices?  In particular, recall the concept of \textbf{elementary matrices}, from last quarter:

\section{Review: Elementary Matrices}
\begin{definition}

The first matrix, $E_{\textrm{multiply entry k by }\lambda}$, is the matrix corresponding to the linear map that multiplies its $k$-th co\"ordinate by $\lambda$ and does not change any of the others:
\begin{align*}
E_{\textrm{multiply entry }k\textrm{ by }\lambda} = \begin{bmatrix}
1 & 0 & \ldots & 0 & 0 & 0 & \ldots  & 0\\
0 & 1 & \ldots & 0 & 0 & 0 &  \ldots  & 0\\
\vdots & \vdots & \ddots &\vdots &\vdots & \vdots & \ddots  & \vdots\\
0 & 0 & \ldots & 1 & 0 & 0 &  \ldots  & 0\\
0 & 0 & \ldots & 0 & \cellcolor{blue!15}\lambda & 0 &  \ldots  & 0\\
0 & 0 & \ldots & 0 & 0 & 1 &  \ldots  & 0\\
\vdots & \vdots & \ddots &\vdots &\vdots & \vdots & \ddots  & \vdots\\
0 & 0 & \ldots & 0 & 0 & 0 &  \ldots  & 1\\
\end{bmatrix}
\end{align*}
This matrix has 1's down its diagonal and 0's elsewhere, with an exception for the value at $(k,k)$, which is $\lambda$.  

The second matrix, $E_{\textrm{switch entry }k\textrm{ and entry }l}$, corresponds to the linear map that swaps its $k$-th co\"ordinate with its $l$-th co\"ordinate, and does not change any of the others:
\newcolumntype{g}{>{\columncolor{blue!15}}c}
\begin{align*}
E_{\textrm{switch entry }k\textrm{ and entry }l} = \left[\begin{array}{cccgcgcc}
1 & 0 & \ldots & 0 & \ldots & 0 & \ldots  & 0\\
0 & 1 & \ldots & 0 & \ldots & 0 &  \ldots  & 0\\
0 & 0 & \ldots & 0 & \ldots & 1 &  \ldots  & 0\\
0 & 0 & \ldots & 0 & \ldots & 0 &  \ldots  & 0\\
0 & 0 & \ldots & 1 & \ldots & 0 &  \ldots  & 0\\
\vdots & \vdots & \ddots & \vdots &\ddots & \vdots & \ddots  & \vdots\\
0 & 0 & \ldots & 0 & \ldots & 0 &  \ldots  & 1\\
\end{array}\right]
\end{align*}
You can create this matrix by starting with a matrix with 1's down its diagonal and 0's elsewhere, and switching the $k$-th and $l$-th columns.

Finally, the third matrix, $E_{\textrm{add }\lambda\textrm{ copies of entry }k\textrm{ to entry }l}$, for $k \neq l$, corresponds to the linear map that adds $\lambda$ copies of its $k$-th co\"ordinate to its $l$-th co\"ordinate and does not change any of the others:
\begin{align*}
E_{\textrm{add }\lambda\textrm{ copies of entry }k\textrm{ to entry }l} = \begin{bmatrix}
1 & \ldots & 0 & 0 & \ldots & 0 & 0 & \ldots & 0\\
\vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots  & \ddots & \vdots\\
0 & \ldots & 1 & 0 & \ldots & 0 & 0 & \ldots & 0\\
0 & \ldots & 0 & 1 & \ldots & 0 & 0 & \ldots & 0\\
\vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots  & \ddots & \vdots\\
0 & \ldots & 0 & 0 & \ldots & 1 & 0 & \ldots & 0\\
0 & \ldots & \cellcolor{blue!15}\lambda & 0 & \ldots & 0 & 1 & \ldots & 0\\
\vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots  & \ddots & \vdots\\
0 & \ldots & 0 & 0 & \ldots & 0 & 0 & \ldots & 1\\
\end{bmatrix}
\end{align*}
This matrix has 1's down its diagonal and 0's elsewhere, with an exception for the value in row $l$, column $k$, which is $\lambda$. 

These three matrices are called the \textbf{elementary} matrices, if you don't remember them from the previous quarter of 108.
\end{definition}

As a reminder, this is how elementary matrices interact with matrix multiplication:
\begin{theorem}\label{matrixmult}
Take any $n \times n$ matrix $A$.  Suppose that we are looking at the products $E \cdot A$, $A \cdot E$, where $E$ is one of our elementary matrices. Then, we have the following three possible situations:
\begin{itemize}
\item If $E = E_{\textrm{multiply entry k by }\lambda}$, then $E\cdot A$ would be the matrix $A$ with its $k$-th row multiplied by $\lambda$, and $A \cdot E$ is the matrix $A$ with its $k$-th columns multiplied by $\lambda$.
\item If $E = E_{\textrm{switch entry }k\textrm{ and entry }l}$, then $E\cdot A$ would be the matrix $A$ with its $k$-th and $l$-th rows swapped, and $A \cdot E$ is the matrix $A$ with its $k$-th and $l$-th columns swapped.
\item If $E = E_{\textrm{add }\lambda\textrm{ copies of entry }k\textrm{ to entry }l}$, then $E\cdot A$ would be the matrix $A$ with $\lambda$ copies of its $k$-th row added to its $l$-th row, and $A \cdot E$ is a matrix $A$ with $\lambda$ copies of its $l$-th column added to its $k$-th column.
\end{itemize}
\end{theorem}

Finally, we also calculated the inverses of these matrices:
\begin{theorem}\label{matrixinv}
Take any $n \times n$ elementary matrix $E$.  Then, we have the following: 
\begin{itemize}
\item If $E = E_{\textrm{multiply entry k by }\lambda}$, then $E^{-1} = E_{\textrm{multiply entry k by }\frac{1}{\lambda}}$, whenever $\lambda \neq 0$.
\item If $E = E_{\textrm{switch entry }k\textrm{ and entry }l}$, then $E^{-1} = E$; i.e. $E$ is its own inverse.
\item If $E = E_{\textrm{add }\lambda\textrm{ copies of entry }k\textrm{ to entry }l}$, then $E^{-1} = E_{\textrm{add }-\lambda\textrm{ copies of entry }k\textrm{ to entry }l}$.
\end{itemize}
\end{theorem}

We calculate a few quick examples to illustrate what's going on here:

\begin{example}
Suppose that $A$ is some arbitrary $5 \times 5$ matrix.  Calculate $EAE^{-1}$, where $E$ is one of the following matrices:
\begin{itemize}
\item $E_{\textrm{multiply entry 2 by }\lambda}$, for $\lambda \neq 0$.
\item $E_{\textrm{switch entry }1\textrm{ and entry }4}$.
\item $E_{\textrm{add }\lambda\textrm{ copies of entry }2\textrm{ to entry }5}$
\end{itemize}
\end{example}
\begin{proof}
We simply calculate, starting first with $E_{\textrm{multiply entry 2 by }\lambda}$:
\begin{align*}
&\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & \lambda & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix} \cdot 
\begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 1/\lambda & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix} \\
= &\begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
\lambda a_{21} &\lambda  a_{22} &\lambda  a_{23} &\lambda  a_{24} &\lambda  a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 1/\lambda & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix}\\
 = &\begin{bmatrix}
a_{11} & a_{12}/\lambda & a_{13} & a_{14} & a_{15} \\
\lambda a_{21} &  a_{22} &\lambda  a_{23} &\lambda  a_{24} &\lambda  a_{25} \\
a_{31} & a_{32}/\lambda & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42}/\lambda & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52}/\lambda & a_{53} & a_{54} & a_{55} \\
\end{bmatrix}_.
\end{align*}
Note that this is just the original matrix $A$, with its second row scaled by $\lambda$ and its second column scaled by $1/\lambda$; furthermore, note that the entry $(2,2)$ that is in both this row and this column is unchanged after this similarity transformation.

We now consider $E_{\textrm{switch entry }1\textrm{ and entry }4}$:
\begin{align*}
&\begin{bmatrix}
0 & 0 & 0 & 1 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix} \cdot 
\begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
0 & 0 & 0 & 1 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix} \\
= &\begin{bmatrix}
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{21} &  a_{22} &  a_{23} &  a_{24} &  a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
0 & 0 & 0 & 1 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
1 & 0 & 0 & 0 & 0\\
0 & 0 & 0 & 0 & 1
\end{bmatrix} \\
 = &\begin{bmatrix}
a_{44} & a_{42} & a_{43} & a_{41} & a_{45} \\
a_{24} &  a_{22} &  a_{23} &  a_{21} &  a_{25} \\
a_{34} & a_{32} & a_{33} & a_{31} & a_{35} \\
a_{14} & a_{12} & a_{13} & a_{11} & a_{15} \\
a_{54} & a_{52} & a_{53} & a_{51} & a_{55} \\
\end{bmatrix}_.
\end{align*}
Note that this is the original matrix $A$ with its $1$st and $4$th rows swapped, as well as its $1$st and $4$th columns.  

We finally consider $E_{\textrm{add }\lambda\textrm{ copies of entry }2\textrm{ to entry }5}$:
\begin{align*}
&\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & \lambda & 0 & 0 & 1
\end{bmatrix} \cdot 
\begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & -\lambda & 0 & 0 & 1
\end{bmatrix} \\
= &\begin{bmatrix}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
\lambda a_{21} + a_{51} & \lambda a_{22} + a_{52} & \lambda a_{21} + a_{53} & \lambda a_{21} + a_{54} & \lambda a_{21} + a_{55} \\
\end{bmatrix} \cdot 
\begin{bmatrix}
1 & 0 & 0 & 0 & 0\\
0 & 1 & 0 & 0 & 0\\
0 & 0 & 1 & 0 & 0\\
0 & 0 & 0 & 1 & 0\\
0 & -\lambda & 0 & 0 & 1
\end{bmatrix}\\
 = &\begin{bmatrix}
a_{11} & a_{12} - \lambda  a_{15}& a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} - \lambda  a_{25} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} - \lambda  a_{35} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} - \lambda  a_{45} & a_{43} & a_{44} & a_{45} \\
\lambda a_{21} + a_{51} & \lambda a_{22} + a_{52} - \lambda  a_{55} & \lambda a_{21} + a_{53} & \lambda a_{21} + a_{54} & \lambda a_{21} + a_{55} \\
\end{bmatrix} _.
\end{align*}
Note that this is just the original matrix $A$ with $\lambda$ copies of its $2$nd row added to its $5$th row, and $-\lambda$ copies of its $5$th column added to its $2$nd column. 
\end{proof}



\section{The Interactions Between Elementary Matrices and Similarity}

The results above from our example hold in general:

\begin{prop}
Suppose that $E$ is an elementary matrix of the form $E_{\textrm{multiply entry k by }\lambda}$, for $\lambda \neq 0$, and $A$ is an $n \times n$ matrix.  Then $EAE^{-1}$ is the matrix $A$ with its $k$-th row scaled by $\lambda$ and $k$-th column scaled by $\frac{1}{\lambda}$.  In particular, note that the entry $A(k,k)$ is unchanged, as it is in both the $k$-th row and $k$-th column.
\end{prop}
\begin{prop}
Suppose that $E$ is an elementary matrix of the form $E_{\textrm{switch entry }k\textrm{ and entry }l}$, and $A$ is an $n \times n$ matrix.  Then $EAE^{-1}$ is the matrix $A$ with its $k$-th and $l$-th rows swapped, as well as its $k$-th and $l$-th columns.  
\end{prop}
\begin{prop}
Suppose that $E$ is an elementary matrix of the form $E_{\textrm{add }\lambda\textrm{ copies of entry }k\textrm{ to entry }l}$, and $A$ is an $n \times n$ matrix.  Then $EAE^{-1}$ is the matrix $A$, with $\lambda$ copies of its $k$-th row added to its $l$-th row, and $-\lambda$ copies of its $l$-th column added to its $k$-th column. 

Note that on the HW, you studied this product!  In particular, you proved that whenever $A$ is upper-triangular and $l < k$, then $EAE^{-1}$ is still an upper-triangular matrix.
\end{prop}



The second of these properties can be generalized in a fairly beautiful way:
\section{Permutation Matrices and Similarity}

\begin{defn}
A \textbf{permutation} $\sigma$ of the list $(1, \ldots n)$ is simply some way to reorder this list into some other $(\sigma(1), \ldots \sigma(n))$.  (If you prefer to think about functions, $\sigma$ is simply a bijection from $\{1, \ldots n\}$ to $\{1, \ldots n\}$.)

Given a permutation $\sigma$, there is clearly some way to ``undo'' $\sigma$ and return our list to its original state.  This ``unshuffling'' is itself a permutation, as it gives us a way to reorder the elements $(1, \ldots n)$; we denote it by $\sigma^{-1}$.  (Again, if you prefer functions, $\sigma^{-1}$ is jus the inverse of the map $\sigma$.)
\end{defn}
For example, the map
\begin{align*}
\sigma(1) = 2, \sigma(2)=  3, \sigma(3) = 1
\end{align*}
 is a permutation of the list $(1,2,3)$.  The inverse of this map is easy to construct; just ``undo'' the above, i.e. set
\begin{align*}
\sigma^{-1}(2) = 1, \sigma^{-1}(3) = 2, \sigma^{-1}(1) = 3.
\end{align*}


\begin{defn}
Given any permutation $\sigma$ of $(1, \ldots n$), the \textbf{permutation matrix} $P_\sigma$ is simply the $n \times n$ matrix whose $i$-th column is given by $e_{\sigma(i)}$.  In other words,
\begin{align*}
P_\sigma = \begin{bmatrix} \vdots & \vdots & &\vdots \\ \vec{e_{\sigma(1)}} & \vec{e_{\sigma(2)}} & \ldots & \vec{e_{\sigma(n)}} \\  \vdots & \vdots & &\vdots \\ \end{bmatrix}
\end{align*}
\end{defn}

For example, if we use the permutation $\sigma$ on the list $(1,2,3)$ that we defined above, we have
\begin{align*}
P_{\sigma} = \begin{bmatrix} \vdots & \vdots & \vdots \\ \vec{e_{\sigma(1)}} & \vec{e_{\sigma(2)}} & \vec{e_{\sigma(3)}} \\ \vdots & \vdots & \vdots \\ \end{bmatrix} = \begin{bmatrix} \vdots & \vdots & \vdots \\ \vec{e_{2}} & \vec{e_{3}} & \vec{e_{1}} \\ \vdots & \vdots & \vdots \\ \end{bmatrix} = \begin{bmatrix} 0 & 0& 1\\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ \end{bmatrix}_.
\end{align*} 


\begin{prop}
If $\sigma$ is a permutation of $(1,\ldots n)$ with inverse $\sigma^{-1}$, then $P_\sigma$ is an invertible matrix with inverse $P_{\sigma^T}$.
\end{prop}
\begin{proof}
The rows of $P_\sigma$ form an orthonormal basis for $\mathbb{R}^n$, because they're all just $\vec{e_i}$'s but reordered.  Therefore $P_\sigma$ is an orthonormal matrix, and we know that orthonormal matrices have inverses given by their transposes!
\end{proof}



These matrices interact with arbitrary matrices in the following way:

\begin{prop}
Take an arbitrary $n \times n$ matrix $A$, and an arbitrary permutation $\sigma$ of $(1, \ldots n)$.  Then $A P_\sigma $ is just the matrix $A$, but with its columns permuted by $\sigma$.
\end{prop}
\begin{proof}
Just look at the product: 
\begin{align*}
\begin{bmatrix} a_{11} & \ldots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{n1} & \ldots & a_{nn} \end{bmatrix} \cdot  \begin{bmatrix} \vdots & \vdots & &\vdots \\ \vec{e_{\sigma(1)}} & \vec{e_{\sigma(2)}} & \ldots & \vec{e_{\sigma(n)}} \\  \vdots & \vdots & &\vdots \\ \end{bmatrix}_.
\end{align*}
An entry $(i,j)$ in this product is simply the dot product of the row  $(a_{i1}, \ldots a_{in})$ of $A$ with the $j$-th column $\vec{e_{\sigma(j)}}$ of $P_\sigma$.  This is precisely $a_{i, \sigma(j)}$!  In other words, the entry in $(i,j)$ of our product is just the entry in $(i, \sigma(j))$ of $A$ itself.  This just means that that  $A P_\sigma $ is the matrix $A$ with its columns permuted by $\sigma$, as claimed.
\end{proof}


\begin{prop}
Take an arbitrary $n \times n$ matrix $A$, and an arbitrary permutation $\sigma$ of $(1, \ldots n)$.  Then $P_\sigma^T A$ is just the matrix $A$, but with its rows permuted by $\sigma$.
\end{prop}
\begin{proof}
Again, just look at the product: 
\begin{align*} \begin{bmatrix} \ldots &  \vec{e_{\sigma(1)}} & \ldots \\
\ldots &  \vec{e_{\sigma(2)}} & \ldots \\
 & \vdots & \\
\ldots & \vec{e_{\sigma(n)}}& \ldots \end{bmatrix} \cdot 
\begin{bmatrix} a_{11} & \ldots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{n1} & \ldots & a_{nn} \end{bmatrix}_.
\end{align*}
An entry $(i,j)$ in this product is simply the dot product of the row  $e_{\sigma(i)}$ of $P_\sigma^T$ with the $j$-th column $(a_{1j}, \ldots a_{nj})$ of $A$.  This is precisely $a_{\sigma(i), j}$!  In other words, the entry in $(i,j)$ of our product is just the entry in $(\sigma(i), j)$ of $A$ itself.  This just means that that  $P_\sigma^T A $ is the matrix $A$ with its rows permuted by $\sigma$, as claimed.
\end{proof}



The reason we care about this is the following proposition.  It's kinda complicated, but the result we get from it is really useful:
\begin{prop}
Suppose that $A$ is a $n \times n$ matrix with the following form:
\begin{align*}
\left[ \begin{array}{c|c|c|c}
B_1 & 0 & \ldots & 0\\\hline
0 & B_2 & \ldots & 0 \\\hline
\vdots & \vdots & \ddots & \vdots \\\hline
0 & 0 & \ldots & B_k
\end{array}\right]_.
\end{align*}
In the above matrix, the matrix $B_1$ is a $l_1 \times l_1$ matrix, $B_2$ is a $l_2 \times l_2$ matrix, and in general $B_i$ is a $l_i \times l_i$ matrix, with the constants $l_1, \ldots l_k$ chosen so that $l_1 + \ldots + l_k = n$.  The rest of the $n \times n$ matrix $A$ is filled in with 0's, as indicated.

Consider the permutation $\sigma_{a,b}$  on $(1, \ldots n)$ defined as follows:
\begin{align*}
\sigma_{a,b}(x) &= x, \textrm{ if } x \leq a.\\
\sigma_{a,b}(x) &= x+b, \textrm{ if } a < x \leq n-b.\\
\sigma_{a,b}(x) &= x+2b-n - 1, \textrm{ if } n-b < x \leq n.\\
\end{align*} 
In other words, $\sigma_{a,b}$ fixes the first $a$ elements of $(1,\ldots n)$, and then cycles the remaining elements of $(1, \ldots n)$ by $b$.  For example, $\sigma_{2,3}$ can be thought of as a permutation of $(1,2,3,4,5,6,7)$ such that
\begin{align*}
&\sigma(1) = 1, \sigma(2) = 2, \sigma(3) = 3+3 = 6, \sigma(4) = 4+3 = 7, \sigma(5) = 5+2\cdot 3  - 7 -1= 3,\\& \sigma(6) = 6 + 2\cdot 3 - 7 - 1 = 4, \sigma(7) = 7 + 2\cdot 3 - 7 - 1 = 5: \textrm{ in other words,}\\
\Rightarrow &(1,2,3,4,5,6,7) \xrightarrow{\sigma} (1,2,6,7,3,4,5).
\end{align*}

Pick any integer $i$, and let $a = l_1 + \ldots l_{i-1}, b = l_k$.  Then 
\begin{align*}
P_{\sigma_{a,b}}^T A P_{\sigma_{a,b}} = 
\left[ \begin{array}{c|c|c||c|c|c|c|c}
B_1 & \ldots & 0 & 0& 0 &0 &\ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & B_{i-1} & 0& 0 &0 & \ldots & 0 \\\hline\hline
0 & \ldots & 0 & B_k & 0& 0 & \ldots & 0 \\\hline
0 & \ldots & 0& 0 & B_{i} & 0 & \ldots & 0\\\hline
0 & \ldots & 0& 0 & 0 & B_{i+1} & \ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & 0& 0& 0 & 0 & \ldots & B_{k-1}\\
\end{array}\right]_.
\end{align*}
In other words, $P_{\sigma_{a,b}}^T A P_{\sigma_{a,b}}$ is the same original matrix as $A$, except the ``blocks'' $B_i, \ldots B_k$ have been ``cycled through:''  i.e. block $B_k$ is where $B_{i}$ used to be, $B_{i}$ is where $B_{i+1}$ was, $B_{i+1}$ is where $B_{i+2}$ was, and so on/so forth.
\end{prop}
\begin{proof}
Again, just look at the product!  In particular, start with $P_\sigma^T A$:
\begin{align*}
P_{\sigma_{a,b}}^T A&=\begin{bmatrix} \ldots &  \vec{e_{\sigma(1)}} & \ldots \\
\ldots &  \vec{e_{\sigma(2)}} & \ldots \\
 & \vdots & \\
\ldots & \vec{e_{\sigma(n)}}& \ldots \end{bmatrix}  \cdot
\left[ \begin{array}{c|c|c||c|c|c|c|c}
B_1 & \ldots & 0 & 0& 0 &0 &\ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & B_{i-1} & 0& 0 &0 & \ldots & 0 \\\hline\hline
0 & \ldots & 0 & B_i & 0& 0 & \ldots & 0 \\\hline
0 & \ldots & 0& 0 & B_{i+1} & 0 & \ldots & 0\\\hline
0 & \ldots & 0& 0 & 0 & B_{i+2} & \ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & 0& 0& 0 & 0 & \ldots & B_{k}\\
\end{array}\right]\\
\end{align*}
As noted before, this is the matrix $A$ but with its rows permuted by $\sigma$.  But what does this mean?  Well, $\sigma$ does nothing to the first $l_1 + \ldots l_{i-1}$ rows, so the blocks $B_1, \ldots B_{i-1}$ are untouched.  After that, $\sigma$ shifts the remaining rows by $l_k$ forward: i.e. it sends $A$ to the following matrix:
\begin{align*}
\left[ \begin{array}{c|c|c||c|c|c|c|c}
B_1 & \ldots & 0 & 0& 0 &0 &\ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & B_{i-1} & 0& 0 &0 & \ldots & 0 \\\hline\hline
0 & \ldots & 0& 0& 0 &  \ldots & 0 & B_{k}\\\hline
0 & \ldots & 0 & B_i & 0&  \ldots & 0 &0 \\\hline
0 & \ldots & 0& 0 & B_{i+1} &  \ldots & 0 &0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \ddots & \vdots & \vdots\\\hline
0 & \ldots & 0& 0& 0  & \ldots & B_{k-1} & 0\\
\end{array}\right]\\
\end{align*}

Now, what happens if we multiply this on the right by $P_\sigma$?  Well, we know that multiplying by $P_\sigma$ on the right just permutes columns by $\sigma$.  Again, what does this mean?  Well, $\sigma$ does nothing to the first $l_1 + \ldots l_{i-1}$ columns, so the blocks $B_1, \ldots B_{i-1}$ are again untouched.  After that, $\sigma$ shifts the remaining columns by $l_k$ forward: i.e.\ we get
\begin{align*}
P_{\sigma_{a,b}}^T A P_{\sigma_{a,b}} = 
\left[ \begin{array}{c|c|c||c|c|c|c|c}
B_1 & \ldots & 0 & 0& 0 &0 &\ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & B_{i-1} & 0& 0 &0 & \ldots & 0 \\\hline\hline
0 & \ldots & 0 & B_k & 0& 0 & \ldots & 0 \\\hline
0 & \ldots & 0& 0 & B_{i} & 0 & \ldots & 0\\\hline
0 & \ldots & 0& 0 & 0 & B_{i+1} & \ldots & 0\\\hline
\vdots & \ddots & \vdots& \vdots & \vdots &  \vdots & \ddots & \vdots\\\hline
0 & \ldots & 0& 0& 0 & 0 & \ldots & B_{k-1}\\
\end{array}\right]_.
\end{align*}
But this is what we claimed!  So we're done with our proof.
\end{proof}

The main reason we care about this proof is that it gives us the following remarkably useful corollary:
\begin{corollary}
Suppose that $A$ is a $n \times n$ matrix with the following form:
\begin{align*}
A = \left[ \begin{array}{c|c|c|c}
B_1 & 0 & \ldots & 0\\\hline
0 & B_2 & \ldots & 0 \\\hline
\vdots & \vdots & \ddots & \vdots \\\hline
0 & 0 & \ldots & B_k
\end{array}\right]_.
\end{align*}

Take any other matrix $C$that consists of the same blocks on the diagonal, except in some other order: i.e.\
\begin{align*}
C = \left[ \begin{array}{c|c|c|c}
B_{\sigma(1)} & 0 & \ldots & 0\\\hline
0 & B_{\sigma(2)} & \ldots & 0 \\\hline
\vdots & \vdots & \ddots & \vdots \\\hline
0 & 0 & \ldots & B_{\sigma(k)}
\end{array}\right]_,
\end{align*}
for some permutation $\sigma$ of $(1, \ldots k)$.  Then $A$ and $C$ are similar.  
\end{corollary}
\begin{proof}
Repeatedly apply our earlier proposition to move the blocks of $A$ around using the $P_{\sigma_{a,b}}$ matrices.  Since we performed these switches by multiplying on the left by $P_\sigma^T$ and on the right by this matrix's inverse, $P_\sigma$, these switches preserve similarity.  So we can just perform whatever sequence of switches is necessary to turn $A$ into $C$, and we're done!
\end{proof}

For notational convenience, make the following definition:
\begin{defn}
If $A$ is a matrix in the form
\begin{align*}
A = \left[ \begin{array}{c|c|c|c}
B_1 & 0 & \ldots & 0\\\hline
0 & B_2 & \ldots & 0 \\\hline
\vdots & \vdots & \ddots & \vdots \\\hline
0 & 0 & \ldots & B_k
\end{array}\right]_,
\end{align*}
call $A$ a \textbf{block-diagonal} matrix.
\end{defn}



In next week's classes, we will prove that all matrices are similar to something called their Jordan Canonical Form, using only three tools:
\begin{itemize}
\item The Schur decomposition.
\item The lemmas about how conjugating by an elementary matrix changes a matrix.
\item The result above about how we can ``rearrange'' the blocks of a block-diagonal matrix while preserving similarity.
\end{itemize}





\end{document}
