Diagonalization

Diagonalization is used all the time in computer graphics, physics, and engineering. It's a way of simplifying a transformation by changing the basis to one of eigenvectors. This is a powerful tool, and it's a great way to understand the power of eigenvectors and eigenvalues.

Diagonalizability
A square matrix is diagonalisable if it is similar to a diagonal matrix.

Thm. An $n\times n$ matrix $A$ is diagonalisable if and only if it has $n$ linearly independent eigenvectors.

While we can't guarantee that $A$ is similar to a unique diagonal matrix, we can guarantee the following: If $A$ is similar to the diagonal matrices $D_1$ and $D_2$, then $D_1$ and $D_2$ have the same set of diagonal elements (with the same multiplicities).

Thm. Let $A$ be a square matrix. For each positive integer $k$, if $x_1,\dots,x_k$ is are eigenvectors of $A$ with distinct eigenvalues $\lambda_1,\dots,\lambda_k$, then $\left\{x_1,\dots,x_k\right\}$ is linearly independent (this is just saying that eigenvectors with distinct eigenvalues are linearly independent).

Thm. If $A$ is $n\times n$ and $A$ has $n$ distinct real eigenvalues, then $A$ is diagonalizable.

But what if $A$ doesn't have $n$ distinct real eigenvalues? It may still be diagonlizable:

Thm. If $A$ is an $n\times n$ matrix with with real eigenvalues $\lambda_1,\dots,\lambda_k$, and $S_{\lambda_j}$ denotes the eigenspace of $\lambda_j$ for each $\lambda_j$. Then, $A$ is diagonalizable if and only if $$\sum_{i=1}^k \dim(S_{\lambda_i})=n.$$ Thm. If $\lambda$ is an eigenvalue of the $n\times n$ matrix $A$, then $$\dim(S_\lambda)=n-r(A-\lambda I_n)$$ because $A-\lambda I_n$ is the kernel of $A:\mathbb{R}^n\to\mathbb{R}^n$.

What's the point of diagonalization?
Let's consider some matrix $A$ that represents a transformation. What if we want to apply it three times ($A^3$)? Well, we could multiply $A$ by itself three times, but that's a lot of work. If we can find a diagonal matrix $D$ that is similar to $A$, then $A=PDP^{-1}$, and $A^3=PD^3P^{-1}$. But $D^3$ is just the diagonal elements cubed! So, we can find $A^3$ by just cubing the diagonal elements of $D$. Diagonalization also makes it easy to find the inverse of a matrix. If $A=PDP^{-1}$, then $A^{-1}=PD^{-1}P^{-1}$, and $D^{-1}$ is just the reciprocal of the diagonal elements.

You'll find diagonalization everywhere you go. For instance, in probability theory, you'll see that the transition matrix of a Markov chain is diagonalizable. In physics, you'll see that the Hamiltonian operator is diagonalizable. In computer graphics, you'll see that the transformation matrix is diagonalizable. It's a powerful tool, and it's worth understanding.

~ The End ~

What I've covered here is just the beginning of linear algebra. There's so much more to learn, and I hope you continue to explore. If you have any questions, feel free to reach out to me. Good luck!