Diagonalization

While this works for any dimension matrix, this will be of the example where the matrix is a $2\times 2$.

Suppose $A$ is a $2\times 2$ matrix with eigenvectors ${\mathbf{\text{v}}}_1,{\mathbf{\text{v}}}_2$ associated with eigenvalues ${\lambda }_1,{\lambda }_2$ respectively. By definition of an eigenvector, this would imply that for any scalars $c_1,c_2$.

$$\begin{array}{}\text{(1)}& A(c_1{\mathbf{\text{v}}}_1+c_2{\mathbf{\text{v}}}_2)={\lambda }_1{c}_1{\mathbf{\text{v}}}_1+{\lambda }_2{c}_2{\mathbf{\text{v}}}_2\end{array}$$

Say we apply $A$ a second time…

We could easily repeat this as many times as we wanted. So it is also the case that

$$\begin{array}{}\text{(2)}& A^n(c_1{\mathbf{\text{v}}}_1+c_2{\mathbf{\text{v}}}_2)={\lambda }_1^n{c}_1{\mathbf{\text{v}}}_1+{\lambda }_2^n{c}_2{\mathbf{\text{v}}}_2\end{array}$$

Since for those vectors, the transformation $A$ is just simple scaling, in the situation where we have to repeatedly apply $A$ to some vector $w$, it would clearly be preferable to get that vector in terms of ${\mathbf{\text{v}}}_1$ and ${\mathbf{\text{v}}}_2$. This seems like a job for change of basis…

So let us define the eigenbasis $\begin{array}{}\text{(3)}& B=\{{\mathbf{\text{v}}}_1,{\mathbf{\text{v}}}_2\}\end{array}$

The change of basis matrix $P_{\epsilon \leftarrow B}$ (where $\epsilon$ is the standard basis $\epsilon =\{e_1,e_2\}$), then is

By extension,

$$\begin{array}{}\text{(5)}& P_{B\leftarrow \epsilon }=(P_{\epsilon \leftarrow B}{)}^{-1}\end{array}$$

So now suppose that

This tells us that the coordinate vector of $w$ with respect to the eigenbasis $B$, $(\mathbf{\text{w}}{)}_B$, is

We know from $\text{(2)}$ then that

So

But that’s also the transformation

If we call the diagonal matrix $D=\left(\begin{array}{cc}{\lambda }_1& 0\\ 0& {\lambda }_2\end{array}\right)$, then we get

$$\begin{array}{}\text{(7)}& (A^n\mathbf{\text{w}}{)}_B=D^n(\mathbf{\text{w}}{)}_B\end{array}$$

We’re so close now. Next, we change back to the standard basis by applying $P_{\epsilon \leftarrow B}$ to both sides,

$$\begin{array}{}\text{(8)}& A^n\mathbf{\text{w}}=P_{\epsilon \leftarrow B}{D}^n(\mathbf{\text{w}}{)}_B\end{array}$$

To get things entirely in terms of the standard basis, we start by rewriting

Using $\text{(5)}$ and the fact that $(\mathbf{\text{w}}{)}_{\epsilon }=w$ (by definition of the standard basis),

$$\begin{array}{}\text{(9)}& (\mathbf{\text{w}}{)}_B=(P_{\epsilon \leftarrow B}{)}^{-1}\mathbf{\text{w}}\end{array}$$

we substitute into $\text{(8)}$

If we just denote $P_{\epsilon \leftarrow B}$ as $P$, then we finally get

$$\begin{array}{}\text{(10)}& A^n\mathbf{\text{w}}=P{D}^n{P}^{-1}\mathbf{\text{w}}\end{array}$$

General Decomposition

In general, if we know how a matrix $A$ acts on a basis, then we may either construct or decompose it via a $PD{P}^{-1}$ decomposition.

Complex Eigendecomposition of Real Matrices

Let’s take the example of a real $2\times 2$ with complex eigenvalues. Suppose $A$ is such a matrix with a complex eigenvalue $a+bi$, where $b\ne 0$, associated with a complex eigenvector $v$.

$$\begin{array}{}\text{(11)}& A\mathbf{\text{v}}=(a+bi)\mathbf{\text{v}}\end{array}$$

Let’s get a bit more specific by decomposing $v$ into its real and imaginary parts.

We can equate the real and imaginary parts on both sides,

$$\begin{array}{}\text{(12)}& A\mathrm{Re}(\mathbf{\text{v}})=a\mathrm{Re}(\mathbf{\text{v}})-b\mathrm{Im}(\mathbf{\text{v}}),A\mathrm{Im}(\mathbf{\text{v}})=b\mathrm{Re}(\mathbf{\text{v}})+a\mathrm{Im}(\mathbf{\text{v}})\end{array}$$

Now we know that $\{\mathrm{Re}(\mathbf{\text{v}}),\mathrm{Im}(\mathbf{\text{v}})\}$ is a basis of ${\mathbb{R}}^2$ because if they were linearly dependent, then $\mathrm{Im}(\mathbf{\text{v}})=k\mathrm{Re}(\mathbf{\text{v}})$. That would change $\text{(11)}$ into

Dividing by the common $1+ki$,

This implies a contradiction since the imaginary part of the left side is zero due to $A$ being real, but the right is not if $b\ne 0$. Therefore,

It then follows that

is invertible.

Well, we know how $A$ acts on the basis $B$ $\text{(13)}$:

So our matrix $D$ is then,

And our eigendecomposition of $A$ is

It can be verified that the eigenvalues of this particular $D$ are $\lambda =a\pm bi$.