Column and Row Perspective

How to simplify matrix multiplication with the best perspectives (and also find certain inverse matrices fast!)

Authors

Affiliations

Taylor F.

None

Published

June 6, 2022

Column Perspective

First, let’s start with general matrix column vector multiplication. We’ll focus on $2 \times 2$ matrices first, as the principles extend very naturally for larger matrices.

If we define the matrices

A = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}), x = (\begin{matrix} x_{1} \\ x_{2} \end{matrix})

Then,

\begin{matrix} (1) & A x = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) = (\begin{matrix} a_{11} x_{1} + a_{12} x_{2} \\ a_{21} x_{1} + a_{22} x_{2} \end{matrix}) \end{matrix}

But look what happens when we separate this result by the $x$ values.

(\begin{matrix} a_{11} x_{1} + a_{12} x_{2} \\ a_{21} x_{1} + a_{22} x_{2} \end{matrix}) = (\begin{matrix} a_{11} x_{1} \\ a_{21} x_{1} \end{matrix}) + (\begin{matrix} a_{12} x_{2} \\ a_{22} x_{2} \end{matrix})

(\begin{matrix} a_{11} x_{1} + a_{12} x_{2} \\ a_{21} x_{1} + a_{22} x_{2} \end{matrix}) = x_{1} (\begin{matrix} a_{11} \\ a_{21} \end{matrix}) + x_{2} (\begin{matrix} a_{12} \\ a_{22} \end{matrix})

(\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) = x_{1} (\begin{matrix} a_{11} \\ a_{21} \end{matrix}) + x_{2} (\begin{matrix} a_{12} \\ a_{22} \end{matrix})

Which are just the original columns of $A$ . So, if we denote the columns of $A$ by $A_{1}, A_{2}$ , we can say that

\begin{matrix} (2) & A (\begin{matrix} x_{1} \\ x_{2} \end{matrix}) = x_{1} A_{1} + x_{2} A_{2} \end{matrix}

Which we can interpret as saying: “we want $x_{1}$ of the first column of $A$ , and $x_{2}$ of the second column of $A$ .”

And, in general, if we denote the columns of an $m \times n$ matrix $A$ by $A_{1}, \dots, A_{n}$ , we can say that

A (\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}) = x_{1} A_{1} + x_{2} A_{2} + \dots + x_{n} A_{n}

Thus, when multiplying a matrix on the right by a column vector, the column vector tells us how many of each column we are taking.

Applying this to general matrix multiplication

Suppose we have that

A = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}), B = (\begin{matrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{matrix})

Consider the product $A B$ ,

A B = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{matrix})

A B = (\begin{matrix} a_{11} b_{11} + a_{12} b_{21} & a_{11} b_{12} + a_{12} b_{22} \\ a_{21} b_{11} + a_{22} b_{21} & a_{21} b_{12} + a_{22} b_{22} \end{matrix})

Now, look at each column of the product. Notice that the second column of $B$ has no bearing on the first column of $A B$ . Similarly, the first column of $B$ has no effect on the second column of $A B$ .

If you don’t quite see what I mean, look at what happens if we separate the multiplication by the columns of $B$ .

A B_{1} = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} b_{11} \\ b_{21} \end{matrix})

A B_{1} = (\begin{matrix} a_{11} b_{11} + a_{12} b_{21} \\ a_{21} b_{11} + a_{22} b_{21} \end{matrix}) = b_{11} A_{1} + b_{21} A_{2}

Which is the first column of $A B$ .

A B_{2} = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} b_{12} \\ b_{22} \end{matrix})

A B_{2} = (\begin{matrix} a_{11} b_{12} + a_{12} b_{22} \\ a_{21} b_{12} + a_{22} b_{22} \end{matrix}) = b_{12} A_{1} + b_{22} A_{2}

Which is the second column of $A B$ .

Thus, we see that

A B = (A B_{1} A B_{2})

And more generally,

\begin{matrix} (3) & A (B_{1} \dots B_{n}) = (A B_{1} \dots A B_{n}) \end{matrix}

That is to say, each column of the product, is just the left matrix times the individual column. Hence, we can use the column perspective for each column individually.

Column Perspective Examples

This can GREATLY speed up certain computations.

For example,

A (\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}) = A_{1}

will just give us the first column of $A$ . And

A (\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}) = A_{1} + A_{2}

is just the first column and the second column added together.

This can also help us in the reverse direction! Take the example of solving

(\begin{array}{cccc} 1 & 0 & 3 & 0 \\ 0 & 1 & - 1 & 0 \\ 0 & 0 & 0 & 0 \end{array})

We can see that if we take the third column as it is (that is to say, taking exactly $1$ of it), we can cancel out the nonzero entries by taking $- 3$ of column one, and $1$ of column two. Thus, a solution will be $(- 3, 1, 1)$ . Since any scalar multiple of this vector will also yield the zero vector, the general solution will be

x = c (\begin{matrix} - 3 \\ 1 \\ 1 \end{matrix})

since taking $- 3$ of the first column, $1$ of the second column, and $1$ of the third column, will cancel everything out. No need to turn it back into equations and solve or whatever.

Let’s use an example of matrix multiplication

(\begin{matrix} - 2 & 1 & 1 \\ 1 & - 2 & 1 \\ 1 & 1 & - 2 \end{matrix}) (\begin{matrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{matrix})

The first column of the right matrix tells us to add up the second and third columns: $(1, - 2, 1) + (1, 1, - 2) = (2, - 1, - 1)$ . The second column tells us to add the first and third columns: $(- 2, 1, 1) + (1, 1, - 2) = (- 1, 2, - 1)$ . And the third column tells us to add up the first and second columns: $(- 2, 1, 1) + (1, - 2, 1) = (- 1, - 1, 2)$ . Therefore, the product will be

(\begin{matrix} - 2 & 1 & 1 \\ 1 & - 2 & 1 \\ 1 & 1 & - 2 \end{matrix}) (\begin{matrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{matrix}) = (\begin{matrix} 2 & - 1 & - 1 \\ - 1 & 2 & - 1 \\ - 1 & - 1 & 2 \end{matrix})

Personally, I can say that I prefer adding up the columns as opposed to doing nine three-dimensional dot products.

Row Perspective

Row perspective, while not quite as useful as column perspective, still has its share of uses and applications. It is essentially the transpose of column perspective. Then, we will define our notation for the rows of an $m \times n$ matrix $A$ as $a_{1}^{T}, \dots, a_{m}^{T}$ .

Row perspective, is as follows.

\begin{matrix} (4) & (\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \\ ⋮ \\ a_{m}^{T} \end{matrix}) B = (\begin{matrix} a_{1}^{T} B \\ a_{2}^{T} B \\ ⋮ \\ a_{m}^{T} B \end{matrix}) \end{matrix}

Where the entries of $a_{i}^{T}$ tell you how many of each row of $B$ to take.

To see this for a $2 \times 2$ ,

(\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \end{matrix}) B = (\begin{matrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{matrix}) (\begin{matrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{matrix})

(\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \end{matrix}) B = (\begin{matrix} a_{11} b_{11} + a_{12} b_{21} & a_{11} b_{12} + a_{12} b_{22} \\ a_{21} b_{11} + a_{22} b_{21} & a_{21} b_{12} + a_{22} b_{22} \end{matrix})

(\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \end{matrix}) B = (\begin{matrix} a_{11} (b_{11} b_{12}) + a_{12} (b_{21} b_{22}) \\ a_{21} (b_{11} b_{12}) + a_{22} (b_{21} b_{22}) \end{matrix})

(\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \end{matrix}) B = (\begin{matrix} (a_{11} a_{12}) (\begin{matrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{matrix}) \\ (a_{21} a_{22}) (\begin{matrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{matrix}) \end{matrix})

(\begin{matrix} a_{1}^{T} \\ a_{2}^{T} \end{matrix}) B = (\begin{matrix} a_{1}^{T} B \\ a_{2}^{T} B \end{matrix})

So an example of this would be

(\begin{matrix} 0 & 1 & 1 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{matrix}) (\begin{matrix} - 2 & 1 & 1 \\ 1 & - 2 & 1 \\ 1 & 1 & - 2 \end{matrix})

The first row of the left matrix tells us we want $1$ of the second row and $1$ of the third row: $(1, - 2, 1) + (1, 1, - 2) = (2, - 1, - 1)$ . The second row tells us to add up the first and second rows: $(- 2, 1, 1) + (1, - 2, 1) = (- 1, - 1, 2)$ . And the third rows tells us to add the first and third rows: $(- 2, 1, 1) + (1, 1, - 2) = (- 1, 2, - 1)$ . Therefore, the product will be

(\begin{matrix} 0 & 1 & 1 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \end{matrix}) (\begin{matrix} - 2 & 1 & 1 \\ 1 & - 2 & 1 \\ 1 & 1 & - 2 \end{matrix}) = (\begin{matrix} 2 & - 1 & - 1 \\ - 1 & - 1 & 2 \\ - 1 & 2 & - 1 \end{matrix})

Application to row reduction

I use row perspective often when row reducing matrices. It helps me do multiple steps at the same time. So, let’s say I am trying to row reduce

A = (\begin{matrix} 1 & 1 & 3 & 2 \\ - 1 & 0 & - 2 & - 3 \\ 2 & 2 & 6 & 7 \end{matrix})

My thought process is as follows:

First, the second row would be better if its negative was the first row, because then we would have a pivot in the first column and a zero in the second entry. Thus, the first row of my row reduction matrix should be $(0, - 1, 0)$ , since we want $- 1$ of the second row.

Next, if we add up the first two rows, then we get a pivot in the second column and a zero in the first entry. So, the second row of our row reduction matrix will be $(1, 1, 0)$ , since we want to add up the first and second row.

Finally, we can cancel out the first three entries of the third row by adding $- 2$ of row one. The third column should then be $(- 2, 0, 1)$ , since we want to take the third row and subtract $2$ times the first row.

Putting it all together, our row reduction matrix is

R = (\begin{matrix} 0 & - 1 & 0 \\ 1 & 1 & 0 \\ - 2 & 0 & 1 \end{matrix})

And if we multiply $A$ by our row reduction matrix, we get

R A = (\begin{matrix} 0 & - 1 & 0 \\ 1 & 1 & 0 \\ - 2 & 0 & 1 \end{matrix}) (\begin{matrix} 1 & 1 & 3 & 2 \\ - 1 & 0 & - 2 & - 3 \\ 2 & 2 & 6 & 7 \end{matrix})

R A = (\begin{matrix} 1 & 0 & 2 & 3 \\ 0 & 1 & 1 & - 1 \\ 0 & 0 & 0 & 3 \end{matrix})

The final steps of row reduction are then very simple. Divide row three by $3$ , and then add the correct multiples of it to the first two rows to get the final pivot column.

Finding inverses by inspection

We can also use these perspectives to find inverses relatively easily (depending on the matrix). Of course, this is relatively pointless for a $2 \times 2$ since there is the very easy formula. But I try to do this when possible for larger matrices.

Let’s say we want to invert our row reduction matrix from before

R = (\begin{matrix} 0 & - 1 & 0 \\ 1 & 1 & 0 \\ - 2 & 0 & 1 \end{matrix})

This is a good candidate for the perspectives, because there are lots of zeros! The more zeros, the easier it is to use this. If a matrix has no nonzero entries, unless there is some amazingly obvious pattern, I would just use either the adjugate matrix if it’s $3 \times 3$ or row reduction.

So, to do this, we want to find combinations of the rows and columns to get the identity matrix.

The most obvious one to me is that if we take just $1$ of the third column, we get the third column of the identity matrix. So that means the third column of $R^{- 1}$ will be $(0, 0, 1)$

R^{- 1} = (\begin{matrix} * & * & 0 \\ * & * & 0 \\ * & * & 1 \end{matrix})

Also, if we take $- 1$ of the first row, we will get the second row of the identity matrix. So the second row should be $(- 1, 0, 0)$

R^{- 1} = (\begin{matrix} * & * & 0 \\ - 1 & 0 & 0 \\ * & * & 1 \end{matrix})

I can also see that to get the second column of the identity matrix we can take $1$ of column one and $2$ of column three: $(0, 1, - 2) + 2 (0, 0, 1) = (0, 1, 0)$ . So the second column of our inverse will be $(1, 0, 2)$

R^{- 1} = (\begin{matrix} * & 1 & 0 \\ - 1 & 0 & 0 \\ * & 2 & 1 \end{matrix})

Finally, to get the first column, we need a combination of columns one and three to get $(1, 0, 0)$ , given that we have $- 1$ of column two (from the 2,1 entry), $- 1 (- 1, 1, 0)$ . To cancel out the second entry, it looks like we need $1$ of column one:

1 (0, 1, - 2) - 1 (- 1, 1, 0) = (1, 0, - 2)

So, to cancel out that $- 2$ , we can take $2$ of column three. Thus, the first column of the inverse is $(1, - 1, 2)$ .

R^{- 1} = (\begin{matrix} 1 & 1 & 0 \\ - 1 & 0 & 0 \\ 2 & 2 & 1 \end{matrix})

We could have also looked at rows one and three individually! I chose to do the whole column at once because that was my personal preference. You can do it in whatever order you like. I just do whatever is most obvious to me first.